# A Generalized Bridge Regression in Fuzzy Environment and Its Numerical Solution by a Capable Recurrent Neural Network

Authors: Delara Karbasi; Mohammad Reza Rabiei; Alireza Nazemi
Journal: Journal of Mathematics
 (2020)

---

## Abstract

Bridge regression is a special family of penalized regressions using a penalty function∑Ajγ with γ≥1 that for γ=1 and γ=2, it concludes lasso and ridge regression, respectively. In case where the output variable in the regression model was imprecise, we developed a bridge regression model in a fuzzy environment. We also exhibited penalized fuzzy estimates for this model when the input variables were crisp. So, we perform the presented optimization problem for the model that leads to a multiobjective program. Also, we try to determine the shrinkage parameter and the tuning parameter from the same optimization problem. In order to estimate fuzzy coefficients of the proposed model, we introduce a hybrid scheme based on recurrent neural networks. The suggested neural network model is constructed based on some concepts of convex optimization and stability theory which guarantees to find the approximate parameters of the proposed model. We use a simulation study to depict the performance of the proposed bridge technique in the presence of multicollinear data. Furthermore, real data analysis is used to show the performance of the proposed method. A comparison between the fuzzy bridge regression model and several other shrinkage models is made with three different well-known fuzzy criteria. In this study, we visualize the performance of the model by Taylor’s diagram and Bubble plot. Also, we examine the predictive ability of the model, thus, obtained by cross validation. The numerical results clearly showed higher accuracy of the proposed fuzzy bridge method compared to the other existing fuzzy regression models: fuzzy bridge regression model, multiobjective optimization, recurrent neural network, stability convergence, and goodness-of-fit measure.


---

## Body

## 1. Introduction

In regression analysis, researchers often face the problem of multicollinearity defined as the existence of nearly linear dependency between the columns of the design matrix leading to wide confidence intervals for individual parameters or a linear combination of the parameters [1–3]. To overcome the multicollinearity problem, the ridge regression methodology has been widely studied in the literature [1, 4]. The proposed ridge estimator performs better than other estimators that rely on ordinary least-squares. Moreover, when multicollinearity happens, the least-squares methods may lead to poor prediction. In such cases, the ridge regression method often gives better accuracies compared to the least square technique. As an extension, Frank and Friedman [3] introduced bridge regression which includes ridge regression with γ=2 and subset selection with γ=1 as special cases. The authors in [3] did not solve the estimator of bridge regression for any given γ>0; however, they indicated that it is more appropriate to optimize the parameter γ. As a special case of the bridge regression (with γ=1), the authors in [5] introduced the lasso regression model. The authors in [6] extended the lasso method, called fused lasso, in cases where the number of features is much greater than the sample size. Furthermore, the lasso method was modified by Zou [7] who proposed the adaptive lasso with adaptive weights to penalize different coefficients in the l1-penalized. Park and Casella [8] proposed the Bayesian lasso which provides interval estimates that can guide variable selection.Bridge regression has been successfully applied to solve problems of wide-ranging scope and complexity, starting from the pioneering works in [3, 9]. Zou and Yuan [10] imposed the F∞ norm on support vector machines in the classification context. Huang et al. [11] studied the asymptotic properties of bridge regression. The authors in [12] suggested an adaptive elastic net combining the strengths of the quadratic regularization and the adaptively weighted lasso shrinkage. The authors in [13] showed that bridge regression possesses oracle, sparsity, and unbiasedness. In [14], the authors provided a bridge regression that adaptively selects the penalty order from data and produces flexible solutions in various settings. Polson et al. [15] proposed a Bayesian bridge estimator for regularized regression. Mallick and Yi [16] considered Bayesian bridge regression and provided sufficient conditions for strong posterior consistency under a sparsity assumption on a high-dimensional parameter. Finally, in [17], the authors consider a bridge-randomized penalty of regression coefficients by incorporating uncertainty penalty into Bayesian bridge quantile regression.In traditional regression models, the observed values of the dependent and independent variables are supposed to be exact numbers. However, in real-world applications, because of parameter uncertainty, human errors data are usually not exact but have uncertainty. As a result, the classical regression models experience some restriction, which need to be adjusted accordingly to deal with fuzzy data (see [18–32]). As in regression models in traditional statistical theory, multicollinearity is also an important issue in fuzzy regression models. Accordingly, the fuzzy bridge regression model which combines bridge regression with the fuzzy regression model in order to reduce the effect of multicollinearity for the first time in this paper is considered. To solve such a problem, in this paper, the fuzzy bridge regression model is converted to a nonlinear multiobjective problem. The proposed multiobjective problem is solved by the scalarization method making the multiobjective function create a single solution, and the weight is determined before the optimization process (see [33]). A large weight that is determined to an objective function shows that the considered function has a higher priority compared to the ones with a smaller weight. There are several approaches to determining the weight of scalarization, and the one that is used in this paper is introduced in [34]. For a more detailed presentation of the methods treated here, as well as other related topics, refer to [35] and the references therein.The fuzzy regression problems can be solved by applying artificial neural networks and evolutionary algorithms (see [36–45]). For example, Ishibuchi et al. [39] proposed a learning algorithm of fuzzy neural networks with triangular fuzzy weights, and Hayashi et al. [40] fuzzified the delta rule. Neural network solutions were examined to fuzzy problems by Buckley and Eslami in [41]. The authors in [46] extended the risk-neutral model suggested in [38, 47] by adopting a multilayered feed-forward neural network where weights, biases, input, and output variables were assumed to be LR-fuzzy. Furthermore, Coppi et al. [48] studied on FLR by using the least-squares approach with LR-fuzzy numbers. In [49], a class of fuzzy clusterwise regression model which integrates fuzzy cluster analysis into the fuzzy regression model was introduced. The authors in [50] also used triangular fuzzy numbers for fuzzy modeling with a robust framework through the least-squares approach. The topic of the numerical solution of fuzzy polynomials by a fuzzy neural network was investigated in [42]. Mosleh et al. [44, 51, 52] and Otadi [53] provide a hybrid approach based on a feed-forward neural network to approximate fuzzy coefficients of fuzzy linear, nonlinear, and polynomial regression models. Roh et al. [54] concerned an FLR model based on the design approach of polynomial neural networks. The authors in [55, 56] used a random weight network to develop a fuzzy nonlinear regression model. A fuzzified radial basis function network for obtaining estimations of fuzzy regression models is utilized in [57]. For further papers on this topic, see [19, 20, 46, 58–61].The aforesaid references in the main refer to a learning algorithm which is used to adjust the parameters of the feed-forward fuzzy neural network. The although authors in [62] presented a hybrid scheme based on recurrent neural networks, and there is no record for solving the fuzzy bridge regression model by a neural network with stability and convergence properties. Furthermore, to obtain weighting coefficients, the authors use a recurrent procedure where the inverse of a matrix in each iteration needs to be computed explicitly. Obviously, in high-dimensional cases, the computational complexity is extremely high. Besides, the convergence of the proposed neural network methods depends on choosing a suitable initial point. Consequently, suggesting a recurrent neural network to deal with fuzzy bridge regression problems without penalty parameter with rigorous stability and convergence analysis is completely necessary and relevant. Proposing a capable neural network for analog hardware implementation which gives a good solution of the fuzzy bridge regression problem is the main advantage of this work. The optimality conditions of convex programming for solving the related quadratic formulation of the fuzzy bridge regression problem is studied according to the Karush–Kuhn–Tucker (KKT). It is proved that the equilibrium point of the proposed neural network is equivalent to the KKT point of the related quadratic programming formulation. The existence and uniqueness of an equilibrium point of the network are also analyzed. An adequate condition to ensure the existence and global asymptotic stability for the unique equilibrium point of the neural network are obtained by constructing an appropriate Lyapunov function. Additionally, the global convergence of the model is analyzed. The proposed recurrent neural networks in this paper are substantially different from the traditional ones for two important reasons: Firstly, the neural network method has its advantage in dealing with real-time optimization problems which are usually imperative in many applications. However, traditional optimization methods may not be competent due to the problem’s stringent requirement for computational time. As a result, many continuous-time neural networks for solving real-time optimization have been widely expanded (see [63–67]). Secondly, neural networks for solving optimization problems are hardware implementable. In other words, neural networks can be implemented by using integrated circuits.The remainder of this paper is organized as follows. InSection 2, some preliminaries of the fuzzy sets and fuzzy calculus are briefly described. In Section 3, the fuzzy bridge regression problem is introduced. In Section 4, an optimization model to solve the fuzzy bridge linear regression model with LR-fuzzy-dependent variable and nonfuzzy explanatory variables is proposed, and it is transformed into the equivalent quadratic form. In Section 5, based on the (KKT) optimality conditions a neural network model for solving the related quadratic formulation with stability and convergence properties is studied. The goodness-of-fit indices are also studied in Section 6. A numerical experiment and a simulation study to illustrate the effectiveness of the model in the presence of multicollinearity are provided in Section 7. Some concluding remarks are given in Section 8. Proofs of technical statements are stated in Appendix, to better focus on the computational part in the body of paper.


## 2. Preliminaries

In this section, first, we summarize some basic concepts of fuzzy set and fuzzy arithmetic which are used throughout the paper (see [68]); then, some definitions and a basic theorem in multiobjective optimization are briefly discussed.


### 2.1. Fuzzy Set and Fuzzy Arithmetic

A fuzzy setA is denoted by A=x,μAx|x∈X where the function μAx:X⟶0,1, is the degree of membership of element x in A. The h-level set of a fuzzy set A is a crisp subset of X denoted by Ah that is defined by Ah=x∈X|μAx≥h, where h∈0,1.A fuzzy numberA is a normal and convex fuzzy subset of the real line ℝ with bounded support. In a special case, a class of fuzzy numbers, so-called LR-fuzzy numbers, is used as a parametric form:(1)μAx=Lam−xal,x≤am,Rx−amau,x>am,where am is the mode of A and al,ar>0 are the left and right spreads of A, respectively. L,R:ℝ+⟶0,1 are decreasing shape functions S, with S0=1,Sz<1 for all z>0,Sz>0 for all z<1 and S1=0 (or Sz>0 for all z and S+∞=0). As an abbreviated notation, we can define an LR-fuzzy number A with the membership function μAx in (1) by A=am,al,auLR. If Lx and Rx are both linear functions on the domains x|0<Lx,Rx<1, the corresponding LR fuzzy number is a triangular fuzzy number, which is symbolically written as A=am,al,auT. Moreover, if al=au, A becomes a symmetric triangular fuzzy number which is denoted by A=am,alT.Suppose two fuzzy numbersA and B are represented by LR-fuzzy numbers of the form A=am,al,auLR and B=bm,bl,buLR. The sum of A and B is again an LR-fuzzy number of the form(2)A+B=am+bm,al+bl,au+buLR.The scalar multiplicationkA is defined as(3)kA=kam,kal,kauLR,k>0,kam,−kau,−kalRL,k<0.


### 2.2. MultiObjective Optimization

A multiobjective optimization problem (MOP) can be defined by the following problem:(4)minimizeFX,subject to(5)gqX≤0,q=1,…,m,(6)X∈ℝl,where FX=f1X,f2X,…,fkX⊤:X⟶ℝk is a vector with the values of objective functions to be minimized, X is the vector containing the decision variables, defined in the design space ℝl, and giX represents the ith inequality constraint function. Equations (5) and (6) define the region of feasible solutions, S, in the design space ℝl.Definition 1.
(See [69]). A decision vector X∗∈S is Pareto optimal if there does not exist another X∈S such that fiX≤fiX∗ for all i=1,2,…,k and fjX<fjX∗ for, at least, one index j.
Definitions1 introduces global optimality. However, it may be difficult to generate globally optimal solutions if the problem is not convex (that is, if some of the objective functions or the feasible region are not convex). In that case, the solutions obtained may be only locally optimal unless a global solver is available. Note that, under the assumptions mentioned in the problem formulation, the Pareto optimal solutions exist (see [70]).
Classically, multiobjective optimization problems are often solved using scalarization techniques. The scalarization method makes the multiobjective function create a single solution, and the weight is determined before the optimization process. A large weight that is given to an objective function shows that the said function has a higher priority compared to the ones with a smaller weight.
One of the most intuitive scalarization methods used to obtain a single unique solution for MOP is the weighted sum method. In this approach, the MOP (4)–(6) is converted into a scalar preference function using a linear weighted sum function of the form(7)minimize∑s=1kwsfsX,subject to(8)gqX≤0,q=1,…,m,(9)∑s=1kws=1,ws≥0,where X∈ℝl.Theorem 1 (see [69]).
The solution of the MOP (4)–(6) generated by the weighted sum method (problem (7)–(9)) is Pareto optimal if ws>0 for s=1,2,…,k.According to Theorem1, for positive weights and convex problem, the optimal solutions of the substitute problem (7)–(9) are Pareto optimal.


## 2.1. Fuzzy Set and Fuzzy Arithmetic

A fuzzy setA is denoted by A=x,μAx|x∈X where the function μAx:X⟶0,1, is the degree of membership of element x in A. The h-level set of a fuzzy set A is a crisp subset of X denoted by Ah that is defined by Ah=x∈X|μAx≥h, where h∈0,1.A fuzzy numberA is a normal and convex fuzzy subset of the real line ℝ with bounded support. In a special case, a class of fuzzy numbers, so-called LR-fuzzy numbers, is used as a parametric form:(1)μAx=Lam−xal,x≤am,Rx−amau,x>am,where am is the mode of A and al,ar>0 are the left and right spreads of A, respectively. L,R:ℝ+⟶0,1 are decreasing shape functions S, with S0=1,Sz<1 for all z>0,Sz>0 for all z<1 and S1=0 (or Sz>0 for all z and S+∞=0). As an abbreviated notation, we can define an LR-fuzzy number A with the membership function μAx in (1) by A=am,al,auLR. If Lx and Rx are both linear functions on the domains x|0<Lx,Rx<1, the corresponding LR fuzzy number is a triangular fuzzy number, which is symbolically written as A=am,al,auT. Moreover, if al=au, A becomes a symmetric triangular fuzzy number which is denoted by A=am,alT.Suppose two fuzzy numbersA and B are represented by LR-fuzzy numbers of the form A=am,al,auLR and B=bm,bl,buLR. The sum of A and B is again an LR-fuzzy number of the form(2)A+B=am+bm,al+bl,au+buLR.The scalar multiplicationkA is defined as(3)kA=kam,kal,kauLR,k>0,kam,−kau,−kalRL,k<0.


## 2.2. MultiObjective Optimization

A multiobjective optimization problem (MOP) can be defined by the following problem:(4)minimizeFX,subject to(5)gqX≤0,q=1,…,m,(6)X∈ℝl,where FX=f1X,f2X,…,fkX⊤:X⟶ℝk is a vector with the values of objective functions to be minimized, X is the vector containing the decision variables, defined in the design space ℝl, and giX represents the ith inequality constraint function. Equations (5) and (6) define the region of feasible solutions, S, in the design space ℝl.Definition 1.
(See [69]). A decision vector X∗∈S is Pareto optimal if there does not exist another X∈S such that fiX≤fiX∗ for all i=1,2,…,k and fjX<fjX∗ for, at least, one index j.
Definitions1 introduces global optimality. However, it may be difficult to generate globally optimal solutions if the problem is not convex (that is, if some of the objective functions or the feasible region are not convex). In that case, the solutions obtained may be only locally optimal unless a global solver is available. Note that, under the assumptions mentioned in the problem formulation, the Pareto optimal solutions exist (see [70]).
Classically, multiobjective optimization problems are often solved using scalarization techniques. The scalarization method makes the multiobjective function create a single solution, and the weight is determined before the optimization process. A large weight that is given to an objective function shows that the said function has a higher priority compared to the ones with a smaller weight.
One of the most intuitive scalarization methods used to obtain a single unique solution for MOP is the weighted sum method. In this approach, the MOP (4)–(6) is converted into a scalar preference function using a linear weighted sum function of the form(7)minimize∑s=1kwsfsX,subject to(8)gqX≤0,q=1,…,m,(9)∑s=1kws=1,ws≥0,where X∈ℝl.Theorem 1 (see [69]).
The solution of the MOP (4)–(6) generated by the weighted sum method (problem (7)–(9)) is Pareto optimal if ws>0 for s=1,2,…,k.According to Theorem1, for positive weights and convex problem, the optimal solutions of the substitute problem (7)–(9) are Pareto optimal.


## 3. Fuzzy Bridge Regression Model

Consider a set of dataxi0,…,xip,Yi|i=1,…,n, in which xiji=1,…,nandj=0,1,…,p is the value of jth independent variable and Yii=1,…,n is the corresponding value of dependent variable Y in the ith case. The purpose of the linear regression model is to fit a linear model to the given data. This model can be considered as follows:(10)Y^i=A0+A1xi1+⋯+Apxip,i=1,…,n,where A0,A1,…,Ap denote the list of regression coefficients (parameters). In some cases, we may need to consider that the relationship expressed in (10) may be fuzzy, and then, the following different cases are obtained:(a)
The case where the predictor variable is fuzzy, but the parameters are crisp(b)
The case of a crisp predictor and fuzzy parameters(c)
The case of a fuzzy predictor and fuzzy parametersIt should be noted that the observed responses in all cases are naturally fuzzy. As in LR models in traditional statistical theory, in most of the empirical works, we often encounter the multicollinearity phenomenon in FLR models, as a result of highly intercorrelated explanatory variables. In the last two decades, different shrinkage methods have been proposed for estimating regression coefficients; among them, the bridge shrinkage method has received considerable attention. The authors in [3] proposed the following optimization problem to solve the bridge regression for any given γ≥0:(11)minimize∑i=1nYi∑j=0pAjxij2,subject to(12)∑j=0pAjγ≤t,where the positive parameter t represents the tuning constant. Also, the optimal γ selection will increase the efficiency of the model. It is shown that for γ≥1, the bridge regression model is a convex problem (see [9]). Moreover, by setting γ=0, γ=1 and γ=2 in optimization problem (11) and (12), the least-squares regression model, the lasso regression model, and the ridge regression model are, respectively, obtained.In the following section, in order to develop the bridge regression model for fuzzy numbers, the aforementioned dataset (b) is considered. Then, a possible approach for solving the problem connected to multicollinearity in an FLR with LR-fuzzy numbers is discussed.


### 3.1. Proposed Model

Under the assumption of model (10), suppose Aj=ajm,ajl,ajuLR and Yi=yim,yil,yiuLR, i=1,…,nandj=0,1,…,p. Let Iij be the identification function which only equals to 1 when xij≥0. Using (2) and (3), the LR-fuzzy number Y^i is defined as(13)Y^i=∑j=0pajmxij,∑j=0pIijajlxij−1−Iijajuxij,∑j=0pIijajuxij−1−IijajlxijLR,,fori=1,…,nandj=0,1,…,p.Now, the proposed MOP for solving the fuzzy bridge regression model consists of three objective functions via the least squares method on the center, and the spreads is presented as follows:(14)minimize∑i=1nyim−∑j=0pajmxij2,(15)minimize∑i=1nyil−∑j=0pIijajlxij−1−Iijajuxij2,(16)minimize∑i=1nyiu−∑j=0pIijajuxij−1−Iijajlxij2,subject to(17)∑j=0pajmγ≤t1,(18)∑j=0pajlγ≤t2,ajl≥0,(19)∑j=0pajuγ≤t3,aju≥0,where t1,t2, and t3 represent three tuning constants. It should be pointed out that the use of different tss=1,2,3 will lead to better results rather than constraining them to be the same throughout the computations [71].


## 3.1. Proposed Model

Under the assumption of model (10), suppose Aj=ajm,ajl,ajuLR and Yi=yim,yil,yiuLR, i=1,…,nandj=0,1,…,p. Let Iij be the identification function which only equals to 1 when xij≥0. Using (2) and (3), the LR-fuzzy number Y^i is defined as(13)Y^i=∑j=0pajmxij,∑j=0pIijajlxij−1−Iijajuxij,∑j=0pIijajuxij−1−IijajlxijLR,,fori=1,…,nandj=0,1,…,p.Now, the proposed MOP for solving the fuzzy bridge regression model consists of three objective functions via the least squares method on the center, and the spreads is presented as follows:(14)minimize∑i=1nyim−∑j=0pajmxij2,(15)minimize∑i=1nyil−∑j=0pIijajlxij−1−Iijajuxij2,(16)minimize∑i=1nyiu−∑j=0pIijajuxij−1−Iijajlxij2,subject to(17)∑j=0pajmγ≤t1,(18)∑j=0pajlγ≤t2,ajl≥0,(19)∑j=0pajuγ≤t3,aju≥0,where t1,t2, and t3 represent three tuning constants. It should be pointed out that the use of different tss=1,2,3 will lead to better results rather than constraining them to be the same throughout the computations [71].


## 4. An Optimization Model

In order to obtain the estimate for coefficientsA0,A1,…,Ap of the fuzzy bridge regression model, it is necessary to minimize objective functions (14)–(16) simultaneously under constraints (17)–(19). A scalarization problem related to (14)–(19) is stated as(20)minimizeCA=w1∑i=1nyim−∑j=0pajmxij2+w2∑i=1nyil−∑j=0pIijajlxij−1−Iijajuxij2+w3∑i=1nyiu−∑j=0pIijajuxij−1−Iijajlxij2,subject to(21)∑j=0pajmγ≤t1,(22)∑j=0pajlγ≤t2,ajl≥0,(23)∑j=0pajuγ≤t3,aju≥0,where(24)A=a0m,…,apm,a0l,…,apl,a0u,…,apu⊤,is a 3p+3-dimensional column vector, ws≥0,s=1,2,3, and ∑s=13ws=1. To solve problem (20)–(23) we first use the following proposition [72]. Let u,v∈ℝn be auxiliary variables such that(25)x=u−v,u≥0,v≥0,where ui=xi+,vi=−xi+, and .+ denotes the positive part defined as x+=max0,x. Therefore, in order to model the absolute values in optimization problem (20)–(23), ajm should be split into positive and negative parts such that ajm=ajm′−ajm″ with ajm′,ajm″≥0. Thus, the following quadratic optimization problems for γ≥1 are proposed:(26)minimizeCbridgeA¯=A¯⊤Q¯A¯+B¯⊤A¯+M,subject to(27)g¯qA¯≤0,q=1,2,3,(28)G¯A¯≤0,where(29)=a0m′,…,apm′,a0m″,…,apm″,a0l,…,apl,a0u,…,apu⊤,is a 4p+4-dimensional column vector, ws≥0,s=1,2,3, and ∑s=13ws=1. Here,(30)Q¯=w1P1−2w1P1OOOw1P1OOOOw3P3+w2P2OOOOw2P3+w3P2,with(31)P1=∑i=1nxi0∑i=1nxi1…∑i=1nxip∑i=1nxi1∑i=1nxi12…∑i=1nxi1xip⋮⋮⋮⋮∑i=1nxip∑i=1nxipxi1…∑i=1nxip2,(32)P2=∑i=1nIi0xi0∑i=1nIi1xi1…∑i=1nIipxip∑i=1nIi1xi1∑i=1nIi1xi12…∑i=1nIi1xi1Iipxip⋮⋮⋮⋮∑i=1nIipxip∑i=1nIipxipIi1xi1…∑i=1nIipxip2,(33)P3=∑i=1n1−Ii0xi0∑i=1n1−Ii1xi1…∑i=1n1−Iipxip∑i=1n1−Ii1xi1∑i=1n1−Ii1xi12…∑i=1n1−Ii1xi11−Iipxip⋮⋮⋮⋮∑i=1n1−Iipxip∑i=1n1−Iipxip1−Ii1xi1…∑i=1n1−Iipxip2,where O is a p+1×p+1 zeros matrix. G¯ is a 4p+4×4p+4 diagonal matrix with diagonal element −1, and B¯ is a column vector:(34)B¯=b10′,b11′,…,b1p′,b10″,…,b1p″,b20,…,b2p,b30,…,b3p⊤,where(35)b1j′=−2w1∑i=1nxijyim,b1j′′=2w1∑i=1nxijyim,b2j=−2w2∑i=1nIijxijyil+2w3∑i=1n1−Iijxijyiu,b3j=−2w3∑i=1nIijxijyiu+2w2∑i=1n1−Iijxijyil.Also,(36)g¯1=∑j=0pajm′−ajm″γ−t1,g¯2=∑j=0pajlγ−t2,g¯3=∑j=0pajuγ−t3,(37)M=w1∑i=1nyim2+w2∑i=1nyil2+w3∑i=1nyiu2.It can be easily verified that the Hessian matrix ofCbridgeA¯ is positive definite matrix because(38)∇2CbridgeA¯=Q¯.Then, problems (26)–(28) are convex quadratic problems with inequality constraints.


### 4.1. Fuzzy Lasso Regression and Fuzzy Ridge Regression

In this section, to achieve the two well-known forms of the fuzzy bridge regression model, letγ=1 and γ=2. Thus, the following quadratic optimization problems are, respectively, obtained as(39)minimizeClassoA¯=A¯⊤Q¯A¯+B¯⊤A¯+M,subject to(40)g¯qA¯≤0,q=1,2,3,(41)G¯A¯≤0,(42)minimizeCridgeA=A⊤QA+B⊤A+M,subject to(43)gqA≤0,q=1,2,3,(44)GA≤0,where Q is the following diagonal block matrix:(45)Q=w1P1OOOw3P3+w2P2OOOw2P3+w3P2,where G is a 3p+3×3p+3-dimensional matrix defined as(46)G=0⋯00⋯00⋯0⋮⋱⋮⋮⋱⋮⋮⋱⋮0⋯00⋯00⋯00⋯0−1⋯00⋯0⋮⋱⋮⋮⋱⋮⋮⋱⋮0⋯00⋯−10⋯00⋯00⋯0−1⋯0⋮⋱⋮⋮⋱⋮⋮⋱⋮0⋯00⋯00⋯−1,O is a p+1×p+1 zeros matrix, and G¯ is a 4p+4×4p+4-diagonal matrix with diagonal element −1. B is a column vector:(47)B=b10,b11,…,b1p,b20,…,b2p,b30,…,b3p⊤,with(48)b1j=−2w1∑i=1nxijyim,b2j=−2w2∑i=1nIijxijyil+2w3∑i=1n1−Iijxijyiu,b3j=−2w3∑i=1nIijxijyiu+2w2∑i=1n1−Iijxijyil.Also,(49)g¯1=∑j=0pajm′−ajm″−t1,g¯2=∑j=0pajl−t2,g¯3=∑j=0paju−t3,g1=∑j=0pajm2−t1,g2=∑j=0pajl2−t2,g3=∑j=0paju2−t3,where A¯, A, Q¯, P1, P2, P3, B¯, and M are the same as defined in (24), (29)–(34), and (37), respectively.It can be easily verified that the Hessian matrix ofClassoA¯ and CridgeA is a positive definite matrix because(50)∇2ClassoA¯=Q¯,∇2CridgeA=Q.Then, problems (39)–(41) and (42)–(44) are convex quadratic problems with inequality constraints.There are several optimization methods for solving the convex quadratic problems with inequality constraints that each one of these methods has its weaknesses and strengths. FollowingSection 5, we proposed a neural network that can approximate coefficients A0,A1,…,Ap of the fuzzy bridge optimization model.


## 4.1. Fuzzy Lasso Regression and Fuzzy Ridge Regression

In this section, to achieve the two well-known forms of the fuzzy bridge regression model, letγ=1 and γ=2. Thus, the following quadratic optimization problems are, respectively, obtained as(39)minimizeClassoA¯=A¯⊤Q¯A¯+B¯⊤A¯+M,subject to(40)g¯qA¯≤0,q=1,2,3,(41)G¯A¯≤0,(42)minimizeCridgeA=A⊤QA+B⊤A+M,subject to(43)gqA≤0,q=1,2,3,(44)GA≤0,where Q is the following diagonal block matrix:(45)Q=w1P1OOOw3P3+w2P2OOOw2P3+w3P2,where G is a 3p+3×3p+3-dimensional matrix defined as(46)G=0⋯00⋯00⋯0⋮⋱⋮⋮⋱⋮⋮⋱⋮0⋯00⋯00⋯00⋯0−1⋯00⋯0⋮⋱⋮⋮⋱⋮⋮⋱⋮0⋯00⋯−10⋯00⋯00⋯0−1⋯0⋮⋱⋮⋮⋱⋮⋮⋱⋮0⋯00⋯00⋯−1,O is a p+1×p+1 zeros matrix, and G¯ is a 4p+4×4p+4-diagonal matrix with diagonal element −1. B is a column vector:(47)B=b10,b11,…,b1p,b20,…,b2p,b30,…,b3p⊤,with(48)b1j=−2w1∑i=1nxijyim,b2j=−2w2∑i=1nIijxijyil+2w3∑i=1n1−Iijxijyiu,b3j=−2w3∑i=1nIijxijyiu+2w2∑i=1n1−Iijxijyil.Also,(49)g¯1=∑j=0pajm′−ajm″−t1,g¯2=∑j=0pajl−t2,g¯3=∑j=0paju−t3,g1=∑j=0pajm2−t1,g2=∑j=0pajl2−t2,g3=∑j=0paju2−t3,where A¯, A, Q¯, P1, P2, P3, B¯, and M are the same as defined in (24), (29)–(34), and (37), respectively.It can be easily verified that the Hessian matrix ofClassoA¯ and CridgeA is a positive definite matrix because(50)∇2ClassoA¯=Q¯,∇2CridgeA=Q.Then, problems (39)–(41) and (42)–(44) are convex quadratic problems with inequality constraints.There are several optimization methods for solving the convex quadratic problems with inequality constraints that each one of these methods has its weaknesses and strengths. FollowingSection 5, we proposed a neural network that can approximate coefficients A0,A1,…,Ap of the fuzzy bridge optimization model.


## 5. A Neural Network Scheme

In this section, using standard optimization techniques, we transform quadratic programming problem (26)–(28) into the equivalent nonlinear dynamical system model. From [73], A¯∗∈ℝ4p+4 is an optimal solution of (26)–(28) if and only if there exists U¯∗∈ℝ4p+4 such that A¯∗⊤,U¯∗⊤⊤ satisfies the following KKT system:(51)U¯∗≥0,G¯A¯∗≤0,g¯qA¯∗≤0,q=1,2,3,U¯∗⊤G¯A¯∗=0,U¯∗⊤g¯qA¯∗=0,q=1,2,3,2Q¯A¯∗+B¯+G¯⊤U¯∗+g¯q⊤A¯∗U¯∗=0,q=1,2,3,where A¯∗ is called a KKT point of (26)–(28) and U¯∗ is called the Lagrangian multiplier vector corresponding to A¯∗.Theorem 2.
(see [73]). A¯∗ is an optimal solution of (26)–(28), if and only if A¯∗ is a KKT point of (24)–(26).Lemma 1.
U¯+G¯A¯+=U¯ if and only if U¯≥0,G¯A¯≤0,U¯⊤G¯A¯=0, where(52)U¯+G¯A¯+=U¯+G¯A¯1+,U¯+G¯A¯2+,…,U¯+G¯A¯4p+4+⊤,U¯+G¯A¯k+=maxU¯+G¯A¯k,0,k=1,2,…,4p+4.Now, letA¯. and U¯. be some time-dependent variables. The aim is to construct a continuous-time dynamical system that will settle down to the KKT point of problem (26)–(28). We propose a neural network as(53)dydt=τϕy,(54)yt0=y0=A¯t0⊤,U¯t0⊤⊤∈ℝ8p+8,τ>0,where for q=1,2,3,(55)ϕy=−2Q¯A¯+B¯+G¯⊤U¯+G¯A¯++∇g¯qA¯⊤U¯+g¯qA¯+,U¯+G¯A¯+−U¯,U¯+g¯qA¯+−U¯,,and τ is a tuning parameter. For simplicity of the analysis, let τ=1.Theorem 3.
Lety∗=A¯∗⊤,U¯∗⊤⊤ be the equilibrium point of neural network (53) and (54). Then, A∗ is a KKT point of problem (26)–(28). On the other hand, if A¯∗∈ℝ4p+4 is an optimal solution of problem (26)–(28), then there exists U¯∗∈ℝ4p+4 such that y∗=A¯∗⊤,U¯∗⊤⊤ is an equilibrium point of network (53) and (54).In the following, it is proved that the proposed neural network (53) and (54) is stable in the sense of Lyapunov.Lemma 2.
LetG¯∈ℝ4p+4×4p+4 be of full rank. The Jacobian matrix ∇ϕy of the mapping ϕ defined in (53) is a negative semidefinite matrix.Lemma 3.
The functionU¯+G¯A¯+2 of U¯+G¯A¯+ defined in (53) is convex and continuously differentiable on ℝ4p+4×ℝ4p+4.Theorem 4.
Let the assumption of Lemma2 be satisfied. Then, the neural network in (53) and (54) is stable in the sense of Lyapunov.To show the global convergence of the model, the following lemma and theorem are considered.Lemma 4.
(i)
For any initial pointyt0=A¯t0⊤,U¯t0⊤⊤, there exists a unique continuous solution yt=A¯t⊤,U¯t⊤⊤ for system (53) and (54).(ii)
Letyt=A¯t⊤,U¯t⊤⊤ be the state trajectory of (53) and (54) with the initial point yt0=A¯t0⊤,U¯t0⊤⊤. If U¯t0≥0, then U¯t≥0.Theorem 5.
The state trajectory of neural network (53) and (54) converges to an equilibrium point for any initial point y0=A¯t0⊤,U¯t0⊤⊤∈ℝ8p+8. In particular, neural network (53) and (54) with any initial point y0=A¯t0⊤,U¯t0⊤⊤∈ℝ8p+8 is globally asymptotically stable when D∗ (the optimal point set of the original problem and its dual) has a unique equilibrium point.Theorem 6.
The convergence rate of the neural network in (53) and (54) increases as τ increases.


## 6. The Goodness of Fit

In this section, before focusing on the measurement of goodness of fit for a bridge fuzzy regression model, some popularly used measures are introduced to define the distance between fuzzy numbers. Then, some goodness-of-fit indices are employed, which are represented in [74–77].Definition 2.
Xu [78] presented the following distance between two fuzzy numbers A and B as(56)D1A,B=∫01fhd2Ah,Bhdh1/2,where(57)d2Ah,Bh=ALh−BLh2+AUh−BUh2,Ah=ALh,AUh,Bh=BLh,BUh,and fh is an increasing function on 0,1 with f0=0 and ∫01fhdh=1/2.
The advantage of this distance is that it can let differenth-level cuts play different parts and can be applied to general fuzzy number space.Definition 3.
authors in [48] defined the distance between two LR-fuzzy numbers A and B as follows:(58)D22A,B=am−bm2+am−γal−bm−γbl2+am+ρau−bm+ρbu2,where parameters γ=∫01L−1ωdω and ρ=∫01R−1ωdω have the twofold role of taking into account the variability of the membership function and decreasing the emphasis on the spreads (see [50]).
Hassanpour et al. [79] defined the distance between two triangular fuzzy numbers as follows.Definition 4.
For two triangular fuzzy numbersA and B, D3 is called the Hassanpour distance defined by(59)D3A,B=am−bm+al−bl+au−bu.
The most important topic in fuzzy regression models is the ability to describe a model. Accordingly, the following goodness-of-fit indices are used in this paper to evaluate the goodness of fit of the proposed model.


### 6.1. Mean Square of Prediction Error

One of the indicators of goodness-of-fit assessment in the statistical regression model is the mean square of prediction error (MSE) between the predicted and the observed values. In the following, this criterion is presented as a definition.Definition 5.
For regression model (10), the MSE between the estimated and the observed values is defined by(60)MSE=1n∑i=1nDj2Y^i,Yi,j=1,2,3,where DjY^i,Yi is the distance between two fuzzy numbers Yi^ and Yi (see Definitions 2– 4). Smaller values of MSE indicate that the model fits better the data (see [74]).


### 6.2. Similarity Measure

Several similarity measures are introduced in the literature to measure the similarity between fuzzy sets. In this work, we use the index that has certain advantages over other similarity measures. Such a similarity measure has been used for evaluating the performance of a fuzzy regression model in [75, 76].Definition 6.
Consider fuzzy regression model (10), with fuzzy observed and fuzzy estimated output. The mean of similarity measures (MSM) is defined as(61)MSM=1n∑i=1nSUIY^i,Yi,where(62)SUIY^i,Yi=CardY^i∩YiCardY^i∪Yi,and CardA=∫xμAxdx where A is a fuzzy number.
The similarity measure ranges from 0 to 1, and the model with a higher value of MSM provides a better fit to the data.


### 6.3. Cross Validation

To further investigate the performance of the model obtained inSection 4, we apply another index based on the cross-validation method to examine the predictive ability of the model (see [80]). To this end, each time, the ith observation is left out from the dataset, while the remaining observations are used to develop a fuzzy regression model. Then, the obtained model is used to predict the response value of the ith observation Y^−ixi. Finally, to compare the ith observed Yi and the predicted value Y^−ixi, we calculate the distance D between Yi and Y^−ixi which is called MDC.Definition 7.
For regression model (10), the index of MDC is defined by(63)MDC=1n∑i=1nDj2Y^−ixi,Yi,j=1,2,3,where Y^−ixi is the fuzzy response prediction by omitting the ith pair xi,Yi and DjY^i,Yi is defined in Definitions 2– 4.
To compare performances of different models, the following well-known diagrams can be used to summarize the power of the proposed model to represent the actual values graphically.


### 6.4. Taylor’s Diagrams

Further to the common performance evaluation criteria, Taylor’s diagrams are depicted to convey information about pattern similarity between estimates and data. This similarity is quantified in terms of the correlation between estimates and data, their centered Root Mean Square (RMS) differences, and their standard deviations. This diagram is suitable for evaluating multiple aspects of complex models. In addition, it is possible to compare the relative performance among different models. The observed dataset is represented by a point on thex-axis as “Actual data.” The estimates are positioned according to three statistics: the radial distance of the point labelled “actual data” from the origin of the plot that is proportional to the standard deviation of each dataset; the RMS difference between data and estimates that is proportional to their distance; and their correlation is given by the azimuthal position of the estimates, i.e. the angle from the x-axis. The formulas for calculating the centered Root Mean Square Error (RMS), the correlation (R), the standard deviation of the observed data σY^, and the standard deviation of the estimated data σY are given as follows:(64)R=1/n∑i=1nY^i−Y^¯Yi−Y^¯σY^σY,RMS=1n∑i=1nY^i−Yi2,σY^=1n∑i=1nY^i−Y^¯2,σY=1n∑i=1nYi−Y¯2,where Y^¯ and Y¯ are the calculated mean values of Y^i and Yi, respectively. In Taylor’s diagrams, when the distance to the point representing the actual data is relatively short, there is a good agreement between the estimated and observed data.


### 6.5. Fuzzy Bubble Plots

Coppi et al. [48] made use of bubble plots to evaluate the fit of the fuzzy regression model with fuzzy output and crisp inputs. The x-axis and the y-axis denote, respectively, the observed and estimated centers, the circles are centered in yim,y^im, and their diameters are given by yil−y^il+yiu−y^iu1/2. In bubble plots, the closeness of these circles to the bisector of the plane shows that the model correctly estimates the centers of the fuzzy data, and the smaller diameters of the circles indicate that the model correctly estimates the spreads of the fuzzy data.


## 6.1. Mean Square of Prediction Error

One of the indicators of goodness-of-fit assessment in the statistical regression model is the mean square of prediction error (MSE) between the predicted and the observed values. In the following, this criterion is presented as a definition.Definition 5.
For regression model (10), the MSE between the estimated and the observed values is defined by(60)MSE=1n∑i=1nDj2Y^i,Yi,j=1,2,3,where DjY^i,Yi is the distance between two fuzzy numbers Yi^ and Yi (see Definitions 2– 4). Smaller values of MSE indicate that the model fits better the data (see [74]).


## 6.2. Similarity Measure

Several similarity measures are introduced in the literature to measure the similarity between fuzzy sets. In this work, we use the index that has certain advantages over other similarity measures. Such a similarity measure has been used for evaluating the performance of a fuzzy regression model in [75, 76].Definition 6.
Consider fuzzy regression model (10), with fuzzy observed and fuzzy estimated output. The mean of similarity measures (MSM) is defined as(61)MSM=1n∑i=1nSUIY^i,Yi,where(62)SUIY^i,Yi=CardY^i∩YiCardY^i∪Yi,and CardA=∫xμAxdx where A is a fuzzy number.
The similarity measure ranges from 0 to 1, and the model with a higher value of MSM provides a better fit to the data.


## 6.3. Cross Validation

To further investigate the performance of the model obtained inSection 4, we apply another index based on the cross-validation method to examine the predictive ability of the model (see [80]). To this end, each time, the ith observation is left out from the dataset, while the remaining observations are used to develop a fuzzy regression model. Then, the obtained model is used to predict the response value of the ith observation Y^−ixi. Finally, to compare the ith observed Yi and the predicted value Y^−ixi, we calculate the distance D between Yi and Y^−ixi which is called MDC.Definition 7.
For regression model (10), the index of MDC is defined by(63)MDC=1n∑i=1nDj2Y^−ixi,Yi,j=1,2,3,where Y^−ixi is the fuzzy response prediction by omitting the ith pair xi,Yi and DjY^i,Yi is defined in Definitions 2– 4.
To compare performances of different models, the following well-known diagrams can be used to summarize the power of the proposed model to represent the actual values graphically.


## 6.4. Taylor’s Diagrams

Further to the common performance evaluation criteria, Taylor’s diagrams are depicted to convey information about pattern similarity between estimates and data. This similarity is quantified in terms of the correlation between estimates and data, their centered Root Mean Square (RMS) differences, and their standard deviations. This diagram is suitable for evaluating multiple aspects of complex models. In addition, it is possible to compare the relative performance among different models. The observed dataset is represented by a point on thex-axis as “Actual data.” The estimates are positioned according to three statistics: the radial distance of the point labelled “actual data” from the origin of the plot that is proportional to the standard deviation of each dataset; the RMS difference between data and estimates that is proportional to their distance; and their correlation is given by the azimuthal position of the estimates, i.e. the angle from the x-axis. The formulas for calculating the centered Root Mean Square Error (RMS), the correlation (R), the standard deviation of the observed data σY^, and the standard deviation of the estimated data σY are given as follows:(64)R=1/n∑i=1nY^i−Y^¯Yi−Y^¯σY^σY,RMS=1n∑i=1nY^i−Yi2,σY^=1n∑i=1nY^i−Y^¯2,σY=1n∑i=1nYi−Y¯2,where Y^¯ and Y¯ are the calculated mean values of Y^i and Yi, respectively. In Taylor’s diagrams, when the distance to the point representing the actual data is relatively short, there is a good agreement between the estimated and observed data.


## 6.5. Fuzzy Bubble Plots

Coppi et al. [48] made use of bubble plots to evaluate the fit of the fuzzy regression model with fuzzy output and crisp inputs. The x-axis and the y-axis denote, respectively, the observed and estimated centers, the circles are centered in yim,y^im, and their diameters are given by yil−y^il+yiu−y^iu1/2. In bubble plots, the closeness of these circles to the bisector of the plane shows that the model correctly estimates the centers of the fuzzy data, and the smaller diameters of the circles indicate that the model correctly estimates the spreads of the fuzzy data.


## 7. Numerical Examples

In order to demonstrate the effectiveness of the proposed scheme, in this section, we test some applicable examples by neural network (51) and (52). For testing problems, we also compare the numerical performance of neural network (53) and (54) with various initial points. To illustrate further the effectiveness of the proposed neural network, we make some comparisons with the existing models in the work of Hesamian and Akbari [71], Taheri and Kelkinnama [76], Zeng et al. [81], Choi and Yoon [82], Choi and Buckley [83], and Kula and Apaydin [84] with different distances explained in Section 6 applied to calculate the goodness-of-fit criteria. It must be noted that each method claimed to exhibit better performance compared to other fuzzy multiple regression models. In addition, we present Taylor’s diagrams [77] which provide information on multiple statistical indices to evaluate the accuracy of neural network (53) and (54). The simulation is conducted on Matlab R2017b, and the ordinary differential equation solver engaged is ode45s.Example 1.
(A Simulation Study). To further illustrate the performance of the proposed procedure, we generated a30 simulated dataset of size n=20 with fuzzy output and crisp inputs. Consider the following linear regression model:(65)Y^i=A0+A1xi1+⋯+A25xi25+εi,where for i=1,2,…,20,(66)xij=z1+e1,forj=1,2,…,5,z2+e2,forj=6,7,…,10,z3+e3,forj=11,12,…,15,z4,forj=16,12,…,25,Aj=0,0,0T,forj=0,1.5,0.5,0.35T,forj=1,2,…,5,0,0.5,0.35T,forj=6,7,…,10,0,0.35,0.5T,forj=11,12,…,15,0,0,0.5T,forj=16,17,…,20,0,0,0T,forj=21,22,…,25,(67)εi=εim,εil,εiuT,εil=εiu=εim,with z1∼N0,0.5, z2∼N0,0.8, z3∼N0,1.2, z4=0,Σ, e1∼N0,0.01, e2∼N0,0.02, e3∼N0,0.03, εim=N0,1.5, and(68)Σlk=Covxjl,xjk=0.5l−k.
Using the proposed methods for every30 simulated dataset, the mean of all the employed goodness-of-fit measures (MSM¯, MSE¯, and MDC¯) with respect to the distances D1, D2, and D3 is shown in Table 1.
To compare the performance of the proposed models with the existing models in [71, 76, 81, 83], the estimated errors of the observed responses are calculated based on all 30 simulated datasets of size n=20. The numerical results are summarized in Table 1. It can be seen that the proposed fuzzy bridge regression model with different values of γ provides more accurate results.
We utilize the proposed neural network method to the15th simulated dataset. According to Section 3, the optimal selection of γ increases the efficiency of the model. Thus, to estimate the best value of Y^ using (26)–(28), we need to find the optimal bridge parameter γ. Table 2 shows the corresponding values of MSM, MSE, and MDC for the proposed fuzzy bridge regression model with different values of γ.
The total errors of the proposed fuzzy bridge regression model withγ=3.7 are obviously better than the total errors calculated from the methods with γ=1,2, and 3 (see Figure 1).
In addition, Table3 shows the estimated fuzzy coefficients of the fuzzy bridge regression model for γ=3.7, t1=1669,t2=41, and t3=38.
Figures2–4 show that the trajectories of neural network model (53) and (54) for solving the fuzzy bridge regression model γ=3.7 are convergent to the optimal solution of the problem. Simulation results show the trajectories of (53) and (54) with any initial point are always convergent to the optimal solution of the problem. Taking A24 for example, Figure 5 displays the transient behaviors of the fuzzy bridge regression model based on (53) and (54) with 20 random initial points.
The model functioning in the graphical state is illustrated in Figures6–8, through Taylor’s diagrams.
Figure6 shows proposed neural network (53) and (54) for the bridge regression model of the centers with γ=1, γ=2, γ=3 and γ=3.7, depicted by brown, green, black, and magenta circle, respectively. It is clear that the proposed model with γ=3.7 lies very close to the actual dataset. This result holds also for the left and right bounds.
In addition, to visualize how the models fit the observed fuzzy data, fuzzy bubble plots are used (see Figure9). The fuzzy bubble plots show that observations fit well to the proposed models because of the closeness of the circles to the bisector of the plane and the short radius of each circle.
Therefore, it is concluded that the proposed model used in this paper has certain merits in practice.Table 1
Mean of 30 simulated performance measures of the proposed fuzzy bridge regression model with different values ofγ and some common fuzzy regression techniques in Example 1.
Performance measuresMSM¯MSE¯MDC¯D1D2D3D1D2D3Proposed fuzzy lassoγ=10.9200.0390.1460.7341.3493.0463.184Proposed fuzzy bridgeγ=1.509220.0390.1440.7831.3183.1483.184Proposed fuzzy ridgeγ=20.9110.0370.1560.8451.3293.1363.189Proposed fuzzy bridgeγ=309220.0390.1530.7821.3183.1483.184The model in [71]0.8050.2043.0091.2251.5223.3443.976The model in [76]0.6070.5045.2332.1303.6615.9125.059The model in [81]0.6540.5925.9972.3632.3255.7344.370The model in [83]0.7210.4844.7681.7602.9224.7404.446Table 2
Performance measures of the proposed fuzzy bridge regression model to the15th simulated dataset with different values of γ in Example 1.
Fuzzy bridge regression modelMSMMSEMDCγD1D2D3D1D2D310.939280.043100.129290.799211.349013.111753.1897020.938210.043180.129520.799211.329123.137483.1897430.939210.043110.129520.799211.318823.111003.188923.70.939400.042150.126790.798621.318553.110853.18892Figure 1
Goodness-of-fit measures for different values ofγ with three different distances D1,D2, and D3 in Example 1. (a) MSM index, (b) MSE index with distance D1, (c) MSE index with distance D2, and (d) MSE index with distance D3.
(a)(b)(c)(d)Table 3
Coefficients of fuzzy bridge regression models withγ=3.7, corresponding to the 15th simulated dataset in Example 1.
FuzzyProposed fuzzy bridgeCoefficientRegression modelγ=3.7A0−2.2756,0,0A1−2.8291,0.6433,0.9030A23.5032,0,0A32.2692,0,0A4−1.1800,0,0A50.2767,0,0A6−0.3055,0.3847,0.2925A70.1711,0.2676,0.2585A8−1.4283,0.9777,0.7293A90.1526,0.6975,0.3733A100.2140,2.4168,2.6125A111.4807,0.3650,0.3956A12−0.5725,0.8036,0.9055A13−2.2556,0.6890,1.0281A141.9892,0,0.1928A154.2537,0,0A160.2557,0.0792,0.4831A170.4384,0,0A180.7506,0,0A190.8426,0,0.4403A202.2863,0,0.4810A210.0551,0.0341,0.1301A22−0.4957,0.8162,0.7252A23−2.2926,0,0A240.6620,0,0A250.0260,0.0953,0.5637Figure 2
Transient behaviors ofa0m,a1m,…,a25m for neural network (53) and (54) in Example 1.
(a)(b)(c)(d)(e)Figure 3
Transient behaviors ofa0l,a1l,…,a25l for neural network (53) and (54) in Example 1.
(a)(b)(c)(d)(e)Figure 4
Transient behaviors ofa0u,a1u,…,a25u for neural network (53) and (54) in Example 1.
(a)(b)(c)(d)(e)Figure 5
Transient behaviors of of fuzzy bridge regression coefficientA24γ=3.7 for neural network (53) and (54) with 20 various initial points in Example 1.Figure 6
Taylor’s diagram for the model of the centers in Example1. The rectangle on the x-axis represents the observations, and the red-dashed contours indicate the centered Root Mean Square Errors.Figure 7
Taylor’s diagram for the model of the lower bounds in Example1. The rectangle on the x-axis represents the observations, and the red-dashed contours indicate the centered Root Mean Square Errors.Figure 8
Taylor’s diagram for the model of the upper bounds in Example1. The rectangle on the x-axis represents the observations, and the red-dashed contours indicate the centered Root Mean Square Errors.Figure 9
Fuzzy bubble plots in Example1. (a) Proposed fuzzy bridge regression model with γ=1, (b) proposed fuzzy bridge regression model with γ=2, (c) proposed fuzzy bridge regression model with γ=3, and (d) proposed fuzzy bridge regression model with γ=3.7.
(a)(b)(c)(d)Example 2.
In this example, the dataset is shown in Table4 is taken from [85], with crisp inputs and a symmetric triangular fuzzy output.
Using these data, we develop an estimated fuzzy regression equation:(69)Y^=A0+A1x1+A2x2+A3x3+A4x4+A5x5.
In order to estimate the value ofY^ using (26)–(28), we need to find the optimal bridge parameter γ. Figure 10, considering two different goodness-of-fit measures and distances D1, D2, and D3, illustrates a dramatic rise in the MSM index when γ=2. The MSE index with three different distances D1, D2, and D3, on the other hand, shows a different trend. The numbers fall significantly by γ=2. Therefore, the optimal fuzzy bridge regression model is obtained using (26) and (28) as(70)Y^=−1831.984,201.7570T+2624.075,0.7776×10−5Tx1+82.16977,0.4976×10−7Tx2+72.51824,3.6588Tx3+−45.344,0Tx4+−39.2779,70.6566Tx5,with γ=2, r1=0.1139×108,r2=0.3893×106.
Figures11 and 12 show that the trajectories of model (53) and (54) for solving the fuzzy bridge regression model with γ=2 are convergent to the optimal solution of the problem. Simulation results show that the trajectories of (53) and (54) with any initial point are always convergent to the optimal solution of the problem. Figure 13 displays the transient behaviors of fuzzy bridge regression coefficient A2 based on (53) and (54) with 20 random initial points. Furthermore, the results of several common fuzzy regression models are presented in Table 5.
Comparing the different methods in terms of the goodness-of-fit criteria (MSM, MSE, and MDC, concerning three distancesD1, D2, and D3), it can be seen that the proposed method exhibits better performance compared to the other methods. This is clarified in addition in Figures 13–16. In Taylor’s diagrams, a comparison of the proposed fuzzy bridge regression model with the existing models in [71, 76, 81–84] is shown.
The position of each circle on the graph represents a different model result and is determined by the values of the correlation coefficient and standard deviation. It is clear that proposed model (53) and (54) has a better capability and the most appropriate compared to the other exiting models.
The fuzzy bubble plots show that observations fit well to the selected model because of the closeness of the circles to the bisector of the plane and the short radius of each circle. Therefore, it is concluded that the proposed model used in this paper has a good fitting effect.
In general terms, all figures and the goodness-of-fit criterion in this example indicate that the fuzzy bridge regression model with neural network (53) and (54) has good fitting performance and this model can more accurately describe the original data than the others models.
To end this section, we summarize some advantages of the proposed method as follows:A suitable neural network for analog hardware implementation is suggested which gives a good solution of the fuzzy bridge regression problems.The proposed recurrent neural network does not require any penalty parameter for solving the fuzzy regression problems.The proposed neural network method convergence does not depend on choosing a suitable initial point, since it is globally convergent.The computational burden can be greatly reduced using the proposed approach compared with some existing methods because it is not needed to calculate the inverse of a matrix (see, for example, [31, 50]) in each iteration in order to obtain weighting coefficients.In order to evaluate the proposed model, three different distances and goodness-of-fit measures are used. However, some manuscripts use only one distance. For example, one can see [44, 51, 53].We use a statistical well-known method called “cross validation” and make a statistical comparison with observation through Taylor’s diagram and bubble plots to show the capability of the model.We provide a simulation and an applicable example through our methodology as Examples1 and 2. It has been shown that the proposed method is completely capable for solving real-world problems in the fuzzy bridge regression.Table 4
Observed and estimated data in Example2.
No.x1x2x3x4x5Y1138.0936.4351(6060, 550)2162.1026.5061(7100, 50)3163.7044.7171(8080, 400)4174.5238.0981(8260, 150)5175.3841.4072(8650, 750)6252.9926.4942(8520, 450)7262.9326.4952(9170, 700)8272.0433.1263(10310, 200)9276.1243.0662(10920, 600)10290.2642.6472(12030, 100)11385.7031.3373(13940, 350)12395.2727.6463(14200, 250)133105.9827.6463(16010, 300)14379.2566.8163(16320, 500)153120.532.2563(16990, 650)Figure 10
Goodness-of-fit measures for different values ofγ with three different distances D1,D2, and D3 in Example7. (a) MSM index, (b) MSE index with distance D1, (c) MSE index with distance D2, and (d) MSE index with distance D3.
(a)(b)(c)(d)Figure 11
Transient behaviors ofa0m,a1m,a2m,a3m,a4m and a5m for neural network (53) and (54) in Example 9.
(a)(b)Figure 12
Transient behaviors ofa0l,a1l,a2l,a3l,a4l and a5l for neural network (53) and (54) in Example 9.Figure 13
Transient behaviors ofA2 for neural network (53) and (54) with 20 various initial points in Example 9.Table 5
Performance measures of the proposed fuzzy bridge regression model forγ=2 and some common fuzzy regression models in Example 9.
Performance measuresMSMMSEMDCD1D2D3D1D2D3Fuzzy bridgeγ=20.320316.355134.004242.447927.178858.048859.8414The model in [71]0.143967.9434143.0799108.861746.6452102.207774.0827The model in [81]0.235533.779171.134374.260629.499472.574175.7516The model in [76]0.134444.2797100.7872118.943953.2842115.009175.5325The model in [82]0.208333.082069.990965.212638.291186.829976.1910The model in [83]0.193144.314996.487986.627639.929987.663464.3055The model in [84]0.113242.803993.525982.480359.3054144.3003113.2669Figure 14
Comparison of performances of all models with Taylor’s diagram for the centers in Example 9. The rectangle on thex-axis represents the observations, and the red-dashed contours indicate the centered Root Mean Square Errors.Figure 15
Comparison of performances of all models with Taylor’s diagram for the lower bounds in Example 9. The rectangle on thex-axis represents the observations, and the red-dashed contours indicate the centered Root Mean Square Errors.Figure 16
Fuzzy bubble plots in Example 9. (a) Proposed fuzzy bridge regression model withγ=2, (b) the model in [71], (c) the model in [81], (d) the model in [76], (e) the model in [82], (f) the model in [83], and (g) the model in [84].
(a)(b)(c)(d)(e)(f)(g)


## 8. Conclusions and Future Work

In this paper, a neural network model is presented for solving the fuzzy bridge regression model with fuzzy coefficients and crisp input. To appraise the efficiency of the presented model, a dataset generates through a simulation study in Example1. The suggested neural network performance is, then, evaluated using the MSM, MSE, and MDC with three different fuzzy distance measures compared to some exiting techniques. As a result, we obtained lesser values for these three measures. To further investigate, and the performance of the given model is achieved using another applicable dataset. These numerical results also show that the model is superior to some other exiting techniques. In addition, we present the bubble plots to evaluate the accuracy of the introduced model. Also, Taylor’s diagrams which provide information on multiple statistical indices are presented to evaluate the accuracy of the introduced model. As future work, one can apply the proposed scheme to solving fully fuzzy regression, interval-valued regression, and fuzzy type-2 regression problems.



---

*Source: 8838040-2020-11-26.xml*