Software defect prediction using K‐PCA and various kernel‐based extreme learning machine: an empirical study

IET SoftwareVolume 14, Issue 7 p. 768-782 Research ArticleOpen Access Software defect prediction using K-PCA and various kernel-based extreme learning machine: an empirical study Sushant Kumar Pandey, Sushant Kumar Pandey Department of Computer Science and Engineering, Indian Institute of Technology-BHU, Varanasi, IndiaSearch for more papers by this authorDeevashwer Rathee, Deevashwer Rathee Department of Computer Science and Engineering, Indian Institute of Technology-BHU, Varanasi, IndiaSearch for more papers by this authorAnil Kumar Tripathi, Corresponding Author Anil Kumar Tripathi sushantkp.rs.cse16@iitbhu.ac.in Department of Computer Science and Engineering, Indian Institute of Technology-BHU, Varanasi, IndiaSearch for more papers by this author Sushant Kumar Pandey, Sushant Kumar Pandey Department of Computer Science and Engineering, Indian Institute of Technology-BHU, Varanasi, IndiaSearch for more papers by this authorDeevashwer Rathee, Deevashwer Rathee Department of Computer Science and Engineering, Indian Institute of Technology-BHU, Varanasi, IndiaSearch for more papers by this authorAnil Kumar Tripathi, Corresponding Author Anil Kumar Tripathi sushantkp.rs.cse16@iitbhu.ac.in Department of Computer Science and Engineering, Indian Institute of Technology-BHU, Varanasi, IndiaSearch for more papers by this author First published: 17 February 2021 https://doi.org/10.1049/iet-sen.2020.0119Citations: 8AboutSectionsPDF ToolsRequest permissionExport citationAdd to favoritesTrack citation ShareShare Give accessShare full text accessShare full-text accessPlease review our Terms and Conditions of Use and check box below to share full-text version of article.I have read and accept the Wiley Online Library Terms and Conditions of UseShareable LinkUse the link below to share a full-text version of this article with your friends and colleagues. Learn more.Copy URL Share a linkShare onFacebookTwitterLinkedInRedditWechat Abstract Predicting defects during software testing reduces an enormous amount of testing effort and help to deliver a high-quality software system. Owing to the skewed distribution of public datasets, software defect prediction (SDP) suffers from the class imbalance problem, which leads to unsatisfactory results. Overfitting is also one of the biggest challenges for SDP. In this study, the authors performed an empirical study of these two problems and investigated their probable solution. They have conducted 4840 experiments over five different classifiers using eight NASA projects and 14 PROMISE repository datasets. They suggested and investigated the varying kernel function of an extreme learning machine (ELM) along with kernel principal component analysis (K-PCA) and found better results compared with other classical SDP models. They used the synthetic minority oversampling technique as a sampling method to address class imbalance problems and k-fold cross-validation to avoid the overfitting problem. They found ELM-based SDP has a high receiver operating characteristic curve over 11 out of 22 datasets. The proposed model has higher precision and F-score values over ten and nine, respectively, compared with other state-of-the-art models. The Mathews correlation coefficient (MCC) of 17 datasets of the proposed model surpasses other classical models' MCC. 1 Introduction Software testing is an essential phase of the software development life cycle. Software defects cause a discrepancy in actual output and expected output, which leads to system failure. A software defect is a condition in a software product that does not meet the software requirement (as stated in the requirement specifications) or end-user expectation (which may not be specified but is reasonable). Software defect prediction (SDP) is a method to predict the fault-prone module of a software system. After detecting the fault-prone module, the project leader can allocate more tester team members to more fault-prone software modules. It reduces the overall software development cost. A lot of SDP methodologies using statistical techniques, machine learning methods, or deep learning architecture [1-3] have already been proposed by many software practitioners. Extreme learning machine (ELM) [4] is a single hidden layer feed-forward neural network (SHLFN). Unlike standard single hidden neural networks, ELM uses Moore–Penrose generalised inverse [5] to update its weight. ELM is faster compared to SHLFN because it provides no iterative tuning to proceed with the training using generalisation pseudo-inverse operation [6]. It has been extensively applied in many application domains [7]. It randomly chooses hidden units and analytically determines the output weights of SHLFN. ELM is a powerful tool used in complex classification problems such as medical data classification [8], medical diagnostics [9], image quality assessment [10, 11], and many more. Few researchers have also applied ELM over SDP [12, 13] and cross-project defect prediction [14]. Kernel principal component analysis (K-PCA) [15] is a statistical procedure that mainly uses an orthogonal transformation to convert the group of observations of most probable co-related variables into a group of linearly uncorrelated variables. When the dataset is multidimensional data, it consists of a few features that are irrelevant or redundant for prediction problems, so K-PCA can remove the redundant dimensions and only retain the most relevant features. K-PCA has also been applied in several application domains such as medical diagnostics [16], medical image processing [17, 18], and other applications. Some software practitioners have utilised K-PCA over SDP models using various kernel methods [12, 19, 20]. SDP now becomes one of the most prominent research areas because of several machine learning and deep learning methods. Fig. 1 shows the basic structure of the SDP model. The instance is generated from a software repository with a class label (defective/non-defective). Datasets are fed into a trained model after preprocessing, and the trained model then predicts the software instance is faulty or not. There are various data repositories of SDP such as NASA, Oracle, PROMISE Eclipse etc., provide datasets for experimental and research purposes. However, few researchers have challenged the quality of these datasets. We have used eight NASA datasets and 14 PROMISE repository [21] datasets for our experiments. Fig. 1Open in figure viewerPowerPoint Basic structure of SDP model Shepperd et al. raised a question over the quality of NASA datasets. They claimed that most of the data instances were replicated, and the distribution of instances was skewed over the majority class, so the total number of positive classes (defective module) of data is far less than the total number of negative classes (non-defective). Most of the machine learning algorithms work befittingly when the number of instances of every class is roughly equal or else it leads to bias results towards the majority class, this problem is known as the class imbalance problem [22, 23]. When the number of instances of one class far exceeds the other, problems arise. We have applied the synthetic minority oversampling technique (SMOTE) [24] sampling method to address this problem. Overfitting [25] happens when a model learns the details and noise in the training data to the extent that it negatively impacts the performance of the model on new data (test data). This means that the noise or random fluctuations in the training data is picked up and learned as concepts by the model. The problem is that these learned concepts do not apply to new datasets and negatively impact the models' ability to generalise. Class imbalance and overfitting are two significant challenges for SDP models. The main objective of this study is to address these two issues and provide a useful SDP model. According to our empirical study, we have found a few problems related to existing SDP models, which are listed as follows: (i) As most of the datasets of software repository are imbalanced, i.e. the cardinality of defective instance is much less than the cardinality of non-defective instance, this leads to unsatisfactory results by any machine learning-based prediction model. (ii) ELM has not been thoroughly investigated in previous SDP models. (iii) Various kernels of ELM have not been effectively utilised with K-PCA to build an effective SDP model. The different kernels produce different accuracy; there is a lack of empirical study over various kernels of ELM and its combination with K-PCA. (iv) Training time can also be a salient feature for any prediction model, as software projects are getting bulky nowadays, predicting buggy modules over such projects can be more time-consuming. It will be more favourable when the learning model takes lesser training time and produces high accuracy. As we have discussed, ELM is mighty for any classification problem, and it will require some hyper-parameters tuning for optimal results. NASA and PROMISE repository datasets have some redundant features that need to be removed using K-PCA. The combined approach of ELM and K-PCA over different public datasets to predict fault-prone modules is the target objective of this study. The contribution of this study is given below. (i) We have proposed an SDP model using various kernels of ELM. (ii) To remove various redundant and irrelevant features, we have used K-PCA in our proposed approach. (iii) The proposed model also addresses the class imbalance problem using the SMOTE sampling method. (iv) The proposed model provides unbiased results towards any of the classes, with lesser training time as compared to other state-of-the-art methodologies. (v) The proposed work outperforms the various existing SDP techniques in terms of different performance metrics. Xu et al. [12] proposed the SDP models based on kernel PCA and weighted ELM. After our empirical study, we have framed three research queries (RQs) related to our proposed approach: RQ-1: Calculate the performance when K-PCA is combined with various versions of the ELM technique and compare it with other traditional SDP techniques. RQ-2: How the proposed approach addresses both class imbalance and overfitting problems? RQ-3: How does the training time taken by the proposed method compare with other classical models? We will address all three RQs in Section 5 The structure of the paper is organised in the following manner: In Section 2, we will discuss the related work. The background related to defect prediction is illustrated in Section 3. The empirical study and proposed approach are discussed in Section 4. Finally, we will describe the results in Section 5, and the conclusion and future work in Section 6. 2 Related work We have categorised the related work into three sections: first regarding various machine learning-based SDP methods, second regarding the application of different feature selection methods, and third about the class imbalance and overfitting problems related to SDP models. 2.1 Machine learning-based SDP methods There has been much research work [26, 27] going on over SDP over the past three decades. Naive Bayes [28-31], support vector machine (SVM) [32-34], classification and regression trees [35, 36], AdaBoost [37-39], random forest [40-42], artificial neural network as defect prediction [43, 44], and development effort estimation [45]. Multilayer perceptrons (MLPs) [46, 47] are the most widely used classifiers in defect prediction. Recently few researchers proposed deep learning-based SDP models [48, 49]. Turhan and Bener [50] and Pain and Dugan [51] applied the Bayesian network over the NASA dataset to build the SDP model and found effective results. Elish and Elish [33] performed experiments over NASA datasets using SVM and concluded that SVM surpasses the performance of the basic logistic regression (LR) model. In a recent work, Wie et al. [52] combined dimension reduction techniques with SVM, which leads to achieving significant good results. Recent works by Pandey et al. [53] and Mori and Uchihira [54] proposed an augmented-based Naive Bayes model and also stated the trade-off between accuracy and interpretability. 2.2 Feature selection-based SDP model Many researchers investigated the influence of feature selection methods on defect prediction. Song et al. [3] suggested that feature selection is an essential phase of any defect prediction framework. Arar and Ayan [55] suggested a feature-dependent Naive Bayes (FDNB) approach and applied it over SDP. As many features are interrelated, the removal of any such features reduces the performance of the model. They employed the FDNB classification method over the NASA dataset and calculated the dependence value between the pair of features. They found results outperform the classical Naive Bayes model. Shivaji et al. [56, 57] applied a filter-based and wrapper-based feature selection method over the NASA dataset to the proposed fault prediction model. Xu et al. [58] conducted substantial experiments using 32 different feature selection methods on three public datasets. They concluded that the performance of K-PCA is the worst, among other methods. Ghotra et al. [59] extended his experiment using 21 different classification models and 30 feature section techniques over 18 NASA and PROMISE datasets. He suggested that the co-relation based filter-subset selection method has the best performance over all datasets. 2.3 Class imbalance and overfitting problems in SDP The class imbalance problem hinders the satisfactory performance of the SDP model. Thus, many oversampling [60] and under sampling methods [61] were introduced to resolve this problem. Kamie et al. [62] conducted experiments over four sampling methods and analysed their performance. They conducted experiments on two industrial datasets and found unbiased results. Bennin et al. [63] applied five statistical methods over six sampling techniques on ten public datasets and found extensively satisfying results. Tong et al. [48] and Pandey et al. [64, 65] applied a dropout regularisation technique to avoid the overfitting problem. Khoshgoftaar and Allen [66] suggested a tree-based approach to avoid overfitting problems. Ratsch et al. [67] claim that using the ensemble learning method can avoid overfitting problems. 3 Background In this section, we will illustrate the basic background definitions; first, we will discuss the software metrics and datasets that we have used for experimental purposes. After that, we will explain the various performance metrics that we have used to evaluate the performance of our proposed approach. 3.1 Software metrics We have used eight NASA and 14 PROMISE repository datasets in our experiments. Both repositories consist of 20 software features. The NASA data metrics consist of features such as the line of code, counts of a blank line, number of operators and operands, design complexity, essential complexity etc. Table 1 shows the various software metrics of both PROMISE and NASA software projects. Table 1. Software metrics of PROMISE and NASA projects S.No PROMISE repository NASA projects 1. afferent couplings count of blank lines 2. average method complexity count of code and comments 3. arithmetic mean value of cyclomatic complexity (CC) (Avg_CC) count of comments 4. cohesion among methods of class cyclomatic_complexity 5. coupling between object classes design_complexity 6. coupling between methods essential_complexity 7. data access metric halstead_content 8. depth of inheritance tree halstead_difficulty 9. efferent couplings halstead_effort 10. greatest value of CC (Max_CC) halstead_error_estimate 11. inheritance coupling halstead_level 12. lines of code halstead_length 13. lack of cohesion in methods (LOCM) halstead_programming_time 14. LOCM3 halstead_volume 15. measure of aggregation line count of code 16. measure of functional abstraction line count of executable code 17. number of children number of operands 18. number of public methods number of operators 19. response for a class number of unique operands 20. weighted methods per class number of unique operators 3.2 Experimental datasets As we have discussed earlier that we have used 22 public datasets, of which 14 are from the PROMISE repository [21] and eight are from NASA defect datasets. Table 2 shows the list of all 22 datasets, CM1, JM1, KC1, KC2, KC3, MC1, MC2, and PC1 are NASA software datasets whereas ant, arc, camel, ivy jedit, log4j, poi, prop, redaktor, synampe, tomcat, velocity, xalan, and xerces are datasets of PROMISE repository. As we can see in Table 2, CM1, MC1, PC1, camel, jedit, and prop datasets are highly imbalanced, i.e. % of defective modules is <10%. The percentage of the faulty module in JM1, KC1, KC3, arc, ivy, and redaktor lies between 10 and 20%. The imbalanced datasets lead to producing unsatisfactory and biased results. Table 2. Dataset description Datasets No. of metrics No. of instances No. of defective instance % of defective instance CM1 21 498 49 9 JM1 21 10,885 2106 19.34 KC1 21 2109 326 15.4 KC2 21 522 107 20.4 KC3 39 194 36 18.5 MC1 38 1988 46 2.3 MC2 39 127 44 32.6 PC1 21 1107 76 6.8 ant 20 745 166 22.28 arc 20 234 27 11.53 camel 20 339 13 3.8 ivy 20 352 40 11.36 jedit 20 492 11 2.24 log4j 20 205 189 92.1 poi 20 442 281 63.57 prop 20 660 66 10 redaktor 20 176 27 15.3 synampe 20 256 86 33.59 tomcat 20 256 86 33.59 velocity 20 229 78 34.06 xalan 20 909 898 98 xerces 20 588 437 74.31 3.3 Performance measure We have used four performance metrics: area under receiver operating characteristics (ROC) [68] curve, F-measure [69], precision [70], and Mathews correlation coefficient (MCC) [71]. The confusion matrix is shown below, it describes the prediction as true positive (TP), false positive (FP), true negative (TN), or false negative (FN). (i) ROC curve: ROC curve is generated by plotting the TP rate against the FP rate (FPR) at different threshold settings. It is one of the key performance metrics to evaluate the binary classification model. The value of ROC lies between 0 and 1. If the prediction is 100% accurate, then the ROC value is 1; if the prediction is 100% inaccurate, then the ROC value is 0. ROC is 0.5 for random prediction. (ii) Precision: It is also known as a positive predictive value. It mainly calculates the fraction of relevant instances over retrieved instances. Equation (1) shows the basic precision formula: Precision = TP TP + FP (1) (iii) F-measure: It is a harmonic mean of precision and recall [72]; it is used to analyse both precision and recall at the same time. It is also known as F 1 score. Its values lie between [0, 1], zero for all incorrect predictions and 1 for all correct predictions. F − measure = 2 × precision × recall precision + recall (2) (iv) MCC: It is an essence, a correlation coefficient between the predicted and observed binary classifications. Its values lie between −1 and +1, −1 when the observation is completely contradicted with prediction and +1 when observation completely agrees with the prediction. Sokolova et al. [69] suggested that MCC is the optimal metric for the class imbalance problem, due to this reason; we have considered MCC as the evaluation metrics. The expression for MCC is given in (3) MCC = ( TP × TN ) − ( FP × FN ) ( TP + FP ) ( TP + FN ) ( TN + FP ) ( TN + FN ) (3) 4 Empirical study and the proposed approach In this section, we will discuss the methodologies that we have used for our empirical study and proposed architecture. Fig. 2 shows the graphical abstract of the proposed model. As shown in Fig. 2, the data instances of the software system are created from the software repository. Then, noise removal techniques are applied to remove noisy instances. After getting noise-free data, instances preprocessing of datasets is required. In preprocessing, we formed three steps. In the first step, the SMOTE sampling technique is utilised to remove the class imbalance problem in datasets. Then, we apply the min–max scaling technique [73] to normalise the data points. After that, K-PCA [15] is applied to select the most relevant features. After preprocessing, the data-split state occurs, in which we split datasets into three sections, train set (80%), test set (10%), and validation set (10%). We have also tried train, test, and validation sets as [70, 15, 15%], [60, 20, 20%] etc. but obtained optimal results at [80, 10, 10%]. We applied 10-fold cross-validation [74] over training and testing sets and then fed these into the ELM model. The validation of ELM is done using a validation set. Finally, we evaluated the performance of the proposed model using ROC, precision, F-measure, and MCC. The detailed illustration of each state will be discussed in the respective subsections. For a fair and unbiased comparison between the proposed model and baseline method, we apply the same SMOTE sampling method, then after the K-PCA feature reduction mechanism. Fig. 2Open in figure viewerPowerPoint Graphical abstract of the proposed model 4.1 Preprocessing The preprocessing consists of two steps. The first is to apply a SMOTE sampling method [24]. SMOTE is a very efficient sampling method that addresses the class imbalance problem [75]. We have also tried to spread subsampling [76], resampling method [77], and class balancer [78]. The sampling method adds or removes data instances to achieve a balanced training set. It might result in the loss of much important information, which can lead to unsatisfactory results [79, 80]. After analysing our empirical evidence, we found SMOTE produces optimal results better compared to these sampling approaches. 4.1.1 SMOTE sampling We have considered the pseudo-code of SMOTE from Chawla et al. [24]. It creates a synthetic instance rather than oversampling with replacement. Minority class is oversampled by considering each minority class sample and then adding synthetic instance along with the line segments joining any/all of the k minority class nearest neighbours. Depending on the amount of oversampling required, neighbours from the k-nearest neighbours are randomly chosen. The pseudo-code of SMOTE is given in Algorithm 1 (see Fig. 3). Fig. 3Open in figure viewerPowerPoint Algorithm 1: pseudo-code of SMOTE algorithm 4.1.2 Normalisation After the SMOTE sampling technique, we have applied min–max normalisation [73] for scaling different features so that each feature gets an equal weight for the learning algorithm. It brings the values of data points from 0 to 1. So we used (4) to calculate the normalised values z = x − min ( x ) max ( x ) − min ( x ) (4) 4.1.3 Kernal principal component analysis After feature scaling, we have to select the most relevant features for the prediction model. The participation of irrelevant features results in unsatisfactory results [81]. As we have discussed earlier that in our proposed model we have employed K-PCA [15] to resolve this problem. It is a non-linear form of principal component analysis (PCA) [82]. The classical PCA always tries to find the linear component of low dimensional data; in a few cases, we required a non-linear principal component for the low-dimensional dataset. However, if we apply classical PCA in such a scenario, then conventional PCA will fail to find good representation. The software metrics are correlated, a non-linear principal component for correlated data leads to a better prediction model. K-PCA overcomes such limitations, a more detailed difference between the PCA and K-PCA is given by Cao et al. [83]. We have also compared the performance between K-PCA- and PCA-based approaches when both of them are applied with different versions of ELM. K-PCA is a transformation to diagonalise an estimate of the covariance matrix of the data x k , where k = 1 , 2 , 3 … , l ⋅ x k ϵ R N , ∑ k = 0 l x k = 0. It first maps the data non-linearly into a feature space F ^ by Ω : R N → F ^ , x → X(5) where F ^ is a very large dimensionality for fixing choice of Ω, PCA can still perform over F ^ . It can be done using kernel function SVM [84]. Let the data mapped into features space Ω ( x 1 ) , Ω ( x 2 ) , … , Ω ( x l ) is centred, i.e. ∑ k = 0 l Ω ( x k ) = 0 C ^ = 1 l ∑ j = 1 l Ω ( x j ) Ω ( x j ) T (6) To perform PCA over a covariance matrix, we need eigenvalues λ and eigenvector V ^ . These should satisfy λ V ^ = C ^ V ^ , and after substituting (6) into it, we obtain λ ( Ω ( x k ) ) ⋅ V ^ = Ω ( x k ) ⋅ C ^ V ^ (7) Let ℵ 1 , … , ℵ l be the co-efficient such that V ^ = ∑ i = 1 l ℵ i Ω ( x i )(8) Substituting (6) and (8) into (7) and defining l × l matrix χ as χ i j = ( Ω ( x i ) T ⋅ Ω ( x j ) )(9) Then (9) can be rewritten as: l ⋅ λ ⋅ χ ⋅ ℵ = χ 2 ℵ(10) After normalisation, the solution of ℵ k belongs to non-zero, eigenvalues by requiring the corresponding vector in F ^ be normalised, i.e. V ^ k T V ^ k . By virtue of (8)–(10), this translates into 1 = ∑ i , j = 1 l ℵ i k ℵ j k ( Ω ( x i ) T ⋅ Ω ( x j ) ) = ( ℵ k ⋅ χ ⋅ ℵ k ) = λ k ( ℵ k ⋅ ℵ k )(11) For the PCA extraction, it computes the projection of the datasets of test point Ω ( x ) into eigenvectors V ^ k in F ^ according to V ^ k ⋅ Ω ( x ) = ∑ i = 1 l ℵ k ( Ω ( x i ) ) T ⋅ Ω ( x )(12) So finally after successfully applying the SVM kernel [84], the data instance kernels x and y are k ( x , y ) = ( x , y ) d (13) Fig. 4 shows the mapped sample of the lth component of the data item. We can see that the lth component of the data item is internally mapped with the inner product item, and the inner product is one-to-one join with the mapped sample. Finally, the mapped samples are extracted into the new sample. Fig. 4Open in figure viewerPowerPoint Feature extraction using K-PCA 4.2 Extreme learning machine Our newly mapped dataset is ( x i , y i ) d where i ∈ [ 1 , 2 , … , l ]. Let the output of the ELM with h hidden units, and activation function as γ ( x d ) is o i . Fig. 5 shows the basic architecture of ELM. It consists of three layers: an input layer, a hidden layer, and an output layer. The detailed description of each layer and function related to then is given below o i = ∑ j = 1 k η j γ j x i d = ∑ j = 1 k η j γ j w j , b j , x j d (14) where i ∈[1–l], w j represents the input weight vector, which contains input nodes, i.e. w j 1 , w j 2 , … , w j l at the jth hidden node. b j denotes the bias value at the jth hidden units. η j represents the output weight vector connecting output units, η j lies in η j 1 , η j 2 , … , η j l , as shown in Fig. 5. Here o i is the output of the ith sample. The most common activation functions γ are sigmoid function, Radial basis function (RBF) [85], hard limit function, and multi-quadratic function [4, 7]. We have considered only RBF and poly [4] kernel in our empirical study. Equation (15) can be rewritten as H ⋅ η = O(15) where H is the output matrix of a hidden layer of single hidden layer feed forward neural networks (SLFNs) H = H w 1 , w 2 , … , w k , b 1 , b 2 , … , b k , x 1 d , x 2 d , x 3 d , … , x k d (16) γ w 1 , b 1 , x 1 d … γ w 1 , b 1 , x 1 d . . . . . . . . . γ w 1 , b 1 , x l d … γ w k , b k , x l d l × k (17) In (16), H represents the output vector of the hidden unit with respect to input sample x d , the jth component of H denotes the output vector of the jth hidden unit with respect to the input sample x 1 d , x 2 d , … , x l d (see Fig. 5). η represents the weights matrix that connects the hidden layer with the output layer η 1 T . . . η k T k × p (18) O is the expected output matrix, every row of which indicates the output vector of every sample. It is defined as follows: O = o 1 T . . . o l T o 11 . . . o 1 p . . . . . . . . . . . . . . . o l 1 . . . o l p l × p (19) Since our target is to minimise the output error in SLFN, we approximate the error to zero, as given in (20) O − Y = ∑ i = 1 l | o i − y i | = 0(20) Here Y is the output vector matrix, which is defined in (21) Y = y 1 T . . . y l T = y 11 … y 1 c . … . . … . . … . y l 1 … y l c l × p (21) Now we need to solve Y = H·η; the optimal solution of Y is given in (22). After using Haung et al. [4, 86] optimal strategy, we obtain λ ^ = H ^ Y = H T ⋅ H (22) where H ^ indicates Moore–Penrose generalisation [87, 88] of the hidden layer output matrix. Here λ ^ ensures a minimum training error and obtains optimal results. To avoid traps in the local optimal solution, applying η ^ is a unique [4] solution, we have obtained from Karush–Kuhn–Tucker (KKT) theorem [89]. Th