# A High-Dimensional Modeling System Based on Analytical Hierarchy Process and Information Criteria

Authors: Tuba Koç
Journal: Mathematical Problems in Engineering
 (2021)

---

## Abstract

High-dimensional data sets frequently occur in several scientific areas, and special techniques are required to analyze these types of data sets. Especially, it becomes important to apply a suitable model in classification problems. In this study, a novel approach is proposed to estimate a statistical model for high-dimensional data sets. The proposed method uses analytical hierarchical process (AHP) and information criteria for determining the optimal PCs for the classification model. The high-dimensional “colon” and “gravier” datasets were used in evaluation part. Application results demonstrate that the proposed approach can be successfully used for modeling purposes.


---

## Body

## 1. Introduction

High-dimensional data refer to the state ofn < p, where the number of unknown parameters is p and the sample size is n. Analyzing high-dimensional data is encountered in many areas. In statistical modeling, the classical solutions fail to produce useful results when there is high-dimensional data. High-dimensional modeling techniques are used to overcome this problem. The dimensional reduction is a convenient approach for high-dimensional modeling [1]. Some of the advantages of size reduction techniques are as follows: (i) it reduces the number of dimensions and data storage; (ii) it requires less time to calculate; (iii) irrelevant, noisy, and unnecessary data can be deleted; (iv) data quality can be well optimized; (v) it helps an algorithm work efficiently and improves accuracy; (vi) it allows visualizing data; and (vii) it simplifies classification and improves performance as well.The researchers have studied on PCA for reducing the dimension of the explanatory variable sets. Liu et al. [2] used kernel PCA for gene expression classification. Nyamundanda et al. [3] gave a novel extension of PPCA called probabilistic principal component and covariates analysis (PPCCA) which provides a flexible approach to jointly model metabolomics data and additional covariate information. In [4], PCA was applied for EEG-based emotion recognition classification. Khedher et al. [5] established a classification system using PCA for early diagnosis of Alzheimer’s disease. Gaikwad and Joshi [6] used PCA within probabilistic neural network to classify the brain tumors. Kondo et al. [7] applied PCA and logistic regression analysis for the diagnosis of lung cancer. Pamukçu et al. [1] used probabilistic PCA and several classification algorithms in gene expression data sets. Passemier et al. [8] developed new statistical theory for probabilistic principal component analysis models in high dimensions. Smallman et al. [9] enhanced a sparse method for unsupervised dimension reduction for data from an exponential-family distribution. In [10], deep neural networks, PCA, and linear support vector machine algorithms were used for Hoda dataset classification. Yao and Lopes [11] demonstrated the role of transformations in bootstrapping methods for high-dimensional PCA with their simulation and numeric experiments. Hung and Huang [12] proposed the generalized information criterion (GIC) for high-dimensional PCA sequence selection. Ayesha et al. [13] presented the state-of-the art size reduction techniques and their suitability for different types of data and application areas. Choubey et al. [14] proposed PCA, particle swarm optimization (PSO), and different machine learning algorithms as feature reduction or feature selection or attribute selection method for the detection of diabetes. The literature can be extended with different studies.While constructing the model with PCA, it is very important issue to determine the optimal number of principal components (PCs). However, there is not an objective criterion of identifying the optimal PCs. Although probabilistic PCA considers information criteria to select, this approach has two drawbacks: (1) probabilistic PCA only considers the explanatory variable set and does not take into account the classification model and (2) each criterion can determine the different number of PCs.In this article, a novel hybrid approach is proposed to optimal model for high-dimensional data sets. The proposed method uses analytical hierarchical process (AHP) and information criteria for determining the optimal PCs for the classification model. The multivariate adaptive regression splines (MARS) algorithm is used as the classification algorithm within selected PCA scores. This article is an important attempt to construct an objective, high-dimensional model in the context of principal component regression modeling.The article is organized as follows.2 introduces the proposed hybrid system for determining the optimal PCs in high-dimensional settings and presents the employment of the MARS algorithm. Section 3 provides the application results on data sets. Section 4 presents the conclusion part.


## 2. Hybrid System Process for Selecting the Optimal PCs

A hybrid system is proposed to perform classification in cases where high-dimensional data are available. The multivariate adaptive regression splines (MARS) model is used as the classification model. The main reason of choosing MARS is inherently having variable selection capability. MARS can exclude redundant dimensions intelligently. The proposed system has three steps:(1)
Once, the principal component analysis is implemented on the explanatory variables. However, for multivariate techniques based on an accurate estimation of true covariance, where then < p problem exists, the classical sample covariance matrix has a systematically distorted Eigen structure. In this case, the structure of the covariance matrix undergoes a distortion such that the largest eigenvalues are up-biased and the smallest eigenvalues are down-biased. To overcome the limitations of the sample covariance matrix, shrinkage approaches are often used to estimate the high-dimensional covariance matrix [15]. Therefore, the shrinkage covariance matrix is used instead of the classical one. The principal component analysis is performed via the shrinkage covariance matrix, and the PCA scores are used as the explanatory variables. In this way, fewer predictors are used to build the MARS model.(2)
The PCA process is performed by selecting the optimal number of PC according to the information criteria using AHP. While performing PCA, the number of PCs should be chosen carefully. Since the PC scores directly affect the classification model, the process of PCA is linked within the MARS model. For each number of PCs, the MARS model is constructed and information criteria values are computed. Obviously, each criterion has the different decision on selecting the number of PCs, and we require a unique solution. This solution is obtained via AHP approach. The TOPSIS method is used for the AHP process. TOPSIS evaluates the several information criteria for every number of PCs, and it gives common scores. Upon these scores, one may select the optimal number of PCs.(3)
After selecting the optimal number of PCs, the MARS model is constructed. Due to the nature of MARS, some redundant dimensions can be reduced to improve the inference capability.


### 2.1. Principal Component Analysis with Shrinkage Covariance Matrix

Principal components analysis (PCA) was first introduced by Pearson [16] and developed by Hoteling [17] and Rao [18]. PCA is a vector-based approach in which the aim is to convert high-dimensional and interrelated vectors into small-sized unrelated vectors. The basic component analysis provides the explanation of the variance-covariance structure through a few linear combinations of the original variables. The general objective is to reduce dimension and to make an interpretation, as well as to take measures against the rank problem and to remove the linear relationship in the variance-covariance matrix. Let X be an (n × p) matrix where n is the number of samples and p is the number of variables or properties. Let us show the covariance matrix of the X data set with C. For the purpose of PCA, when D=1/nYY′ is a diagonal matrix, in the Y=PX transformation, there must be an orthonormal V matrix such that the rows of P are the principal components of X. If the D matrix is rewritten as(1)D=1nPXPX′=1nPXX′P′=P1nXX′P′=PCP′.C variance-covariance matrix (by definition of self-decomposition) can be written as C=VΛV′. In this case, the D matrix is D=PCP′=PVΛV′P′, and if we take P≡V′, we obtain(2)D=V′VΛV′V=V′VΛV′V=Λ.That is, selectingP≡V′ makes D diagonal, and this means that the covariance of the newly obtained variables is zero, which is the purpose of PCA. Thus, the principal components of X are C’s eigenvectors [19]. The general process of the principal component analysis is summarized above. In this study, the eigenvalues of the covariance matrix C, which was used in the analysis of principal components, were negative. The shrinkage covariance matrix is considered instead of C matrix to eliminate possible bias in the analysis results.Shrinkage estimators of theC variance-covariance matrix shrink the eigenvalues of C^MLE to the center. The purpose of shrinkage estimators is to take a convex combination of D^ which is a target diagonal matrix that has been selected appropriately with the sample variance C^MLE of C [20]. Then, the covariance matrix’s shrinkage estimator is as follows:(3)C^S=1−ρ^C^MLE+ρ^D^,where ρ^ is the optimal shrinkage coefficient (or density) and takes the values between 0 and 1. This value can also be a function of observations. Then, the D^ matrix is called the shrinkage target. D^ Naive form is given as follows:(4)D^=trC^MLEpIp=1p∑j=1pλjIp=λ¯Ip.Further information on the different shrinkage target matrices of the shrinkage covariance matrix is given in [21].


### 2.2. Multivariate Adaptive Regression Splines with Several Information Criteria

Multivariate adaptive regression splines (MARS) are a nonparametric regression technique which models the nonlinear relationship between a response variable and the set of predictors via basis functions [22]. Friedman [23] contrived MARS to capture the nonlinearities in the model. In MARS, it is not necessary to know the functional relationship. MARS can detect the relationships among predictors and response using basis functions. A general model form of MARS is shown as follows:(5)fx=∑r=1pβrBXr+ε,where β represents the coefficient vector, B. shows the basis functions, and ε indicates the random error term. The basis functions are described as follows:(6)x−t+=x−t,x>t,0,x≥t,x−t−=x−t,x<t,0,x≥t,where “‒” shows the negative, “+” positive region, and “t” shows the knot points. Knots can be defined as the numbers that controls the starting and ending points in local relationships. MARS uses these knots for identifying the linearity of nonlinear relationships in the model.Figure1 shows a graphical representation for the knots. Two knots points x1 and x2 are selected according to behaviors of the relationship between predictors and response. MARS handles the optimal knot selection with a goodness of fit measure. The most common used measure is generalized cross validation (GCV) which is defined as follows:(7)GCV=∑i=1nyi−yi^21−CM/n2,where y^ indicates the predicted values and CM shows a penalty measure which is related with the number of selected parameters.Figure 1
Example graph for knot.In this study, we used the following information criteria after the most commonly used GCV measurement in MARS. It should be noted that the following criteria are used for the model evaluation part. The included information criteria are Akaike Information Criteria (AIC), Bayesian Information Criteria (BIC), and Consistent Information Complexity Criteria (CICOMP) [24–26]. The formulations of these criteria are as follows:(8)AIC=−2logLM^−2k,BIC=−2logLM^+klogn,CAIC=−2logLM^+klogn+1,where LM^ denotes the log-likelihood of the regression model, k shows the number of variables, and n indicates the sample size.


### 2.3. Analytical Hierarchy Process for Choosing the Number of PCs

The analytical hierarchy process (AHP) that has developed by Saaty [27] is a mathematical decision-making method that enables more efficient tools for researchers to organizing and analyzing complex decisions. It is easy to use and take into account both measurable and nonmeasurable criteria. The AHP method organizes the selected factors in a hierarchical structure according to the criteria, subcriteria, and alternatives under a general target and tries to reach the result. The following steps are followed in the AHP method:By making pairwise comparisons, decision-makers create one matrix for each alternative and one matrix for criteriaThese matrices are normalized, and their consistency is checked (whether the decision-makers two-point comparisons are consistent)Then, with the help of matrix algebra, an average score is obtained for each alternativeThe alternative with the highest score is the most appropriate alternative to the decision-makers comparisons [28].In this study for each number of PCs, the MARS model is constructed and information criteria values are computed. Since each criterion has different decisions on selecting the number of PCs, we require a unique solution. We obtained this solution via AHP approach. The TOPSIS method is used for the AHP process. TOPSIS evaluates the several information criteria for every number of PCs, and it gives common scores.


## 2.1. Principal Component Analysis with Shrinkage Covariance Matrix

Principal components analysis (PCA) was first introduced by Pearson [16] and developed by Hoteling [17] and Rao [18]. PCA is a vector-based approach in which the aim is to convert high-dimensional and interrelated vectors into small-sized unrelated vectors. The basic component analysis provides the explanation of the variance-covariance structure through a few linear combinations of the original variables. The general objective is to reduce dimension and to make an interpretation, as well as to take measures against the rank problem and to remove the linear relationship in the variance-covariance matrix. Let X be an (n × p) matrix where n is the number of samples and p is the number of variables or properties. Let us show the covariance matrix of the X data set with C. For the purpose of PCA, when D=1/nYY′ is a diagonal matrix, in the Y=PX transformation, there must be an orthonormal V matrix such that the rows of P are the principal components of X. If the D matrix is rewritten as(1)D=1nPXPX′=1nPXX′P′=P1nXX′P′=PCP′.C variance-covariance matrix (by definition of self-decomposition) can be written as C=VΛV′. In this case, the D matrix is D=PCP′=PVΛV′P′, and if we take P≡V′, we obtain(2)D=V′VΛV′V=V′VΛV′V=Λ.That is, selectingP≡V′ makes D diagonal, and this means that the covariance of the newly obtained variables is zero, which is the purpose of PCA. Thus, the principal components of X are C’s eigenvectors [19]. The general process of the principal component analysis is summarized above. In this study, the eigenvalues of the covariance matrix C, which was used in the analysis of principal components, were negative. The shrinkage covariance matrix is considered instead of C matrix to eliminate possible bias in the analysis results.Shrinkage estimators of theC variance-covariance matrix shrink the eigenvalues of C^MLE to the center. The purpose of shrinkage estimators is to take a convex combination of D^ which is a target diagonal matrix that has been selected appropriately with the sample variance C^MLE of C [20]. Then, the covariance matrix’s shrinkage estimator is as follows:(3)C^S=1−ρ^C^MLE+ρ^D^,where ρ^ is the optimal shrinkage coefficient (or density) and takes the values between 0 and 1. This value can also be a function of observations. Then, the D^ matrix is called the shrinkage target. D^ Naive form is given as follows:(4)D^=trC^MLEpIp=1p∑j=1pλjIp=λ¯Ip.Further information on the different shrinkage target matrices of the shrinkage covariance matrix is given in [21].


## 2.2. Multivariate Adaptive Regression Splines with Several Information Criteria

Multivariate adaptive regression splines (MARS) are a nonparametric regression technique which models the nonlinear relationship between a response variable and the set of predictors via basis functions [22]. Friedman [23] contrived MARS to capture the nonlinearities in the model. In MARS, it is not necessary to know the functional relationship. MARS can detect the relationships among predictors and response using basis functions. A general model form of MARS is shown as follows:(5)fx=∑r=1pβrBXr+ε,where β represents the coefficient vector, B. shows the basis functions, and ε indicates the random error term. The basis functions are described as follows:(6)x−t+=x−t,x>t,0,x≥t,x−t−=x−t,x<t,0,x≥t,where “‒” shows the negative, “+” positive region, and “t” shows the knot points. Knots can be defined as the numbers that controls the starting and ending points in local relationships. MARS uses these knots for identifying the linearity of nonlinear relationships in the model.Figure1 shows a graphical representation for the knots. Two knots points x1 and x2 are selected according to behaviors of the relationship between predictors and response. MARS handles the optimal knot selection with a goodness of fit measure. The most common used measure is generalized cross validation (GCV) which is defined as follows:(7)GCV=∑i=1nyi−yi^21−CM/n2,where y^ indicates the predicted values and CM shows a penalty measure which is related with the number of selected parameters.Figure 1
Example graph for knot.In this study, we used the following information criteria after the most commonly used GCV measurement in MARS. It should be noted that the following criteria are used for the model evaluation part. The included information criteria are Akaike Information Criteria (AIC), Bayesian Information Criteria (BIC), and Consistent Information Complexity Criteria (CICOMP) [24–26]. The formulations of these criteria are as follows:(8)AIC=−2logLM^−2k,BIC=−2logLM^+klogn,CAIC=−2logLM^+klogn+1,where LM^ denotes the log-likelihood of the regression model, k shows the number of variables, and n indicates the sample size.


## 2.3. Analytical Hierarchy Process for Choosing the Number of PCs

The analytical hierarchy process (AHP) that has developed by Saaty [27] is a mathematical decision-making method that enables more efficient tools for researchers to organizing and analyzing complex decisions. It is easy to use and take into account both measurable and nonmeasurable criteria. The AHP method organizes the selected factors in a hierarchical structure according to the criteria, subcriteria, and alternatives under a general target and tries to reach the result. The following steps are followed in the AHP method:By making pairwise comparisons, decision-makers create one matrix for each alternative and one matrix for criteriaThese matrices are normalized, and their consistency is checked (whether the decision-makers two-point comparisons are consistent)Then, with the help of matrix algebra, an average score is obtained for each alternativeThe alternative with the highest score is the most appropriate alternative to the decision-makers comparisons [28].In this study for each number of PCs, the MARS model is constructed and information criteria values are computed. Since each criterion has different decisions on selecting the number of PCs, we require a unique solution. We obtained this solution via AHP approach. The TOPSIS method is used for the AHP process. TOPSIS evaluates the several information criteria for every number of PCs, and it gives common scores.


## 3. Application

In this part, we applied the proposed method based on the hybridization of principal component analysis (PCA), analytical hierarchical processing (AHP), and information criteria to high-dimensional data sets.Example 1.
Colon data set as gene expression data is available inR software of packages “plsgenomics” [29]. The data set includes p=2000 genes and n = 62 samples. The response variable has two groups as tumor tissues and normal tissues [30]. The task is to classify the tissues using 2000 predictors. Obviously, the data set is high dimensional for the reason of n < p.
If there existsn < p situation for multivariate techniques that is based on an accurate estimation of true covariance, the classical sample covariance matrix has a systematically deteriorated self-structure. As mentioned earlier, the negative eigenvalues of the covariance matrix are a major problem for the analysis.
Table1 presents the eigenvalues of both the classical and shrinkage covariance matrices. As it is seen, the classical covariance matrix produced negative eigenvalues because of the high dimensionality. This problem was handled by using the shrinkage covariance matrix which carries the negative eigenvalues into the positive range. After obtaining positive eigenvalues, we applied the principal component analysis with a positive definite shrinkage covariance matrix.
Table2 shows the information criteria of the MARS model for each number of principal components. In Table 2, while AIC and BIC select 14 principal components, CAIC and MARS model’s GCV measurement selects 15 and more principal component. The reported results clearly denote that each criterion determined different number of principal components. In this case, deciding the number of principal components for researchers is becoming a problem. We have addressed the AHP process to solve this problem and find an optimum number of principal components.
Table3 shows the results for the AHP scores for each criterion. According to the results in Table 3, the number of principal components with a rank value of 1 is 14. So that this information shows us that the number of optimum principal components for this study is 14. After the principal component number was determined by the AHP process, we fitted the MARS regression model.
Table4 shows the results of the MARS regression model. We applied logit transformation in the MARS regression model, and the link function is as follows:(9)Predclass=11−e−z.
The fitted model for the MARS regression is given as follows:(10)z=0.45763+4.31099×BF1−18.35969×ΒF1+3.728584×BF2−2.031415×BF3−7.754506×BF4+22.28331×BF5−7.823195×BF6+22.29569×BF7.
Also, as shown in Table4, the MARS regression model is used to select variables as well as model selection (the model did not select PC1, PC2, and PC7 principal components among 14 principal components).
Table5 shows the performance measurements of the model. Performance measurement formulas we use in the study are as follows:(11)sensitivity=TPTP+FN,specifity=TNTN+FN,accuracy=TP+TNTP+TN+FN+FP,precision=TPTP+FP,recall=TPTP+FN,F−measure=2TP2TP+FP+FN,MCCR=TP∗TN−FP∗FNTP+FPTP+FNTN+FPTN+FN.
In these equations, TP (True positive): means correct positive prediction, FP (False positive) means incorrect positive prediction, TN (True negative) means correct negative prediction, and FN (False negative) means incorrect negative prediction.
When the performance measurement values of the MARS model that we applied to colon cancer data are examined, it is seen that all measurement values are good. The point of particular interest in Table5 is that the MCCR value (0.789) is quite good because this measurement generally does not give good results in model selection methods.
To evaluate the performance of the proposed hybrid approach, we used two high-dimensional modeling techniques: least absolute shrinkage and selection operator (Lasso) and adaptive elastic net (Aenet). Lasso and Aenet models are appropriate for modeling the high-dimensional data sets. Both of these models require a tuning parameter selection, and generally the information criteria are used to select the optimal one. We need to choose an information criterion which is adopted for high-dimensional data sets, and therefore we prefer to use Extended Bayesian Information Criterion (EBIC) during the selection of the tuning parameter in Lasso and Aenet [27]. EBIC is formulated as follows:(12)EBIC=BIC+2γlogpk.
In the EBIC formula,γ=0.5 and logpk shows the combination function. The Lasso and Aenet applications are handled with glmnet and msanet packages [28, 29].
Table6 shows the performance measures for Lasso-EBIC, Aenet-EBIC, and the proposed model. The comparisons are mainly based on three important measures: accuracy, F-measure, and MCCR values. It is pretty clear that the proposed model has the highest accuracy, F-measure, and MCCR values. Especially, the MCCR represents the classification performance for the proposed modeling approach. According to the results, the proposed hybrid approach gives more accurate results when comparing with the other classification models in high-dimensional data sets.Table 1
Eigenvalues of covariance matrices.
IndexClassical-SShrinkage-S10.414220.2594120.367060.2300630.280620.1770240.265210.1674150.205030.12669...............96−8.66E − 180.0113297−1.20E − 170.0111098−1.70E − 170.0108199−4.40E − 170.01075100−6.37E − 170.01048Table 2
Information criteria for each principal component of the MARS model.
PCInformation criteriaAICBICCAICGCV1−87.40951−83.15524−81.155240.236502−87.40951−83.15524−81.155240.236503−91.30137−84.91997−81.919970.229884−102.30053−93.79199−89.791990.199715−112.04647−97.15653−90.156530.193526−114.09778−97.08070−89.080700.196387−111.78619−94.76912−86.769120.203848−111.78619−94.76912−86.769120.203849−111.78619−94.76912−86.769120.2038410−111.78619−94.76912−86.769120.2038411−114.09778−97.08070−89.080700.1963812−110.15391−97.39111−91.391110.1908013−109.15987−94.26993−87.269930.2027414−122.57154−101.30020−91.300200.1904115−109.21318−98.57751−93.577510.1857816−109.21318−98.57751−93.577510.1857817−109.21318−98.57751−93.577510.1857818−109.21318−98.57751−93.577510.1857819−109.21318−98.57751−93.577510.1857820−109.21318−98.57751−93.577510.18578Table 3
Number of basic components by AHP process.
PCScoreRank102026.91E − 161930.111231840.554481750.750401060.75683370.647201580.647201390.6472013100.6472013110.756832120.7407611130.6190116140.929861150.753306.5160.753306.5170.753306.5180.753306.5190.753306.5200.753306.5Table 4
MARS regression model.
Basis functionFormulaBF1Pmax (0, 0.23409 − PC3)BF2Pmax (0, 0.26568 − PC4)BF3Pmax (0, PC5 + 0.38728)BF4Pmax (0, PC6 − 0.15427)BF5Pmax (0, PC8 − 0.10529)BF6Pmax (0, PC8 − 0.46778)BF7Pmax (0, PC11 − 0.53962)BF8Pmax (0, PC14 − 0.03291)BF9Pmax (0, PC8 − 0.46778)BF10Pmax (0, PC11 − 0.53962)BF11Pmax (0, PC14 − 0.03291)Table 5
Performance measurements for the MARS model.
Performance measureValueSensitivity0.864Specificity0.925Accuracy0.903Precision0.925Recall0.925F-measure0.925MCCR0.789Table 6
The performance measures for the classification models.
Performance measureModelProposed model (colon data)Lasso-EBICAenet-EBICSensitivity0.8640.8670.824Specificity0.9250.8090.822Accuracy0.9030.8230.823Precision0.9250.8090.822Recall0.9250.9500.925F-measure0.9250.8740.871MCCR0.7890.6040.602Example 2.
Gravier data set as breast cancer data is available inR software of packages “datamicroarray.” The data set includes p = 2,905 genes and n = 168 samples. The response variable has two groups in which patients with no events after diagnosis were labeled good and early metastases were labeled as poor [31]. Analyzes were made by selecting the first p = 1000 features from the data set. The data set is again high dimensional for the reason of n < p.
Table7 presents the eigenvalues of both the classical and shrinkage covariance matrices. The negative eigenvalue problem is solved using the shrinkage covariance matrix. After obtaining positive eigenvalues, we applied the principal component analysis with a positive definite shrinkage covariance matrix.
Table8 shows the information criteria of the MARS model for each number of principal components. In Table 8, while AIC, BIC, and MARS model’s GCV measurement selects 19 principal components, CAIC selects 20 principal components. We discussed the AHP process to find the optimum number of principal components.
Table9 shows the results for the AHP scores for each criterion. According to the results in Table 9, the number of principal components with a rank value of 1 is 19. After the principal component number was determined by the AHP process, we fitted the MARS regression model.
Table10 shows the results of the MARS regression model. We applied logit transformation in the MARS regression model. The fitted model for the MARS regression is given as follows:(13)z=5.057191+4.150164×BF1+2.523352×BF2−3.483849×BF3−12.72563×BF4+28.27023×BF5−17.27563×BF6−6.074134×BF7−5.962305×BF8+12.79007×BF9−9.163705×BF10+0.6654436×BF11−17.70061×BF12−6.683544×BF13+22.49807×BF14−6.951832×BF15+5.628957×BF16.
Table11 shows the performance measures for Lasso-EBIC, Aenet-EBIC, and the proposed model. In Table 11, the proposed hybrid approach gives more accurate results than other classification models.Table 7
Eigenvalues of covariance matrices.
IndexClassical-SShrinkage-S12.790031.8823021.595431.1707430.944760.6849240.819980.6102950.747200.54854...............996−1.27E − 160.0020997−1.62E − 160.0020998−7.78E − 160.0020999−1.67E − 150.00191000−2.69E − 150.0017Table 8
Information criteria for each principal component of the MARS model.
PCInformation criteriaAICBICCAICGCV1−247.21820−240.97030−238.970300.226862−247.21820−240.97030−238.970300.226863−269.25770−253.63780−248.637800.206574−289.79220−271.04840−265.048400.185215−288.36040−266.49260−259.492600.189326−295.30230−270.31060−262.310600.184177−295.30230−270.31060−262.310600.184178−305.26260−274.02290−264.022900.178589−311.14600−276.78240−265.782400.1750010−311.14600−276.78240−265.782400.1750011−318.66110−278.04960−265.049600.1725512−326.31440−279.45490−264.454900.1702513−326.31440−279.45490−264.454900.1702514−326.31440−279.45490−264.454900.1702515−326.31440−279.45490−264.454900.1702516−311.89210−277.52850−266.528500.1742217−338.05580−281.82440−263.824400.1671018−331.97960−281.99620−265.996200.1673719−339.77790−283.54650−265.546500.1654020−311.67530−277.31170−266.311700.17445Table 9
Number of basic components by AHP process.
PCScoreRank10.00E + 002021.34E − 151930.294861840.603251650.551201760.6296814.570.6296814.580.722261390.7799711.5100.7799711.5110.834818120.890286130.890287140.890284150.890285160.789979170.966872180.942683190.992371200.7871010Table 10
MARS regression model.
Basis functionFormulaBF1Pmax (0, 0.265434 − PC3)BF2Pmax (0, PC3 − 0.265434)BF3Pmax (0, PC3 − 2.178976)BF4Pmax (0, PC4 + 0.06377673)BF5Pmax (0, PC4 − 0.5227402)BF6Pmax (0, PC4 − 0.8542159)BF7Pmax (0, PC6 − 0.5927528)BF8Pmax (0, PC8 − 0.08261739)BF9Pmax (0, PC8 − 0.6397357)BF10Pmax (0, PC8 − 1.085249)BF11Pmax (0, PC9 + 0.8624089)BF12Pmax (0, PC15 + 0.7457308)BF13Pmax (0, −0.3014017 − PC15)BF14Pmax (0, PC15 + 0.3014017)BF15Pmax (0, PC15 − 0.4687581)BF16Pmax (0, PC17 − 0.3960287)Table 11
Performance measurements for the MARS model.
Performance measureModelProposed model (gravier data)Lasso-EBICAenet-EBICSensitivity0.9750.6420.692Specificity0.9470.3020.526Accuracy0.9660.6420.654Precision0.9470.7520.526Recall0.9470.5230.333F-measure0.9470.6560.408MCCR0.9220.2330.191


## 4. Conclusions

In high-dimensional data sets where the number of variables is greater than the number of samples, the classical covariance matrix structure has a systematic degeneration. The degeneration of the classical covariance matrix structure leads to a negative value of the eigenvalues. This causes the results of PCA analysis to be misleading. Besides that fact, the reduction phase requires choosing the optimal number of components. In this study, we have introduced a new approach in model estimation to overcome this problem. The proposed approach objectively identifies the number of PCs for a high-dimensional setting. The proposed system also enables to reduce the irrelevant component through MARS. One of the major advantages of this approach is to associate the selection process within the estimation model. Empirical findings prove that the developed hybrid system produces accurate results for the high-dimensional datasets.



---

*Source: 6198317-2021-08-29.xml*