# CappedL1-Norm Proximal Support Vector Machine

Authors: Pei-Wei Ren; Chun-Na Li; Yuan-Hai Shao
Journal: Mathematical Problems in Engineering
 (2022)

---

## Abstract

Compared to the standard support vector machine, the generalized eigenvalue proximal support vector machine coped well with the “Xor” problem. However, it was based on the squared Frobenius norm and hence was sensitive to outliers and noise. To improve the robustness, this paper introduces cappedL1-norm into the generalized eigenvalue proximal support vector machine, which employs nonsquared L1-norm and “capped” operation, and further proposes a novel capped L1-norm proximal support vector machine, called CPSVM. Due to the use of capped L1-norm, CPSVM can effectively remove extreme outliers and suppress the effect of noise data. CPSVM can also be viewed as a weighted generalized eigenvalue proximal support vector machine and is solved through a series of generalized eigenvalue problems. The experimental results on an artificial dataset, some UCI datasets, and an image dataset demonstrate the effectiveness of CPSVM.


---

## Body

## 1. Introduction

A support vector machine (SVM) [1] is a popular tool for coping with binary classification problems. It aims to find parallel hyperplanes with the largest margin to distinguish two class data, but it cannot deal with “Xor” problems well. To solve this problem, Mangasarian and Wild [2] proposed the first nonparallel hyperplane generalized eigenvalue support vector machine (GEPSVM). GEPSVM minimizes the squared L2-norm distance of the samples to its own class and maximizes the squared L2-norm distance of the samples to the other class to find a pair of nonparallel optimal classification hyperplanes to proximate two classes. After that, many researchers started to study nonparallel support vector machines (NSVMs) [3–7], such as twin support vector machine (TSVM) and its modifications [8–11] and nonparallel proximal support vector machine (NPSVM) and its variants [5, 12]. For example, to improve its generalization ability, Shao et al. proposed an improved generalized eigenvalue proximal support vector machine (IGEPSVM) [13]. In order to keep the consistency between the decision process and the training process, the authors further proposed a proximal classifier with consistency (PCC) [14], which is based on comparing two distances between a point and two hyperplanes.As we know, the squaredL2-norm distance metric is sensitive to outliers since the effect of outliers can be magnified by the square operation. In order to reduce the effect of outliers, the L1-norm is usually used as a replacement for the squared L2-norm to achieve robustness. Li et al. [15] reformulated the objective function of GEPSVM by replacing the squared L2-norm terms with the corresponding L1-norm terms and proposed an L1-norm nonparallel SVM (L1-NPSVM). Yan et al. [16] converted the L1-norm generalized eigenvalue problem into a strong convex programming problem and implemented a simple iterative algorithm to solve it. Gu et al. proposed an L1-norm twin projection support vector machine (TPSVM-L1) [17] for the recognition of images and robust representation. In fact, the robustness of L1-norm has been applied in many machine learning tasks, including feature extraction [18], dimensionality reduction [19], or clustering [20]. Due to its robustness, modifications of the L1-norm that were applied on NSVMs were also extensively studied, for example, Lp-norm p>0 and generalized elastic network-based nonparallel support vector machines [12, 21–23].However, though theL1-norm is more robust, it still will be affected by outliers that are very far from the normal points. In this case, even the L1-norm is not robust enough to reduce the impact of outliers, so one hopes to eliminate the impact of these large outliers or limit it to a certain range. An effective way to achieve this is to apply an idea of “capped” operation, which adds an upper bound to the common norms. In fact, this idea was used in previous studies to establish a robust machine learning model, for example, capped L2-norm for robust feature learning [24], capped L2,1-norm for regression and classification [25, 26], capped Lp-norm for classification and feature extraction [27–29], capped nuclear norm for matrix factorization or completion [30, 31], and capped trace norm [31] for robust principal component analysis. For SVM classifiers, Wang et al. [32] proposed a new robust capped L1-norm twin support vector machine (CTWSVM), which sustained the advantages of TWSVM and promoted robustness in solving a binary classification problem with outliers. Yuan et al. [33] proposed a novel robust least squares twin support vector machine framework by using a capped L2,p-norm distance metric to reduce the influence of noise and outliers.However, there is no corresponding study on proximal support vector machines. To further improve the robustness of GEPSVM and L1NPSVM, in this paper, we propose a cappedL1-norm based proximal support vector machine (CPSVM). For a given ϵ>0, the capped L1-norm is defined as(1)vcapL1=∑i=1nminvi,ϵ,where v is an n-dimensional column vector. Note that, strictly speaking, capped L1-norm is not an actual norm. In fact, though it satisfies the positive scalability and triangle inequality, it does not satisfy homogeneity. However, it still can be used as a distance or similarity measure, and it also does not affect its robustness property. From the definition of capped L1-norm, it is seen that when encountering outliers, capped L1-norm can effectively remove them beyond ϵ and favor the input control. Therefore, ϵ is a thresholding parameter for identifying extreme data outliers that will affect recognition results and bring robustness. Compared to the existing proximal support vector machines, which utilized squared L2-norm or L1-norm, CPSVM proposed in this paper can improve the robustness to outliers. In specific, CPSVM has the following characteristics:(i)
The application of cappedL1-norm on CPSVM makes it robust to data outliers and feature noise. In specific, the capped parameter ϵ in capped L1-norm helps identify data outliers, and the L1-norm in capped L1-norm can resist sensitivity to feature noise. In order to enhance the generalization ability of CPSVM and prevent overfitting, an L2-norm regularization term is also introduced.(ii)
An effective iterative algorithm is designed to solve the proposed nonsmooth and nonconvex optimization problem. In each iteration, it solves two generalized eigenvalue problems.(iii)
The effectiveness and robustness of CPSVM are demonstrated by experiments on an artificial dataset, some UCI datasets, and a handwritten digit image database.The paper is organized as follows. Section2 briefly reviews the related proximal support vector machines. Section 3 proposes CPSVM and its solving algorithm. Section 4 makes comparisons of CPSVM with its related approaches. Concluding remarks are given in Section 5.The notation of the paper is as follows. All vectors are column ones, and vectors and matrices are shown in bold. Consider the training datasetT=x1,x2,…,xN with the associated class labels y1,y2,…,yN belonging to 1,2, where xl∈ℝn for l=1,2,…,N. Write X=x1,x2,…,xN∈ℝn×N as the corresponding data matrix of T. Assume that the i-th class contains Ni samples, i=1,2. We organize the N1 inputs of Class 1 into a matrix X1=x11,x12,…,x1N1∈ℝn×N1 and the N2 inputs of Class 2 into a matrix X2=x21,x22,…,x2N2∈ℝn×N2, where N1+N2=N. Let x¯=1/N∑l=1Nxl be the mean of all samples and let x¯i=1/Ni∑j=1Nixij be the mean of samples in the i-th class, i=1,2, where xij is the j-th sample in the i-th class. For a vector v=v1,v2,…,vnT∈ℝn, its L2-norm is defined as v2=∑i=1nvi2, and L1-norm is defined as v1=∑i=1nvi. For simplicity, we write ⋅2 as ⋅ for brevity.


## 2. Related Works

### 2.1. GEPSVM

GEPSVM [2] aims to find two nonparallel hyperplanes:(2)f1x=w1Tx+b1=0andf2x=w2Tx+b2=0,such that the i-th hyperplane is close to the samples of the i-th class and meanwhile far away from the samples of the other class, where wi∈ℝn, bi∈ℝ, i=1,2, x∈ℝn. The optimization problems of GEPSVM are formulated as(3)minw1,b1≠0w1TX1+eb12+δw1b12w1TX2+eb12,(4)minw2,b2≠0w2TX2+eb22+δw2b22w2TX1+eb22,where e is the all one vector of an appropriate dimension, and δ>0 is a regularization parameter.To simplify the notation, we introducew¯1=w1T,b1T∈ℝn+1, w¯2=w2T,b2T∈ℝn+1 and X¯1=X1,eTT∈ℝn+1×N1, X¯2=X2,eTT∈ℝn+1×N2. Then, the solution of GEPSVM can be obtained via the following generalized eigenvalue problems(5)X¯1X¯1T+δIw¯1=λX¯2X¯2Tw¯1,w¯1≠0,X¯2X¯2T+δIw¯2=λX¯1X¯1Tw¯2,w¯2≠0,where λ is the eigenvalue of (5), and I is the identity matrix of an appropriate dimension.After obtaining the optimalw¯1 and w¯2, an unseen sample x is classified according to(6)labelx=argmini=1,2fixwi.


### 2.2. L1-NPSVM

Since the squaredL2-norm is very sensitive to noise, in order to improve the robustness, a robust L1-norm support vector machine (L1-NPSVM) was proposed in [15], which was formulated as(7)minw¯1≠0w¯1TX¯11w¯1TX¯21andminw¯2≠0w¯2TX¯21w¯2TX¯11.By introducing a nonconvex surrogate function, (7) is solved by the gradient ascending (GA) algorithm. However, the nonconvexity of the surrogate function may cause the miss of an optimal solution. Besides, an appropriate step size is also needed to be chosen carefully.After obtaining the optimalw¯1 and w¯2, an unseen sample x was classified according to(8)labelx=argmini=1,2fix.


## 2.1. GEPSVM

GEPSVM [2] aims to find two nonparallel hyperplanes:(2)f1x=w1Tx+b1=0andf2x=w2Tx+b2=0,such that the i-th hyperplane is close to the samples of the i-th class and meanwhile far away from the samples of the other class, where wi∈ℝn, bi∈ℝ, i=1,2, x∈ℝn. The optimization problems of GEPSVM are formulated as(3)minw1,b1≠0w1TX1+eb12+δw1b12w1TX2+eb12,(4)minw2,b2≠0w2TX2+eb22+δw2b22w2TX1+eb22,where e is the all one vector of an appropriate dimension, and δ>0 is a regularization parameter.To simplify the notation, we introducew¯1=w1T,b1T∈ℝn+1, w¯2=w2T,b2T∈ℝn+1 and X¯1=X1,eTT∈ℝn+1×N1, X¯2=X2,eTT∈ℝn+1×N2. Then, the solution of GEPSVM can be obtained via the following generalized eigenvalue problems(5)X¯1X¯1T+δIw¯1=λX¯2X¯2Tw¯1,w¯1≠0,X¯2X¯2T+δIw¯2=λX¯1X¯1Tw¯2,w¯2≠0,where λ is the eigenvalue of (5), and I is the identity matrix of an appropriate dimension.After obtaining the optimalw¯1 and w¯2, an unseen sample x is classified according to(6)labelx=argmini=1,2fixwi.


## 2.2. L1-NPSVM

Since the squaredL2-norm is very sensitive to noise, in order to improve the robustness, a robust L1-norm support vector machine (L1-NPSVM) was proposed in [15], which was formulated as(7)minw¯1≠0w¯1TX¯11w¯1TX¯21andminw¯2≠0w¯2TX¯21w¯2TX¯11.By introducing a nonconvex surrogate function, (7) is solved by the gradient ascending (GA) algorithm. However, the nonconvexity of the surrogate function may cause the miss of an optimal solution. Besides, an appropriate step size is also needed to be chosen carefully.After obtaining the optimalw¯1 and w¯2, an unseen sample x was classified according to(8)labelx=argmini=1,2fix.


## 3. The Proposed CappedL1-Norm Proximal Support Vector Machine

### 3.1. Problem Formulation

As we see above, the application of squaredL2-norm metric in GEPSVM amplifies the influence of outliers, which will affect the construction of the optimal classification hyperplanes. Though L1-NPSVM aims at giving a more robust performance on the problem with outliers, it still may perform not very well when encountering large enough outliers or noise. In contrast, capped L1-norm metric that involves an upper bound can effectively control the influence of extreme outliers. Therefore, we propose the following capped L1-norm proximal support vector machine (CPSVM), the optimization problem of which is expressed as(9)minw¯1≠0w¯1TX¯1capL1+δw¯12w¯1TX¯2capL1,(10)minw¯2≠0w¯2TX¯2capL1+δw¯22w¯2TX¯1capL1,where δ>0 is a regularization parameter. Minimizing the first item in the numerator of (9) makes the samples in Class 1 as close as possible to the hyperplane f1x=0. Maximizing the denominator of (9) forces the samples in Class 2 far away from the hyperplane f1x=0. Both the distances of Class 1 data and Class 2 data to the hyperplane of Class 1 are measured by capped L1-norm, which reduces the negative influence of noise and outliers, due to the application of L1-norm sum over the “capped” items on two class data. The above two terms make sure that CPSVM classifies two classes well. Furthermore, to give a better generalization performance, CPSVM also minimizes a regularization term, as seen in the second term in the numerator, which helps control the model complexity and avoid the overfitting problem. The optimization problem (10) has a similar geometric meaning as (9). After optimal w¯1 and w¯2 are obtained, w1,b1, and w2,b2 are also obtained accordingly. For a new coming sample x, it is assigned by(11)labelx=argmini=1,2wiTx+biwi.To better understand the effect of the cappedL1-norm, we transform (9) and (10) into their different formulations. Firstly, write the explicative form of (9) and (10) as(12)minw¯1≠0∑i=1N1minw¯1Tx¯1i,ϵ+δw¯12∑j=1N2minw¯1Tx¯1j,ϵ,(13)minw¯2≠0∑i=1N1minw¯2Tx¯2i,ϵ+δw¯22∑j=1N2minw¯2Tx¯2j,ϵ.Further, transform (12) to a GEPSVM-like formulation. Introduce F1i=w¯1Tx¯1i−1⋅Indw¯1Tx¯1i≤ϵ, where Ind⋅ is the indicator function satisfying Indw¯1Tx¯1i≤ϵ=1 if w¯1Tx¯1i≤ϵ, and 0 otherwise, and F1∈ℝm1×m1 as the diagonal matrix with its i,i-th diagonal element F1i, i=1,2,…,m1. Then, the numerator of (12) is equivalent to(14)∑i=1N1F1iw¯1Tx¯1i2+δw¯12+const=w¯1TX¯1F1X¯1Tw¯1+δw¯12+const=w¯1TX¯1F1X¯1T+δIw¯1+const=w¯1TH1w¯1+const,where H1=X¯1F1+δIX¯1T, I∈ℝn+1×n+1 and const is some constant. Similarly, denote G1j=w¯1Tx¯2j−1⋅Indw¯1Tx¯2j≤ϵ, and G1∈ℝm2×m2 as the diagonal matrix with its j,j-th diagonal element G1j, j=1,2,…,m2; then, the denominator of (12) is equivalent to(15)∑i=1N2G1jw¯1Tx¯2j2+const=w¯1TX¯1G1X¯1Tw¯1+const=w¯1TM1w¯1+const,where M1=X¯2G1X¯2T and const is some constant. Therefore, CPSVM (9) is recast as(16)minw¯1≠0w¯1TH1w¯1w¯1TM1w¯1.Similarly, (10) is recast as(17)minw¯1≠0w¯2TH2w¯2w¯2TM2w¯2,where H2=X¯2F2X¯2T, M2=X¯1G2X¯1T, and F2∈ℝm2×m2 and G2∈ℝm1×m1 are defined similarly to F1 and G1.By observing (16) and (17), one sees that CPSVM can be viewed as weighted GEPSVM. A key advantage of these weights in CPSVM is that they improve robustness to outliers and noise. In one respect, the L1-norm carries robustness to noise. In another respect, the data owning a norm larger than ϵ will not receive any further utilization because the corresponding weight is 0, and these data are perceived as outliers. For the data have relatively large norm but smaller than ϵ, small weights are assigned to them, and they do not exhibit much strong importance. This shows that CPSVM is robust to outliers and noise.


### 3.2. The Solving Algorithm of CPSVM

In the following, we solve (16), and (17) can be solved similarly. Formally, (16) is a generalized eigenvalue problem. However, both H1 and M1 depend on F1 and G1, respectively, and hence depend on w1, which can not be solved directly. For fixed H1 and M1, and (16) is a generalized eigenvalue problem. Therefore, to solve this problem, we employ an iterative technique.In specific, we first initializew10=e and w20=e. Then, in the t-th iteration, H1t and M1t are computed by(18)H1t=X¯1F1tX¯1T,M1t=X¯2G1tX¯2T,where the diagonal elements of F1t and G1t are given by(19)F1it=w¯1t−1Tx¯1i−1⋅Indw¯1t−1Tx¯1i≤ϵ,(20)G1jt=w¯1t−1Tx¯2j−1⋅Indw¯1t−1Tx¯2j≤ϵ,where i=1,2,…,m1, j=1,2,…,m2.After obtainingH1t, and M1t, the optimal solution w¯1t is given by the eigenvector corresponding to the smallest nonzero eigenvalue λ of the generalized eigenvalue problem H1tw¯1=λM1tw¯1. After reaching the maximum iteration number or convergence, the optimal w¯1∗ is set to w¯1t. w¯2t is obtained similarly.We summarize the solving procedure of CPSVM in Algorithm 1.Algorithm 1: CPSVM solving algorithm.
Input: Training input data matrix X¯1, X¯2, maximum iteration number Itmax, and parameter ϵ,δ>0.Process:(1)
Initialize iteration number t=0, w¯10=w¯20=e.(2)
RepeatComputeHit and Mit by (18), i=1,2;Computew¯1t and w¯2t by solving Hitw¯i=λMitw¯i, i=1,2;Sett=t+1;Until reaching maximum iteration number Itmax or convergence.Output: w¯1∗=w¯1t+1 and w¯2∗=w¯2t+1.Predict: Assign the class of x according to (11).


## 3.1. Problem Formulation

As we see above, the application of squaredL2-norm metric in GEPSVM amplifies the influence of outliers, which will affect the construction of the optimal classification hyperplanes. Though L1-NPSVM aims at giving a more robust performance on the problem with outliers, it still may perform not very well when encountering large enough outliers or noise. In contrast, capped L1-norm metric that involves an upper bound can effectively control the influence of extreme outliers. Therefore, we propose the following capped L1-norm proximal support vector machine (CPSVM), the optimization problem of which is expressed as(9)minw¯1≠0w¯1TX¯1capL1+δw¯12w¯1TX¯2capL1,(10)minw¯2≠0w¯2TX¯2capL1+δw¯22w¯2TX¯1capL1,where δ>0 is a regularization parameter. Minimizing the first item in the numerator of (9) makes the samples in Class 1 as close as possible to the hyperplane f1x=0. Maximizing the denominator of (9) forces the samples in Class 2 far away from the hyperplane f1x=0. Both the distances of Class 1 data and Class 2 data to the hyperplane of Class 1 are measured by capped L1-norm, which reduces the negative influence of noise and outliers, due to the application of L1-norm sum over the “capped” items on two class data. The above two terms make sure that CPSVM classifies two classes well. Furthermore, to give a better generalization performance, CPSVM also minimizes a regularization term, as seen in the second term in the numerator, which helps control the model complexity and avoid the overfitting problem. The optimization problem (10) has a similar geometric meaning as (9). After optimal w¯1 and w¯2 are obtained, w1,b1, and w2,b2 are also obtained accordingly. For a new coming sample x, it is assigned by(11)labelx=argmini=1,2wiTx+biwi.To better understand the effect of the cappedL1-norm, we transform (9) and (10) into their different formulations. Firstly, write the explicative form of (9) and (10) as(12)minw¯1≠0∑i=1N1minw¯1Tx¯1i,ϵ+δw¯12∑j=1N2minw¯1Tx¯1j,ϵ,(13)minw¯2≠0∑i=1N1minw¯2Tx¯2i,ϵ+δw¯22∑j=1N2minw¯2Tx¯2j,ϵ.Further, transform (12) to a GEPSVM-like formulation. Introduce F1i=w¯1Tx¯1i−1⋅Indw¯1Tx¯1i≤ϵ, where Ind⋅ is the indicator function satisfying Indw¯1Tx¯1i≤ϵ=1 if w¯1Tx¯1i≤ϵ, and 0 otherwise, and F1∈ℝm1×m1 as the diagonal matrix with its i,i-th diagonal element F1i, i=1,2,…,m1. Then, the numerator of (12) is equivalent to(14)∑i=1N1F1iw¯1Tx¯1i2+δw¯12+const=w¯1TX¯1F1X¯1Tw¯1+δw¯12+const=w¯1TX¯1F1X¯1T+δIw¯1+const=w¯1TH1w¯1+const,where H1=X¯1F1+δIX¯1T, I∈ℝn+1×n+1 and const is some constant. Similarly, denote G1j=w¯1Tx¯2j−1⋅Indw¯1Tx¯2j≤ϵ, and G1∈ℝm2×m2 as the diagonal matrix with its j,j-th diagonal element G1j, j=1,2,…,m2; then, the denominator of (12) is equivalent to(15)∑i=1N2G1jw¯1Tx¯2j2+const=w¯1TX¯1G1X¯1Tw¯1+const=w¯1TM1w¯1+const,where M1=X¯2G1X¯2T and const is some constant. Therefore, CPSVM (9) is recast as(16)minw¯1≠0w¯1TH1w¯1w¯1TM1w¯1.Similarly, (10) is recast as(17)minw¯1≠0w¯2TH2w¯2w¯2TM2w¯2,where H2=X¯2F2X¯2T, M2=X¯1G2X¯1T, and F2∈ℝm2×m2 and G2∈ℝm1×m1 are defined similarly to F1 and G1.By observing (16) and (17), one sees that CPSVM can be viewed as weighted GEPSVM. A key advantage of these weights in CPSVM is that they improve robustness to outliers and noise. In one respect, the L1-norm carries robustness to noise. In another respect, the data owning a norm larger than ϵ will not receive any further utilization because the corresponding weight is 0, and these data are perceived as outliers. For the data have relatively large norm but smaller than ϵ, small weights are assigned to them, and they do not exhibit much strong importance. This shows that CPSVM is robust to outliers and noise.


## 3.2. The Solving Algorithm of CPSVM

In the following, we solve (16), and (17) can be solved similarly. Formally, (16) is a generalized eigenvalue problem. However, both H1 and M1 depend on F1 and G1, respectively, and hence depend on w1, which can not be solved directly. For fixed H1 and M1, and (16) is a generalized eigenvalue problem. Therefore, to solve this problem, we employ an iterative technique.In specific, we first initializew10=e and w20=e. Then, in the t-th iteration, H1t and M1t are computed by(18)H1t=X¯1F1tX¯1T,M1t=X¯2G1tX¯2T,where the diagonal elements of F1t and G1t are given by(19)F1it=w¯1t−1Tx¯1i−1⋅Indw¯1t−1Tx¯1i≤ϵ,(20)G1jt=w¯1t−1Tx¯2j−1⋅Indw¯1t−1Tx¯2j≤ϵ,where i=1,2,…,m1, j=1,2,…,m2.After obtainingH1t, and M1t, the optimal solution w¯1t is given by the eigenvector corresponding to the smallest nonzero eigenvalue λ of the generalized eigenvalue problem H1tw¯1=λM1tw¯1. After reaching the maximum iteration number or convergence, the optimal w¯1∗ is set to w¯1t. w¯2t is obtained similarly.We summarize the solving procedure of CPSVM in Algorithm 1.Algorithm 1: CPSVM solving algorithm.
Input: Training input data matrix X¯1, X¯2, maximum iteration number Itmax, and parameter ϵ,δ>0.Process:(1)
Initialize iteration number t=0, w¯10=w¯20=e.(2)
RepeatComputeHit and Mit by (18), i=1,2;Computew¯1t and w¯2t by solving Hitw¯i=λMitw¯i, i=1,2;Sett=t+1;Until reaching maximum iteration number Itmax or convergence.Output: w¯1∗=w¯1t+1 and w¯2∗=w¯2t+1.Predict: Assign the class of x according to (11).


## 4. Experiments

In this section, we experimentally compare the proposed CPSVM with GEPSVM [2], L1-GEPSVM [16], PCC [14], L1-NPSVM [15], IGEPSVM [13] LpNPSVM [12], and GLpNPSVM [21]. Experiments are conducted on an artificial dataset with outliers, some benchmark datasets [34], and a handwritten digit image dataset. All these methods are implemented in MATLAB R2019a environment on a PC with an Intel i5 processor (3.30 GHz) with 4 GB RAM. The parameter δ in GEPSVM, L1-GEPSVM, PCC, IGEPSVM, LpNPSVM, and GLpNPSVM is selected from 10−6,10−5,…,105,106, σ in GLpNPSVM is selected from 10−6,10−5,…,10−1,100, and the learning rate in L1-NPSVM is chosen from the set 0.0005,0.001,0.005,0.01,0.05, as suggested in [15]. The parameter p in LpNPSVM and p and q in GLpNPSVM are selected from 0.1,0.5,1,1.5,2,5.For CPSVM, in real experiments, choosing an appropriateϵ is difficult. In fact, by observing the CPSVM model construction, we see ϵ “capped” the projected data. Therefore, it is not only relevant to the involved data but also should be affected by current w¯1 and w¯2 during each iteration. Thus, it is reasonable to update ϵ during each iteration. Suppose the projected data contain q% outliers; then, we may set ϵ to the 1−q% largest value of the projected data. In this situation, we implicitly use 1−q% data to compute w¯1 or w¯2, so 1−q% is selected from 0.7,0.75,0.8,0.85,0.9,0.95,1.


### 4.1. Artificial Data

We first apply CPSVM to a simple two-dimensional artificial dataset. The artificial dataset contains two classes, with Class 1 and Class 2 having 180 data points, respectively. Class 1 is generated from the liney=5x with variance 2, and Class 2 is generated from the line y=−5x with variance 2, Class 1 is represented by the red “o,” and Class 2 is represented by the blue “o.” 50% of samples of each class are randomly selected for training, and the rest are used for testing. To explore the robustness of the proposed method, 16 noise samples for each class are added to the training data. The noise of Class 1 is represented by the red “Δ,” and the noise of Class 2 is represented by the blue “Δ.” Denote “Ori” as the set that only contains training data, “Noi1” as the set containing original training data and the noise samples of Class 1, and “Noi2” as the set containing original training data noise samples of two classes. fiOrix=0, fiNoi1x=0, and fiNoi2x=0 represent the optimal approximating planes of Class i obtained from the above three kinds of training sets, respectively, where i=1,2. The training data and its outliers are shown in Figure 1(a), and the test data are shown in Figure 1(b).Figure 1
(a) Training data with outliers and (b) test data of the artificial dataset.
(a)(b)We apply CPSVM and GEPSVM, L1-GEPSVM, IGEPSVM, PCC, L1-NPSVM, LpNPSVM, and GLpNPSVM to these three kinds of training sets and obtain their optimal approximating planes, respectively. Table1 shows the test classification accuracy of these three kinds of training sets. Figures 2–4 show approximating planes for “Ori,” “Noi1,” and “Noi2” training data, respectively, and the error test data are also demonstrated. From the results of Table 1, we see the proposed CPSVM possesses the best classification performance for all three cases. To further evaluate these methods, we calculate the average distance ratio of all test points to the approximating plane belonging to their own class and the approximating plane belonging to the other class. Clearly, the smaller the distance, the better. The results are shown in Figure 5, which demonstrate that our CPSVM owns the best generalization ability for noise data compared to other methods.Table 1
Classification results on artificial test data.
MethodGEPSVML1-GEPSVMIGEPSVMPCCL1-NPSVMLpNPSVMGLpNPSVMCPSVMOri94.8995.4594.8995.4594.8994.8994.3295.45Noi194.8995.4594.3294.8994.8995.4595.4595.45Noi295.4595.4594.3294.8995.4595.4594.8995.45Each bold value is the highest accuracy within each line.Figure 2
Classification results on test data for “Ori” training data. (a) GEPSVM. (b) L1-GEPSVM. (c) IGEPSVM. (d) PCC. (e) L1-NPSVM. (f) LpNPSVM. (g) GLpNPSVM. (h) CPSVM.
(a)(b)(c)(d)(e)(f)(g)(h)Figure 3
Classification results on test data for “Noi1” training data. (a) GEPSVM. (b) L1-GEPSVM. (c) IGEPSVM. (d) PCC. (e) L1-NPSVM. (f) LpNPSVM. (g) GLpNPSVM. (h) CPSVM.
(a)(b)(c)(d)(e)(f)(g)(h)Figure 4
Classification results on test data for “Noi2” training data. (a) GEPSVM. (b) L1-GEPSVM. (c) IGEPSVM. (d) PCC. (e) L1-NPSVM. (f) LpNPSVM. (g) GLpNPSVM. (h) CPSVM.
(a)(b)(c)(d)(e)(f)(g)(h)Figure 5
The average distance ratio of all methods on artificial data.


### 4.2. Benchmark UCI Datasets

In this section, the proposed CPSVM and its comparison methods are applied to 18 small-size benchmark datasets and 9 large-size benchmark datasets. 10-fold cross-validation is used for searching optimal parameters, and 10-time average test classification accuracy is adopted for each method.


#### 4.2.1. Small-Size UCI Datasets

This subsection first compares each method on some small-size UCI datasets, whose information is listed in Table2. Classification accuracies along with standard deviations for all methods on these original datasets are listed in Table 3. “Acc” is short for accuracy %, and “Std” is short for standard deviation %. By observing the results in Table 3, we see that the proposed CPSVM outperforms other methods on most of these small-size datasets. On other small-size datasets, CPSVM performs comparably to the one with the highest accuracy for most cases.Table 2
Small-size UCI datasets information.
DatasetSample no.Feature no.BUPA3456Echo13110Heartstatlog27013Heartc30314Diabetics7688Spect26744Hourse30026TicTacToe95827Ionosphere35134Monks34326House _ votes43516Sonar20860Australian69014Haberman3063PimaIndian7688Credit69015Breast6999Hepatitis15519Table 3
Classification results on original small-size UCI datasets.
DatasetGEPSVML1-GEPSVMIGEPSVMPCCL1-NPSVMLpNPSVMGLpNPSVMCPSVMAcc ± stdAcc ± stdAcc ± stdAcc ± stdAcc ± stdAcc ± stdAcc ± stdAcc ± std1−q%BUPA59.99 ± 6.4359.97 ± 5.4966.08 ± 5.8057.96 ± 4.8957.37 ± 5.2063.00 ± 9.2363.68 ± 12.9273.77 ± 10.67 (0.8)Echo81.83 ± 17.6090.22 ± 7.4891.61 ± 9.0680.01 ± 9.0968.24 ± 9.9192.23 ± 7.9389.94 ± 10.7689.34 ± 8.95 (0.75)Heartstatlog82.96 ± 8.0483.70 ± 2.4681.48 ± 6.3066.30 ± 6.8664.81 ± 9.2881.85 ± 7.5083.70 ± 5.8495.93 ± 6.86 (0.85)Heartc88.87 ± 6.0589.48 ± 4.1396.12 ± 3.7476.64 ± 7.5663.97 ± 13.9987.88 ± 5.4789.02 ± 6.6789.17 ± 11.96 (0.95)Diabetics75.91 ± 4.0875.39 ± 3.9374.22 ± 5.4764.72 ± 6.8563.93 ± 7.1675.27 ± 5.7576.17 ± 4.7079.95 ± 5.33 (0.85)Spect79.37 ± 4.8580.03 ± 1.1578.26 ± 4.3079.00 ± 4.4968.59 ± 9.4879.37 ± 4.8578.63 ± 4.7580.88 ± 4.18 (0.9)Hourse80.34 ± 5.8882.78 ± 3.4180.64 ± 5.2667.31 ± 9.1657.95 ± 7.3181.30 ± 5.1279.36 ± 5.0081.68 ± 4.13 (1)TicTacToe91.25 ± 22.2298.33 ± 1.2598.33 ± 1.4167.75 ± 6.1256.90 ± 4.1298.33 ± 1.4198.33 ± 1.4189.79 ± 12.52 (1)Ionosphere72.39 ± 17.5273.30 ± 5.8879.46 ± 6.2866.97 ± 7.2964.40 ± 8.0383.46 ± 6.1784.32 ± 6.0597.12 ± 5.74 (0.8)Monks387.80 ± 6.4467.35 ± 7.0087.29 ± 4.9770.62 ± 6.4554.19 ± 3.7970.40 ± 3.9374.07 ± 4.3284.72 ± 3.26 (0.95)House_votes95.61 ± 2.5972.31 ± 27.8995.17 ± 2.2991.02 ± 3.0068.22 ± 10.4094.93 ± 2.4295.16 ± 2.0994.69 ± 3.49 (0.9)Sonar74.51 ± 8.9777.00 ± 9.5274.05 ± 8.0057.65 ± 9.1259.58 ± 10.1474.84 ± 12.2774.54 ± 7.1286.34 ± 7.96 (0.9)Australian85.67 ± 3.7284.63 ± 3.1884.94 ± 3.6275.11 ± 7.3569.90 ± 9.9086.68 ± 3.9085.38 ± 3.5585.23 ± 3.98 (1)Haberman72.91 ± 10.4375.84 ± 5.8171.26 ± 7.8368.68 ± 10.7769.00 ± 8.2472.23 ± 7.9673.25 ± 8.9176.88 ± 12.04 (0.85)PimaIndian76.69 ± 5.5575.27 ± 4.2574.35 ± 3.7764.98 ± 4.5764.43 ± 7.1874.61 ± 5.9474.99 ± 5.7077.06 ± 6.17 (0.95)Credit84.64 ± 6.2385.65 ± 3.4385.37 ± 5.2573.00 ± 9.8562.00 ± 8.4186.38 ± 4.2884.93 ± 4.5782.02 ± 8.31 (1)Breast95.28 ± 10.0595.57 ± 3.2291.14 ± 6.4892.99 ± 2.3869.39 ± 6.9591.85 ± 5.9095.72 ± 7.0596.14 ± 6.34 (0.8)Hepatitis79.17 ± 0.3181.34 ± 3.0778.63 ± 0.7979.38 ± 0.0360.75 ± 3.3981.25 ± 0.7183.21 ± 0.3189.08 ± 0.30 (0.9)Average81.399580.453982.688972.227463.534281.992182.466886.0995In order to test the robustness of the proposed CPSVM, we consider the noise-contaminated data. Specifically, random 30% features of random 10%, 20%, and 30% percent samples are assigned with 0. The classification results on noise data are listed in Tables4–6.Table 4
Classification results on 10% noise small-size UCI datasets.
DatasetGEPSVML1-GEPSVMIGEPSVMPCCL1-NPSVMLpNPSVMGLpNPSVMCPSVMAcc ± stdAcc ± stdAcc ± stdAcc ± stdAcc ± stdAcc ± stdAcc ± stdAcc ± std1−q%BUPA60.35 ± 6.7661.15 ± 5.0667.89 ± 5.9058.38 ± 6.0558.53 ± 5.8362.77 ± 10.1967.55 ± 9.1974.66 ± 8.04 (0.85)Echo86.53 ± 9.6388.49 ± 7.9492.23 ± 7.9381.65 ± 14.2469.07 ± 10.8592.23 ± 7.9385.72 ± 11.6591.39 ± 5.62 (0.7)Heartstatlog83.70 ± 7.2484.07 ± 3.3381.85 ± 7.9074.44 ± 10.5471.11 ± 11.0182.22 ± 7.9682.59 ± 8.7493.70 ± 4.64 (0.8)Heartc89.12 ± 4.7388.22 ± 3.9690.19 ± 5.0678.20 ± 11.9766.27 ± 12.1689.21 ± 5.0189.78 ± 5.4093.08 ± 10.48 (0.95)Diabetics75.91 ± 4.1375.25 ± 3.7774.22 ± 5.4765.37 ± 5.7963.29 ± 7.2075.40 ± 5.5674.87 ± 4.3580.06 ± 5.78 (0.8)Spect79.37 ± 4.8579.35 ± 1.4379.37 ± 4.8579.37 ± 4.8567.35 ± 7.3679.37 ± 4.8577.85 ± 6.4780.88 ± 4.18 (0.95)Hourse79.06 ± 5.6280.78 ± 7.6680.32 4.0469.62 ± 5.4958.99 ± 8.0180.97 ± 5.5580.02 ± 4.8382.36 ± 4.32 (0.95)TicTacToe75.52 ± 30.3698.33 ± 0.6898.33 ± 1.4169.33 ± 9.8054.80 ± 3.5698.33 ± 1.4198.33 ± 1.4191.97 ± 7.89 (1)Ionosphere77.77 ± 7.0873.84 ± 8.3976.62 ± 7.3863.54 ± 7.5468.69 ± 7.3783.48 ± 5.4882.62 ± 5.2696.86 ± 6.93 (0.75)Monks368.84 ± 8.7866.69 ± 6.4770.86 ± 4.6363.27 ± 15.6555.73 ± 6.7272.03 ± 6.4974.28 ± 4.5473.85 ± 7.89 (1)House _ votes95.61 ± 2.5995.67 ± 3.7293.32 ± 3.7288.96 ± 5.2565.22 ± 11.9995.61 ± 3.3695.16 ± 2.5896.78 ± 1.60 (1)Sonar73.19 ± 7.8176.86 ± 9.4072.81 ± 9.9258.21 ± 11.9863.89 ± 8.0173.26 ± 11.6276.41 ± 7.2480.56 ± 16.44 (0.7)Australian85.96 ± 3.5885.35 ± 2.8585.39 ± 4.3376.40 ± 5.0161.86 ± 8.8986.82 ± 3.1984.78 ± 3.7784.66 ± 4.10 (1)Haberman73.26 ± 11.2774.52 ± 4.5070.94 ± 9.7873.52 ± 6.8168.32 ± 9.7273.52 ± 6.2873.52 ± 6.5177.87 ± 10.01 (0.75)PimaIndian75.39 ± 3.5375.64 ± 4.9665.63 ± 5.0764.07 ± 4.6764.06 ± 4.6076.56 ± 4.1775.64 ± 5.6576.41 ± 5.72 (1)Credit85.07 ± 5.6483.79 ± 3.8485.81 ± 5.4878.85 ± 3.6059.28 ± 4.7186.38 ± 4.8983.48 ± 4.4384.48 ± 8.72 (1)Breast95.14 ± 1.5395.14 ± 3.0890.85 ± 2.4093.41 ± 5.1763.79 ± 4.1391.84 ± 3.7196.00 ± 2.4096.29 ± 1.59 (0.7)Hepatitis78.46 ± 12.9881.92 ± 4.8777.38 ± 4.8180.00 ± 1.8662.75 ± 8.7478.04 ± 8.8683.88 ± 4.5988.96 ± 7.30 (0.8)Average79.902881.393380.778373.143963.500082.113382.360085.8233Each bold value is the highest accuracy within each line.Table 5
Classification results on 20% noise small-size UCI datasets.
DatasetGEPSVML1-GEPSVMIGEPSVMPCCL1-NPSVMLpNPSVMGLpNPSVMCPSVMAcc ± stdAcc ± stdAcc ± stdAcc ± stdAcc ± stdAcc ± stdAcc ± stdAcc ± std1−q%BUPA57.96 ± 3.9561.12 ± 7.7152.06 ± 6.8457.96 ± 4.8957.37 ± 5.0256.70 ± 5.8855.89 ± 5.9760.16 ± 7.96 (0.7)Echo85.98 ± 9.2889.45 ± 6.0085.98 ± 9.2883.64 ± 9.0970.75 ± 9.5985.98 ± 9.2881.18 ± 13.5794.51 ± 6.76 (0.75)Heartstatlog82.96 ± 7.8583.33 ± 5.5682.59 ± 5.8070.37 ± 10.6258.52 ± 5.1875.19 ± 6.9983.33 ± 4.3692.22 ± 8.63 (0.75)Heartc87.56 ± 6.8787.86 ± 5.3490.47 ± 5.2470.38 ± 16.1960.77 ± 11.3487.20 ± 5.8489.46 ± 4.3992.22 ± 9.50 (0.9)Diabetics73.44 ± 3.7773.45 ± 4.0765.89 ± 5.6764.45 ± 5.6463.54 ± 7.6073.96 ± 4.8074.35 ± 4.2279.30 ± 5.48 (0.8)Spect79.37 ± 4.8579.12 ± 1.6679.00 ± 4.4979.37 ± 4.8571.92 ± 11.7078.99 ± 5.2179.00 ± 5.1279.76 ± 4.43 (0.7)Hourse82.35 ± 6.7082.33 ± 5.7880.97 ± 5.5564.67 ± 10.7160.66 ± 6.0279.96 ± 5.3579.99 ± 5.0182.71 ± 7.07 (1)TicTacToe69.84 ± 4.8268.12 ± 1.0172.14 ± 4.8665.66 ± 7.9557.62 ± 5.4466.92 ± 4.5473.18 ± 4.9773.80 ± 5.56 (1)Ionosphere62.23 ± 25.8473.24 ± 9.1275.23 ± 8.5064.13 ± 6.0761.80 ± 7.2582.36 ± 5.8082.08 ± 8.0097.45 ± 5.76 (0.8)Monks388.92 ± 4.8766.47 ± 9.6182.76 ± 9.5966.39 ± 9.7458.60 ± 5.6871.34 ± 5.0174.78 ± 4.8384.95 ± 4.62 (0.8)House _ votes88.27 ± 3.3581.64 ± 15.2789.88 ± 3.5083.88 ± 6.0358.45 ± 7.5187.82 ± 4.4687.82 ± 3.7589.91 ± 4.32 (0.9)Sonar73.42 ± 9.2473.14 ± 9.4071.75 ± 10.2755.18 ± 10.4461.49 ± 8.7673.50 ± 10.1472.86 ± 9.0192.96 ± 9.70 (0.75)Australian76.38 ± 4.7884.34 ± 3.5475.95 ± 5.3971.76 ± 8.6859.86 ± 7.1274.80 ± 5.0773.77 ± 3.6574.36 ± 3.76 (1)Haberman74.90 ± 8.3374.55 ± 5.8473.19 ± 6.1473.84 ± 6.2267.98 ± 8.3973.52 ± 6.2870.70 ± 14.2575.25 ± 9.74 (0.8)PimaIndian75.00 ± 5.2874.74 ± 4.6465.63 ± 5.2264.06 ± 4.8464.46 ± 4.7474.48 ± 4.1575.12 ± 5.5278.90 ± 4.32 (0.9)Credit85.50 ± 5.2583.76 ± 4.8585.66 ± 5.2976.09 ± 6.5560.00 ± 7.9685.79 ± 3.4284.20 ± 5.9387.09 ± 6.52 (1)Breast95.57 ± 2.1794.71 ± 2.8689.99 ± 2.5596.29 ± 2.5168.12 ± 6.8195.00 ± 2.1595.43 ± 2.3095.57 ± 2.24 (0.7)Hepatitis81.83 ± 6.3380.58 ± 5.5380.04 ± 5.4476.75 ± 4.6558.63 ± 7.7881.25 ± 6.6983.83 ± 6.4188.33 ± 5.66 (0.75)Average78.971178.442177.732271.381762.252278.042278.720684.4139Each bold value is the highest accuracy within each line.Table 6
Classification results on 30% noise small-size UCI datasets.
DatasetGEPSVML1-GEPSVMIGEPSVMPCCL1-NPSVMLpNPSVMGLpNPSVMCPSVMAcc stdAcc stdAcc stdAcc stdAcc stdAcc stdAcc stdAcc std1−q%BUPA56.24 ± 6.4158.79 ± 2.5157.44 ± 5.7857.96 ± 4.8957.21 ± 6.0458.59 ± 5.4653.16 ± 7.9662.90 ± 7.91 (0.75)Echo86.98 ± 9.1088.30 ± 5.6692.23 ± 7.9383.82 ± 12.1069.63 ± 10.9392.23 ± 7.9387.72 ± 10.0990.11 ± 11.90 (0.75)Heartstatlog81.85 ± 7.0881.85 ± 5.3582.22 ± 7.7772.59 ± 13.9959.26 ± 7.4182.22 ± 8.3484.07 ± 7.6289.63 ± 8.15 (0.7)Heartc88.01 ± 4.4087.46 ± 4.1390.04 ± 5.7265.91 ± 8.8259.98 ± 6.7489.72 ± 5.2588.75 ± 5.2497.64 ± 1.64 (0.75)Diabetics66.01 ± 6.5367.18 ± 3.5065.11 ± 6.0265.38 ± 6.4462.38 ± 5.1166.54 ± 6.8666.01 ± 6.7972.39 ± 6.41 (0.85)Spect79.00 ± 4.4978.83 ± 1.4679.37 ± 4.8579.37 ± 4.8572.25 ± 8.9279.36 ± 5.5574.83 ± 9.7379.76 ± 4.43 (0.8)Hourse80.01 ± 5.9480.15 ± 5.6680.68 ± 5.9767.38 ± 14.0860.33 ± 7.1381.31 ± 4.2780.97 ± 4.0081.65 ± 4.28 (0.95)TicTacToe71.49 ± 30.2767.92 ± 1.2398.33 ± 1.4168.18 ± 5.5455.53 ± 4.1998.33 ± 1.4198.33 ± 1.5798.33 ± 1.41 (1)Ionosphere73.79 ± 17.8572.43 ± 6.4581.18 ± 5.7966.42 ± 9.8565.48 ± 9.4581.79 ± 5.2682.93 ± 6.5898.57 ± 3.09 (0.8)Monks388.69 ± 4.9466.66 ± 4.9082.29 ± 10.0565.53 ± 8.1253.94 ± 2.7971.55 ± 3.8672.69 ± 6.0684.97 ± 6.34 (0.9)House _ votes95.61 ± 2.5995.66 ± 2.3892.86 ± 3.3885.46 ± 8.4263.53 ± 9.8995.61 ± 3.0095.38 ± 2.7097.00 ± 2.18 (0.75)Sonar76.28 ± 8.6777.00 ± 9.5271.79 ± 10.4453.67 ± 9.1559.32 ± 5.6375.22 ± 11.8674.06 ± 10.8888.83 ± 15.47 (0.75)Australian85.67 ± 3.6085.51 ± 3.8785.53 ± 3.4576.83 ± 5.0556.51 ± 5.0784.51 ± 3.6382.89 ± 7.5387.26 ± 2.97 (1)Haberman74.25 ± 8.9373.16 ± 7.5173.52 ± 6.2873.52 ± 6.2873.85 ± 6.3673.52 ± 6.2871.94 ± 5.5477.15 ± 7.79 (0.75)PimaIndian75.91 ± 4.4176.05 ± 4.4265.63 ± 5.0765.11 ± 4.0761.59 ± 5.6576.95 ± 4.2775.91 ± 4.3278.51 ± 5.12 (0.85)Credit86.08 ± 5.6583.20 ± 3.1585.22 ± 5.7080.73 ± 8.6957.83 ± 6.5283.63 ± 6.0984.93 ± 4.8985.64 ± 6.39 (1)Breast93.42 ± 2.3893.99 ± 1.9384.12 ± 2.0692.29 ± 5.0469.07 ± 6.7192.42 ± 3.5093.99 ± 3.9494.13 ± 2.67 (0.75)Hepatitis79.83 ± 7.5778.65 ± 2.1079.96 ± 8.5278.75 ± 6.4558.83 ± 7.3677.88 ± 13.0885.13 ± 8.2387.08 ± 7.09 (0.75)Average79.951178.487680.417872.161162.028981.187880.760686.1972Each bold value is the highest accuracy within each line.By observing the results in these tables, we can see that on small-size datasets, we have the following: (i) the performance of all methods declines after adding noise, and the higher the noise percentage, the worse the performance; (ii) compared with other methods, the proposed CPSVM is less affected by noise and is better than other methods on most datasets; (iii) the optimal1−q% may be different for different levels of outliers. In addition, more outliers may require a small 1−q%, because 1−q% is set to exclude the q percent outliers. On the data without noise and outliers, the percentage of 1−q% should be close to 1; (iv) the proposed CPSVM can deal with small-size data and noisy datasets well. To better compare various methods, we compute the rank of each method according to their accuracies, and the results are shown in Table 7. In the table, for each data and each method, four numbers represent the ranks of the method on this original data and corresponding three types of noise data. For example, 5/5/3/6 for BUPA data and classifier GEPSVM in the table means that GEPSVM has rank 5 on original BUPA data and has ranks 5, 3, and 6 on three noise BUPA data. It can be seen that CPSVM has the highest rank on most datasets and has certain comparability on other datasets. In addition, CPSVM has a higher average rank in the case of noise than that of the original data, which shows the robustness of CPSVM.Table 7
Classification ranks on original and noise small-size UCI datasets.
DatasetGEPSVML1-GEPSVMIGEPSVMPCCL1-NPSVMLpNPSVMGLpNPSVMCPSVMRankRankRankRankRankRankRankRankBUPA5/6/3/76/5/1/22/2/8/57/8/4/48/7/5/64/4/6/33/3/7/81/1/2/1Echo6/5/3/63/4/2/42/1/4/17/7/6/78/8/8/81/2/5/24/6/7/55/3/1/3Heartstatlog4/3/4/52/2/2/66/6/5/37/7/7/78/8/8/85/5/6/43/4/3/21/1/1/1Heartc5/5/5/52/6/4/61/2/2/27/7/7/78/8/8/86/4/6/34/3/3/43/1/1/1Diabetics3/2/5/44/4/4/26/6/6/77/7/7/68/8/8/85/3/3/32/5/2/51/1/1/1Spect3/2/2/52/6/4/67/3/5/25/4/3/38/8/8/84/5/7/46/7/6/71/1/1/1Hourse5/6/2/61/3/3/54/4/4/47/7/7/78/8/8/83/2/6/26/5/5/32/1/1/1TicTacToe5/6/4/51/1/5/72/2/3/17/7/7/68/8/8/83/3/6/24/4/2/36/5/1/4Ionosphere6/4/7/55/6/5/64/5/4/47/8/6/78/7/8/83/2/2/32/3/3/21/1/1/1Monks31/5/1/17/6/6/62/4/3/35/7/7/78/8/8/86/3/5/54/1/4/43/2/2/2House _ votes1/3/3/37/2/7/22/6/2/66/7/6/78/8/8/84/4/4/43/5/5/55/1/1/1Sonar5/5/3/32/2/4/26/6/6/68/8/8/87/7/7/73/4/2/44/3/5/51/1/1/1Australian2/2/2/26/4/1/45/3/3/37/7/7/78/8/8/81/1/4/53/5/6/64/6/5/1Haberman4/6/2/22/2/3/76/7/6/48/3/4/57/8/8/35/4/5/63/5/7/81/1/1/1PimaIndian2/5/3/43/3/4/36/6/6/67/7/8/78/8/7/85/1/5/24/4/2/51/2/1/1Credit5/3/4/12/5/6/63/2/3/37/7/7/78/8/8/81/1/2/54/6/5/46/4/1/2Breast4/3/2/43/4/6/27/7/7/75/5/1/68/8/8/86/6/5/52/2/4/31/1/3/1Hepatitis6/5/3/43/3/5/67/7/6/35/4/7/58/8/8/84/6/4/72/2/2/21/1/1/1Average4.0/4.2/3.4/3.8/4.3/4.4/6.6/6.5/7.9/7.8/3.8/3.3/3.5/4.1/2.4/1.9/3.2/4.04.0/4.64.6/3.96.1/6.37.7/7.64.6/3.84.3/4.51.4/1.4Each bold value is the highest accuracy within each line.


#### 4.2.2. Large-Size UCI Datasets

To see the performance of the proposed CPSVM on relatively large-size data, we then test the performance of these methods on large-size UCI data shown in Table8. The classification accuracies along with standard deviations for all methods on original data are listed in Table 9. By observing the results in Table 9, we see that the proposed CPSVM outperforms other methods on some of the datasets. On other datasets, CPSVM performs comparably to the classier with the highest accuracy.Table 8
Large-size UCI datasets information.
DatasetSample no.Feature no.HTRU2178988Page-blocks547210Spanbas460157Yeast14848Occupancy205605Titanic22013Hypothyroid316325Waveform500021Two-norm740020Table 9
Classification results on original large-size UCI datasets.
DatasetGEPSVML1-GEPSVMIGEPSVMPCCL1-NPSVMLpNPSVMGLpNPSVMCPSVMAcc ± stdAcc ± stdAcc ± stdAcc ± stdAcc ± stdAcc ± stdAcc ± stdAcc ± std1−q%HTRU297.36 ± 0.3295.26 ± 0.5794.22 ± 0.3292.34 ± 0.1979.13 ± 13.2297.35 ± 0.2897.01 ± 0.2696.54 ± 0.31 (0.95)Page-blocks94.34 ± 0.8594.02 ± 0.6590.04 ± 0.2289.78 ± 0.0587.77 ± 7.6792.32 ± 1.0994.21 ± 0.7894.00 ± 1.04 (0.9)Spanbas76.42 ± 1.3782.13 ± 2.9788.55 ± 1.5160.66 ± 0.1458.03 ± 2.7587.50 ± 0.9587.69 ± 1.9688.87 ± 1.62 (0.8)Yeast90.70 ± 3.2688.96 ± 4.5990.90 ± 1.0189.02 ± 0.3089.02 ± 0.3091.44 ± 2.8391.73 ± 1.3293.28 ± 1.18 (0.85)Occupancy98.75 ± 0.3197.68 ± 0.4183.11 ± 0.7976.91 ± 0.0378.34 ± 3.3998.64 ± 0.7198.70 ± 0.2797.22 ± 0.30 (0.9)Titanic77.69 ± 3.9277.60 ± 3.0377.51 ± 3.7865.42 ± 5.5766.24 ± 6.1777.60 ± 3.8578.01 ± 3.6973.15 ± 4.75 (0.8)Hypothyroid95.35 ± 1.6695.23 ± 0.0995.19 ± 1.4795.23 ± 1.4495.23 ± 1.4496.23 ± 0.6796.04 ± 0.7095.23 ± 1.44 (1)Waveform82.41 ± 1.8178.12 ± 0.9684.82 ± 1.4366.86 ± 1.8663.76 ± 5.4982.56 ± 1.7983.89 ± 1.9983.42 ± 1.44 (1)Two-norm97.82 ± 0.3897.76 ± 0.6197.78 ± 0.3450.59 ± 3.0553.15 ± 2.7396.62 ± 0.6397.61 ± 0.3397.74 ± 0.55 (0.9)Average90.093389.640489.124476.312274.518991.140091.654491.0500Each bold value is the highest accuracy within each line.To further test the robustness of the proposed CPSVM on large-size datasets, we consider the noise-contaminated data the same as on the small-size datasets. The classification results on these noise data are listed in Tables10–12. The corresponding accuracy ranks are shown in Table 13.Table 10
Classification results on 10% noise large-size UCI datasets.
DatasetGEPSVML1-GEPSVMIGEPSVMPCCL1-NPSVMLpNPSVMGLpNPSVMCPSVMAcc ± stdAcc ± stdAcc ± stdAcc ± stdAcc ± stdAcc ± stdAcc ± stdAcc ± std1−q%HTRU297.20 ± 0.4095.63 ± 0.7993.97 ± 0.2691.70 ± 0.1486.76 ± 8.3796.93 ± 0.2096.79 ± 0.4297.10 ± 0.29 (0.85)Page-blocks92.82 ± 1.0093.91 ± 0.7790.04 ± 0.2489.78 ± 0.0586.83 ± 7.2392.18 ± 0.5493.89 ± 0.8394.15 ± 0.93 (0.9)Spanbas77.31 ± 1.4679.16 ± 2.2489.07 ± 1.5160.44 ± 0.3357.77 ± 2.3785.02 ± 2.4587.04 ± 2.1287.22 ± 0.79 (0.75)Yeast90.43 ± 2.9586.08 ± 8.6589.02 ± 0.3089.02 ± 0.3088.95 ± 0.3390.97 ± 0.8691.20 ± 2.1493.06 ± 0.81 (0.75)Occupancy98.75 ± 0.3197.77 ± 0.3583.11 ± 0.7976.91 ± 0.0380.56 ± 6.0398.36 ± 0.1698.37 ± 0.6197.39 ± 0.21 (0.9)Titanic69.92 ± 2.2576.46 ± 1.9266.97 ± 5.1267.70 ± 2.7161.56 ± 8.9877.70 ± 2.7173.65 ± 10.7475.23 ± 4.04 (0.9)Hypothyroid95.83 ± 1.8695.20 ± 0.1294.97 ± 1.3695.23 ± 1.4495.23 ± 1.4496.20 ± 0.7296.23 ± 0.6595.23 ± 1.44 (0.9)Waveform82.66 ± 1.8378.22 ± 1.9083.08 ± 1.6466.86 ± 1.8665.20 ± 3.1780.33 ± 0.6083.10 ± 1.4784.08 ± 2.32 (1)Two-norm97.11 ± 0.4596.97 ± 0.3597.04 ± 0.3850.66 ± 3.1052.96 ± 2.4294.73 ± 0.3896.97 ± 0.3997.22 ± 0.54 (0.8)Average89.114488.821887.474476.477875.091190.268990.804491.1867Each bold value is the highest accuracy within each line.Table 11
Classification results on 20% noise large-size UCI datasets.
DatasetGEPSVML1-GEPSVMIGEPSVMPCCL1-NPSVMLpNPSVMGLpNPSVMCPSVMAcc ± stdAcc ± stdAcc ± stdAcc ± stdAcc ± stdAcc ± stdAcc ± stdAcc ± std1−q%HTRU296.90 ± 0.3191.46 ± 1.2094.33 ± 0.3092.21 ± 0.1681.90 ± 11.7796.79 ± 0.2196.69 ± 0.4596.93 ± 0.28 (0.8)Page-blocks92.77 ± 0.8993.84 ± 0.7789.95 ± 0.1789.78 ± 0.0586.53 ± 12.6191.48 ± 0.3492.66 ± 0.3993.97 ± 0.98 (0.85)Spanbas75.96 ± 2.4378.72 ± 2.6188.76 ± 1.1060.62 ± 0.1259.29 ± 1.0385.38 ± 0.5083.93 ± 3.9286.36 ± 2.38 (0.75)Yeast90.63 ± 3.3685.52 ± 8.9189.02 ± 0.3089.02 ± 0.3089.02 ± 0.3090.65 ± 0.8090.37 ± 2.1093.81 ± 1.31 (0.8)Occupancy96.55 ± 0.3697.24 ± 0.3578.99 ± 0.3676.90 ± 0.0286.43 ± 8.3395.71 ± 1.8096.42 ± 4.3296.95 ± 0.37 (0.8)Titanic77.69 ± 3.9275.72 ± 3.2177.51 ± 3.7866.97 ± 2.9467.60 ± 7.9977.60 ± 3.8577.29 ± 4.2277.03 ± 3.86 (0.9)Hypothyroid95.73 ± 1.8995.20 ± 0.1295.26 ± 1.4995.23 ± 1.4488.50 ± 10.3796.20 ± 0.6796.07 ± 0.8296.14 ± 0.45 (0.8)Waveform82.97 ± 1.6078.32 ± 1.4282.81 ± 1.7066.86 ± 1.8661.69 ± 6.1179.98 ± 0.7679.70 ± 2.0584.21 ± 1.13 (0.95)Two-norm96.35 ± 0.6196.42 ± 0.5596.32 ± 0.6650.04 ± 2.7353.50 ± 2.7894.31 ± 0.3695.04 ± 1.7596.58 ± 0.63 (0.7)Average89.505688.047988.105676.403374.940089.788989.796791.3311Each bold value is the highest accuracy within each line.Table 12
Classification results on 30% noise large-size UCI datasets.
DatasetGEPSVML1-GEPSVMIGEPSVMPCCL1-NPSVMLpNPSVMGLpNPSVMCPSVMAcc ± stdAcc ± stdAcc ± stdAcc ± stdAcc ± stdAcc ± stdAcc ± stdAcc ± std1−q%HTRU296.87 ± 0.3594.98 ± 0.5495.13 ± 0.3091.47 ± 0.1588.98 ± 3.5996.80 ± 0.3496.61 ± 0.2796.90 ± 0.31 (0.75)Page-blocks92.66 ± 0.8893.64 ± 0.7189.91 ± 0.1689.78 ± 0.0588.94 ± 4.4891.94 ± 1.1092.64 ± 0.3593.90 ± 0.93 (0.7)Spanbas72.92 ± 1.7877.27 ± 6.485.09 ± 1.7960.60 ± 0.0856.82 ± 4.1377.59 ± 1.0283.99 ± 2.4585.37 ± 2.83 (0.75)Yeast88.88 ± 3.5683.95 ± 11.8489.02 ± 0.3089.02 ± 0.3088.14 ± 1.7787.60 ± 2.9587.99 ± 0.8993.34 ± 0.82 (0.85)Occupancy84.23 ± 0.7384.57 ± 0.9676.87 ± 0.7376.16 ± 0.3276.95 ± 0.5484.35 ± 0.9284.52 ± 1.0286.20 ± 0.71 (0.7)Titanic77.60 ± 3.8573.97 ± 3.377.60 ± 3.8577.60 ± 3.8569.24 ± 4.9767.70 ± 2.7172.28 ± 8.3477.60 ± 3.85 (0.85)Hypothyroid93.55 ± 0.9495.23 ± 0.0995.23 ± 1.5395.23 ± 1.4491.50 ± 4.2995.94 ± 0.8295.91 ± 0.7095.80 ± 0.98 (0.7)Waveform76.17 ± 1.5577.34 ± 1.3175.78 ± 1.2066.86 ± 1.8664.73 ± 4.1075.79 ± 1.2176.61 ± 1.2677.73 ± 1.53 (0.9)Two-norm95.31 ± 0.6695.43 ± 0.8595.35 ± 0.8150.04 ± 2.7353.72 ± 4.1293.53 ± 1.3294.26 ± 0.3195.53 ± 0.74 (0.7)Average86.465686.265586.664477.417875.446785.693387.201189.1522Each bold value is the highest accuracy within each line.Table 13
Classification ranks on original and noise large-size UCI datasets.
DatasetGEPSVML1-GEPSVMIGEPSVMPCCL1-NPSVMLpNPSVMGLpNPSVMCPSVMRankRankRankRankRankRankRankRankHTRU21/1/2/25/5/7/66/6/5/57/7/6/78/8/8/82/3/3/33/4/4/44/2/1/1Page-blocks1/4/3/33/2/2/26/6/6/67/7/7/78/8/8/85/5/5/52/3/4/44/1/1/1Spanbas6/6/6/65/5/5/52/1/1/27/7/7/78/8/8/84/4/3/43/3/4/31/2/2/1Yeast5/4/3/48/8/8/84/5/5/26/6/6/37/7/7/53/3/2/72/2/4/61/1/1/1Occupancy1/1/3/54/4/1/26/6/7/78/8/8/87/7/6/63/3/5/42/2/4/35/5/2/1Titanic2/5/1/13/2/6/55/7/3/28/6/8/37/8/7/74/1/2/81/4/4/66/3/5/4Hypothyroid3/3/4/74/7/7/48/8/5/55/4/6/66/5/8/81/2/1/12/1/3/27/6/2/3Waveform5/4/2/46/6/6/21/3/3/67/7/7/78/8/8/84/5/4/52/2/5/33/1/1/1Two-norm1/2/3/43/4/2/22/3/4/38/8/8/87/7/7/76/6/6/65/5/5/54/1/1/1Average2.8/3.3/4.6/4.8/4.4/5.0/7.0/6.7/7.3/7.3/3.6/3.6/2.4/2.9/3.9/2.4/3.0/4.04.9/4.04.3/4.27/6.27.4/7.23.4/4.84.1/4.01.8/1.6By observing the results in these tables, we can see that on relatively large-size datasets, CPSVM has similar behavior compared to small-size datasets. In specific, noise has an influence on all methods. However, CPSVM is influenced less compared to other methods. In addition, the optimal 1 −q% still plays an important role in CPSVM, which may vary on different data and different levels of noise. The results in Table 13 confirm the superiority of the proposed method.


### 4.3. MNIST Handwritten Dataset

In this subsection, we apply our proposed CPSVM to MNIST handwritten symbol recognition. To test the robustness, we also consider polluting the data by adding black or Gaussian rectangular block to training data. The MNIST database consists of grayscale images of handwritten digits from 0 to 9, and the size of each image is 16× 16 pixels with 256 gray levels, as shown in Figure 6(a). We choose three pairs of numbers (0 vs 6, 0 vs 9, and 6 vs 9) for our comparisons because these groups of numbers are similar and not easy to distinguish. Each dataset is randomly divided into two subsets: 50% images for training and 50% images for testing. Then, to generate outliers, we randomly select 10% images from the training set and add black block noise or Gaussian block noise of 10%, 20%, 30%, and 40% area on it. Figure 6(b) shows sample images of black block noise of 30% area outliers, and Figure 6(c) shows sample images of Gaussian block noise of 30% area. These datasets are denoted as “Ori,” “B10,” “B20,” “B30,” “B40,” “G10,” “G20,” “G30,” and “G40,” where 10, 20, 30, and 40 represent the percentage of blocks, and B and G represent black block and Gaussian block, respectively.Figure 6
Training samples from MNIST  data. (a) Original training data. (b) Training data with 30% area black block noise. (c) Training data with 30% area Gaussian block noise.
(a)(b)(c)The recognition results of all methods on these datasets are listed in Tables14–19. For the black block noise case, the results of L1-GEPSVM, IGEPSVM, LpNPSVM, GLpNPSVM, and CPSVM are relatively close, while they are all not better than CPSVM for most cases. When gradually increasing the area of black block noise, L1-GEPSVM, IGEPSVM, LpNPSVM, and GLpNPSVM are affected by noise more seriously than CPSVM, which shows the robustness of CPSVM. For the Gaussian noise experiment, the performance of CPSVM is the best or is comparable to the best method for most cases. In addition, with the increase of noise level, the performance of CPSVM is stable, while the noise has a greater impact on other methods.Table 14
Classification results on 0 vs 6 data with black block noise.
DatasetGEPSVML1-GEPSVMIGEPSVMPCCL1-NPSVMLpNPSVMGLpNPSVMCPSVMAccAccAccAccAccAccAccAcc1−q%Ori92.0697.5897.7492.1469.0693.0698.4999.00 (0.9)B1094.9897.1697.6692.6464.7298.7599.0098.91 (0.9)B2089.9796.7496.8391.6561.9098.3396.9198.25 (0.85)B3091.8295.5897.6692.2466.6196.9996.7498.08 (0.85)B4094.1395.2296.9092.7173.7696.996.0698.08 (0.7)Each bold value is the highest accuracy within each line.Table 15
Classification results on 0 vs 9 data with black block noise.
DatasetGEPSVML1-GEPSVMIGEPSVMPCCL1-NPSVMLpNPSVMGLpNPSVMCPSVMAccAccAccAccAccAccAccAcc1−q%Ori98.4098.3299.1696.6260.9099.5899.1699.58 (0.9)B1097.7298.1699.1696.7177.8799.7599.0799.58 (0.85)B2096.8097.7599.0796.2190.6693.8699.1699.41 (0.75)B3098.8297.4998.8296.2083.0495.8699.1699.41 (0.7)B4094.3197.7597.5796.8280.3391.4699.0899.41 (0.75)Each bold value is the highest accuracy within each line.Table 16
Classification results on 6 vs 9 data with black block noise.
DatasetGEPSVML1-GEPSVMIGEPSVMPCCL1-NPSVMLpNPSVMGLpNPSVMCPSVMAccAccAccAccAccAccAccAcc1−q%Ori92.6994.1597.2182.0565.2998.0198.2798.01 (0.95)B1092.1593.9397.6178.1986.8497.8797.2198.54 (0.85)B2084.7493.4396.4578.9581.9795.9296.8497.24 (0.85)B3093.0493.2797.1176.7790.4296.0696.7297.24 (0.7)B4093.4793.0496.4879.7766.8495.3096.3497.24 (0.7)Each bold value is the highest accuracy within each line.Table 17
Classification results on 0 vs 6 data with Gaussian block noise.
DatasetGEPSVML1-GEPSVMIGEPSVMPCCL1-NPSVMLpNPSVMGLpNPSVMCPSVMAccAccAccAccAccAccAccAcc1−q%Ori97.4998.4196.1692.4074.5299.0899.0899.16 (0.85)G1097.6697.4995.9192.1581.5498.3399.0098.75 (0.85)G2097.91970795.9992.1570.3497.7498.7599.08 (0.8)G3097.0897.1695.9192.1572.8597.9998.9198.91 (0.75)G4097.0895.5795.9192.2371.0997.9198.8398.91 (0.7)Each bold value is the highest accuracy within each line.Table 18
Classification results on 0 vs 9 data with Gaussian block noise.
DatasetGEPSVML1-GEPSVMIGEPSVMPCCL1-NPSVMLpNPSVMGLpNPSVMCPSVMAccAccAccAccAccAccAccAcc1−q%Ori99.2498.1698.5796.9887.3299.5099.6699.50 (1)G1098.6698.3398.4996.8976.6699.2499.5899.50 (0.9)G2098.8298.1698.4096.8991.0298.9998.1599.41 (0.85)G3098.3298.1698.3296.8164.8298.7498.2499.16 (0.75)G4098.3298.0898.4096.8184.0598.1598.3299.16 (0.75)Table 19
Classification results on 6 vs 9 data with Gaussian block noise.
DatasetGEPSVML1-GEPSVMIGEPSVMPCCL1-NPSVMLpNPSVMGLpNPSVMCPSVMAccAccAccAccAccAccAccAcc1−q%Ori95.0597.3996.7482.5373.5395.1898.7098.04 (1)G1095.7097.0096.8781.6280.4493.4897.7898.17 (0.95)G2094.1396.4896.7482.2761.5494.0097.7898.04 (0.85)G3094.7896.3596.7481.7572.2392.0596.6197.39 (0.8)G4093.8796.2296.3582.6679.4093.8796.2297.39 (0.7)Each bold value is the highest accuracy within each line.


## 4.1. Artificial Data

We first apply CPSVM to a simple two-dimensional artificial dataset. The artificial dataset contains two classes, with Class 1 and Class 2 having 180 data points, respectively. Class 1 is generated from the liney=5x with variance 2, and Class 2 is generated from the line y=−5x with variance 2, Class 1 is represented by the red “o,” and Class 2 is represented by the blue “o.” 50% of samples of each class are randomly selected for training, and the rest are used for testing. To explore the robustness of the proposed method, 16 noise samples for each class are added to the training data. The noise of Class 1 is represented by the red “Δ,” and the noise of Class 2 is represented by the blue “Δ.” Denote “Ori” as the set that only contains training data, “Noi1” as the set containing original training data and the noise samples of Class 1, and “Noi2” as the set containing original training data noise samples of two classes. fiOrix=0, fiNoi1x=0, and fiNoi2x=0 represent the optimal approximating planes of Class i obtained from the above three kinds of training sets, respectively, where i=1,2. The training data and its outliers are shown in Figure 1(a), and the test data are shown in Figure 1(b).Figure 1
(a) Training data with outliers and (b) test data of the artificial dataset.
(a)(b)We apply CPSVM and GEPSVM, L1-GEPSVM, IGEPSVM, PCC, L1-NPSVM, LpNPSVM, and GLpNPSVM to these three kinds of training sets and obtain their optimal approximating planes, respectively. Table1 shows the test classification accuracy of these three kinds of training sets. Figures 2–4 show approximating planes for “Ori,” “Noi1,” and “Noi2” training data, respectively, and the error test data are also demonstrated. From the results of Table 1, we see the proposed CPSVM possesses the best classification performance for all three cases. To further evaluate these methods, we calculate the average distance ratio of all test points to the approximating plane belonging to their own class and the approximating plane belonging to the other class. Clearly, the smaller the distance, the better. The results are shown in Figure 5, which demonstrate that our CPSVM owns the best generalization ability for noise data compared to other methods.Table 1
Classification results on artificial test data.
MethodGEPSVML1-GEPSVMIGEPSVMPCCL1-NPSVMLpNPSVMGLpNPSVMCPSVMOri94.8995.4594.8995.4594.8994.8994.3295.45Noi194.8995.4594.3294.8994.8995.4595.4595.45Noi295.4595.4594.3294.8995.4595.4594.8995.45Each bold value is the highest accuracy within each line.Figure 2
Classification results on test data for “Ori” training data. (a) GEPSVM. (b) L1-GEPSVM. (c) IGEPSVM. (d) PCC. (e) L1-NPSVM. (f) LpNPSVM. (g) GLpNPSVM. (h) CPSVM.
(a)(b)(c)(d)(e)(f)(g)(h)Figure 3
Classification results on test data for “Noi1” training data. (a) GEPSVM. (b) L1-GEPSVM. (c) IGEPSVM. (d) PCC. (e) L1-NPSVM. (f) LpNPSVM. (g) GLpNPSVM. (h) CPSVM.
(a)(b)(c)(d)(e)(f)(g)(h)Figure 4
Classification results on test data for “Noi2” training data. (a) GEPSVM. (b) L1-GEPSVM. (c) IGEPSVM. (d) PCC. (e) L1-NPSVM. (f) LpNPSVM. (g) GLpNPSVM. (h) CPSVM.
(a)(b)(c)(d)(e)(f)(g)(h)Figure 5
The average distance ratio of all methods on artificial data.


## 4.2. Benchmark UCI Datasets

In this section, the proposed CPSVM and its comparison methods are applied to 18 small-size benchmark datasets and 9 large-size benchmark datasets. 10-fold cross-validation is used for searching optimal parameters, and 10-time average test classification accuracy is adopted for each method.


### 4.2.1. Small-Size UCI Datasets

This subsection first compares each method on some small-size UCI datasets, whose information is listed in Table2. Classification accuracies along with standard deviations for all methods on these original datasets are listed in Table 3. “Acc” is short for accuracy %, and “Std” is short for standard deviation %. By observing the results in Table 3, we see that the proposed CPSVM outperforms other methods on most of these small-size datasets. On other small-size datasets, CPSVM performs comparably to the one with the highest accuracy for most cases.Table 2
Small-size UCI datasets information.
DatasetSample no.Feature no.BUPA3456Echo13110Heartstatlog27013Heartc30314Diabetics7688Spect26744Hourse30026TicTacToe95827Ionosphere35134Monks34326House _ votes43516Sonar20860Australian69014Haberman3063PimaIndian7688Credit69015Breast6999Hepatitis15519Table 3
Classification results on original small-size UCI datasets.
DatasetGEPSVML1-GEPSVMIGEPSVMPCCL1-NPSVMLpNPSVMGLpNPSVMCPSVMAcc ± stdAcc ± stdAcc ± stdAcc ± stdAcc ± stdAcc ± stdAcc ± stdAcc ± std1−q%BUPA59.99 ± 6.4359.97 ± 5.4966.08 ± 5.8057.96 ± 4.8957.37 ± 5.2063.00 ± 9.2363.68 ± 12.9273.77 ± 10.67 (0.8)Echo81.83 ± 17.6090.22 ± 7.4891.61 ± 9.0680.01 ± 9.0968.24 ± 9.9192.23 ± 7.9389.94 ± 10.7689.34 ± 8.95 (0.75)Heartstatlog82.96 ± 8.0483.70 ± 2.4681.48 ± 6.3066.30 ± 6.8664.81 ± 9.2881.85 ± 7.5083.70 ± 5.8495.93 ± 6.86 (0.85)Heartc88.87 ± 6.0589.48 ± 4.1396.12 ± 3.7476.64 ± 7.5663.97 ± 13.9987.88 ± 5.4789.02 ± 6.6789.17 ± 11.96 (0.95)Diabetics75.91 ± 4.0875.39 ± 3.9374.22 ± 5.4764.72 ± 6.8563.93 ± 7.1675.27 ± 5.7576.17 ± 4.7079.95 ± 5.33 (0.85)Spect79.37 ± 4.8580.03 ± 1.1578.26 ± 4.3079.00 ± 4.4968.59 ± 9.4879.37 ± 4.8578.63 ± 4.7580.88 ± 4.18 (0.9)Hourse80.34 ± 5.8882.78 ± 3.4180.64 ± 5.2667.31 ± 9.1657.95 ± 7.3181.30 ± 5.1279.36 ± 5.0081.68 ± 4.13 (1)TicTacToe91.25 ± 22.2298.33 ± 1.2598.33 ± 1.4167.75 ± 6.1256.90 ± 4.1298.33 ± 1.4198.33 ± 1.4189.79 ± 12.52 (1)Ionosphere72.39 ± 17.5273.30 ± 5.8879.46 ± 6.2866.97 ± 7.2964.40 ± 8.0383.46 ± 6.1784.32 ± 6.0597.12 ± 5.74 (0.8)Monks387.80 ± 6.4467.35 ± 7.0087.29 ± 4.9770.62 ± 6.4554.19 ± 3.7970.40 ± 3.9374.07 ± 4.3284.72 ± 3.26 (0.95)House_votes95.61 ± 2.5972.31 ± 27.8995.17 ± 2.2991.02 ± 3.0068.22 ± 10.4094.93 ± 2.4295.16 ± 2.0994.69 ± 3.49 (0.9)Sonar74.51 ± 8.9777.00 ± 9.5274.05 ± 8.0057.65 ± 9.1259.58 ± 10.1474.84 ± 12.2774.54 ± 7.1286.34 ± 7.96 (0.9)Australian85.67 ± 3.7284.63 ± 3.1884.94 ± 3.6275.11 ± 7.3569.90 ± 9.9086.68 ± 3.9085.38 ± 3.5585.23 ± 3.98 (1)Haberman72.91 ± 10.4375.84 ± 5.8171.26 ± 7.8368.68 ± 10.7769.00 ± 8.2472.23 ± 7.9673.25 ± 8.9176.88 ± 12.04 (0.85)PimaIndian76.69 ± 5.5575.27 ± 4.2574.35 ± 3.7764.98 ± 4.5764.43 ± 7.1874.61 ± 5.9474.99 ± 5.7077.06 ± 6.17 (0.95)Credit84.64 ± 6.2385.65 ± 3.4385.37 ± 5.2573.00 ± 9.8562.00 ± 8.4186.38 ± 4.2884.93 ± 4.5782.02 ± 8.31 (1)Breast95.28 ± 10.0595.57 ± 3.2291.14 ± 6.4892.99 ± 2.3869.39 ± 6.9591.85 ± 5.9095.72 ± 7.0596.14 ± 6.34 (0.8)Hepatitis79.17 ± 0.3181.34 ± 3.0778.63 ± 0.7979.38 ± 0.0360.75 ± 3.3981.25 ± 0.7183.21 ± 0.3189.08 ± 0.30 (0.9)Average81.399580.453982.688972.227463.534281.992182.466886.0995In order to test the robustness of the proposed CPSVM, we consider the noise-contaminated data. Specifically, random 30% features of random 10%, 20%, and 30% percent samples are assigned with 0. The classification results on noise data are listed in Tables4–6.Table 4
Classification results on 10% noise small-size UCI datasets.
DatasetGEPSVML1-GEPSVMIGEPSVMPCCL1-NPSVMLpNPSVMGLpNPSVMCPSVMAcc ± stdAcc ± stdAcc ± stdAcc ± stdAcc ± stdAcc ± stdAcc ± stdAcc ± std1−q%BUPA60.35 ± 6.7661.15 ± 5.0667.89 ± 5.9058.38 ± 6.0558.53 ± 5.8362.77 ± 10.1967.55 ± 9.1974.66 ± 8.04 (0.85)Echo86.53 ± 9.6388.49 ± 7.9492.23 ± 7.9381.65 ± 14.2469.07 ± 10.8592.23 ± 7.9385.72 ± 11.6591.39 ± 5.62 (0.7)Heartstatlog83.70 ± 7.2484.07 ± 3.3381.85 ± 7.9074.44 ± 10.5471.11 ± 11.0182.22 ± 7.9682.59 ± 8.7493.70 ± 4.64 (0.8)Heartc89.12 ± 4.7388.22 ± 3.9690.19 ± 5.0678.20 ± 11.9766.27 ± 12.1689.21 ± 5.0189.78 ± 5.4093.08 ± 10.48 (0.95)Diabetics75.91 ± 4.1375.25 ± 3.7774.22 ± 5.4765.37 ± 5.7963.29 ± 7.2075.40 ± 5.5674.87 ± 4.3580.06 ± 5.78 (0.8)Spect79.37 ± 4.8579.35 ± 1.4379.37 ± 4.8579.37 ± 4.8567.35 ± 7.3679.37 ± 4.8577.85 ± 6.4780.88 ± 4.18 (0.95)Hourse79.06 ± 5.6280.78 ± 7.6680.32 4.0469.62 ± 5.4958.99 ± 8.0180.97 ± 5.5580.02 ± 4.8382.36 ± 4.32 (0.95)TicTacToe75.52 ± 30.3698.33 ± 0.6898.33 ± 1.4169.33 ± 9.8054.80 ± 3.5698.33 ± 1.4198.33 ± 1.4191.97 ± 7.89 (1)Ionosphere77.77 ± 7.0873.84 ± 8.3976.62 ± 7.3863.54 ± 7.5468.69 ± 7.3783.48 ± 5.4882.62 ± 5.2696.86 ± 6.93 (0.75)Monks368.84 ± 8.7866.69 ± 6.4770.86 ± 4.6363.27 ± 15.6555.73 ± 6.7272.03 ± 6.4974.28 ± 4.5473.85 ± 7.89 (1)House _ votes95.61 ± 2.5995.67 ± 3.7293.32 ± 3.7288.96 ± 5.2565.22 ± 11.9995.61 ± 3.3695.16 ± 2.5896.78 ± 1.60 (1)Sonar73.19 ± 7.8176.86 ± 9.4072.81 ± 9.9258.21 ± 11.9863.89 ± 8.0173.26 ± 11.6276.41 ± 7.2480.56 ± 16.44 (0.7)Australian85.96 ± 3.5885.35 ± 2.8585.39 ± 4.3376.40 ± 5.0161.86 ± 8.8986.82 ± 3.1984.78 ± 3.7784.66 ± 4.10 (1)Haberman73.26 ± 11.2774.52 ± 4.5070.94 ± 9.7873.52 ± 6.8168.32 ± 9.7273.52 ± 6.2873.52 ± 6.5177.87 ± 10.01 (0.75)PimaIndian75.39 ± 3.5375.64 ± 4.9665.63 ± 5.0764.07 ± 4.6764.06 ± 4.6076.56 ± 4.1775.64 ± 5.6576.41 ± 5.72 (1)Credit85.07 ± 5.6483.79 ± 3.8485.81 ± 5.4878.85 ± 3.6059.28 ± 4.7186.38 ± 4.8983.48 ± 4.4384.48 ± 8.72 (1)Breast95.14 ± 1.5395.14 ± 3.0890.85 ± 2.4093.41 ± 5.1763.79 ± 4.1391.84 ± 3.7196.00 ± 2.4096.29 ± 1.59 (0.7)Hepatitis78.46 ± 12.9881.92 ± 4.8777.38 ± 4.8180.00 ± 1.8662.75 ± 8.7478.04 ± 8.8683.88 ± 4.5988.96 ± 7.30 (0.8)Average79.902881.393380.778373.143963.500082.113382.360085.8233Each bold value is the highest accuracy within each line.Table 5
Classification results on 20% noise small-size UCI datasets.
DatasetGEPSVML1-GEPSVMIGEPSVMPCCL1-NPSVMLpNPSVMGLpNPSVMCPSVMAcc ± stdAcc ± stdAcc ± stdAcc ± stdAcc ± stdAcc ± stdAcc ± stdAcc ± std1−q%BUPA57.96 ± 3.9561.12 ± 7.7152.06 ± 6.8457.96 ± 4.8957.37 ± 5.0256.70 ± 5.8855.89 ± 5.9760.16 ± 7.96 (0.7)Echo85.98 ± 9.2889.45 ± 6.0085.98 ± 9.2883.64 ± 9.0970.75 ± 9.5985.98 ± 9.2881.18 ± 13.5794.51 ± 6.76 (0.75)Heartstatlog82.96 ± 7.8583.33 ± 5.5682.59 ± 5.8070.37 ± 10.6258.52 ± 5.1875.19 ± 6.9983.33 ± 4.3692.22 ± 8.63 (0.75)Heartc87.56 ± 6.8787.86 ± 5.3490.47 ± 5.2470.38 ± 16.1960.77 ± 11.3487.20 ± 5.8489.46 ± 4.3992.22 ± 9.50 (0.9)Diabetics73.44 ± 3.7773.45 ± 4.0765.89 ± 5.6764.45 ± 5.6463.54 ± 7.6073.96 ± 4.8074.35 ± 4.2279.30 ± 5.48 (0.8)Spect79.37 ± 4.8579.12 ± 1.6679.00 ± 4.4979.37 ± 4.8571.92 ± 11.7078.99 ± 5.2179.00 ± 5.1279.76 ± 4.43 (0.7)Hourse82.35 ± 6.7082.33 ± 5.7880.97 ± 5.5564.67 ± 10.7160.66 ± 6.0279.96 ± 5.3579.99 ± 5.0182.71 ± 7.07 (1)TicTacToe69.84 ± 4.8268.12 ± 1.0172.14 ± 4.8665.66 ± 7.9557.62 ± 5.4466.92 ± 4.5473.18 ± 4.9773.80 ± 5.56 (1)Ionosphere62.23 ± 25.8473.24 ± 9.1275.23 ± 8.5064.13 ± 6.0761.80 ± 7.2582.36 ± 5.8082.08 ± 8.0097.45 ± 5.76 (0.8)Monks388.92 ± 4.8766.47 ± 9.6182.76 ± 9.5966.39 ± 9.7458.60 ± 5.6871.34 ± 5.0174.78 ± 4.8384.95 ± 4.62 (0.8)House _ votes88.27 ± 3.3581.64 ± 15.2789.88 ± 3.5083.88 ± 6.0358.45 ± 7.5187.82 ± 4.4687.82 ± 3.7589.91 ± 4.32 (0.9)Sonar73.42 ± 9.2473.14 ± 9.4071.75 ± 10.2755.18 ± 10.4461.49 ± 8.7673.50 ± 10.1472.86 ± 9.0192.96 ± 9.70 (0.75)Australian76.38 ± 4.7884.34 ± 3.5475.95 ± 5.3971.76 ± 8.6859.86 ± 7.1274.80 ± 5.0773.77 ± 3.6574.36 ± 3.76 (1)Haberman74.90 ± 8.3374.55 ± 5.8473.19 ± 6.1473.84 ± 6.2267.98 ± 8.3973.52 ± 6.2870.70 ± 14.2575.25 ± 9.74 (0.8)PimaIndian75.00 ± 5.2874.74 ± 4.6465.63 ± 5.2264.06 ± 4.8464.46 ± 4.7474.48 ± 4.1575.12 ± 5.5278.90 ± 4.32 (0.9)Credit85.50 ± 5.2583.76 ± 4.8585.66 ± 5.2976.09 ± 6.5560.00 ± 7.9685.79 ± 3.4284.20 ± 5.9387.09 ± 6.52 (1)Breast95.57 ± 2.1794.71 ± 2.8689.99 ± 2.5596.29 ± 2.5168.12 ± 6.8195.00 ± 2.1595.43 ± 2.3095.57 ± 2.24 (0.7)Hepatitis81.83 ± 6.3380.58 ± 5.5380.04 ± 5.4476.75 ± 4.6558.63 ± 7.7881.25 ± 6.6983.83 ± 6.4188.33 ± 5.66 (0.75)Average78.971178.442177.732271.381762.252278.042278.720684.4139Each bold value is the highest accuracy within each line.Table 6
Classification results on 30% noise small-size UCI datasets.
DatasetGEPSVML1-GEPSVMIGEPSVMPCCL1-NPSVMLpNPSVMGLpNPSVMCPSVMAcc stdAcc stdAcc stdAcc stdAcc stdAcc stdAcc stdAcc std1−q%BUPA56.24 ± 6.4158.79 ± 2.5157.44 ± 5.7857.96 ± 4.8957.21 ± 6.0458.59 ± 5.4653.16 ± 7.9662.90 ± 7.91 (0.75)Echo86.98 ± 9.1088.30 ± 5.6692.23 ± 7.9383.82 ± 12.1069.63 ± 10.9392.23 ± 7.9387.72 ± 10.0990.11 ± 11.90 (0.75)Heartstatlog81.85 ± 7.0881.85 ± 5.3582.22 ± 7.7772.59 ± 13.9959.26 ± 7.4182.22 ± 8.3484.07 ± 7.6289.63 ± 8.15 (0.7)Heartc88.01 ± 4.4087.46 ± 4.1390.04 ± 5.7265.91 ± 8.8259.98 ± 6.7489.72 ± 5.2588.75 ± 5.2497.64 ± 1.64 (0.75)Diabetics66.01 ± 6.5367.18 ± 3.5065.11 ± 6.0265.38 ± 6.4462.38 ± 5.1166.54 ± 6.8666.01 ± 6.7972.39 ± 6.41 (0.85)Spect79.00 ± 4.4978.83 ± 1.4679.37 ± 4.8579.37 ± 4.8572.25 ± 8.9279.36 ± 5.5574.83 ± 9.7379.76 ± 4.43 (0.8)Hourse80.01 ± 5.9480.15 ± 5.6680.68 ± 5.9767.38 ± 14.0860.33 ± 7.1381.31 ± 4.2780.97 ± 4.0081.65 ± 4.28 (0.95)TicTacToe71.49 ± 30.2767.92 ± 1.2398.33 ± 1.4168.18 ± 5.5455.53 ± 4.1998.33 ± 1.4198.33 ± 1.5798.33 ± 1.41 (1)Ionosphere73.79 ± 17.8572.43 ± 6.4581.18 ± 5.7966.42 ± 9.8565.48 ± 9.4581.79 ± 5.2682.93 ± 6.5898.57 ± 3.09 (0.8)Monks388.69 ± 4.9466.66 ± 4.9082.29 ± 10.0565.53 ± 8.1253.94 ± 2.7971.55 ± 3.8672.69 ± 6.0684.97 ± 6.34 (0.9)House _ votes95.61 ± 2.5995.66 ± 2.3892.86 ± 3.3885.46 ± 8.4263.53 ± 9.8995.61 ± 3.0095.38 ± 2.7097.00 ± 2.18 (0.75)Sonar76.28 ± 8.6777.00 ± 9.5271.79 ± 10.4453.67 ± 9.1559.32 ± 5.6375.22 ± 11.8674.06 ± 10.8888.83 ± 15.47 (0.75)Australian85.67 ± 3.6085.51 ± 3.8785.53 ± 3.4576.83 ± 5.0556.51 ± 5.0784.51 ± 3.6382.89 ± 7.5387.26 ± 2.97 (1)Haberman74.25 ± 8.9373.16 ± 7.5173.52 ± 6.2873.52 ± 6.2873.85 ± 6.3673.52 ± 6.2871.94 ± 5.5477.15 ± 7.79 (0.75)PimaIndian75.91 ± 4.4176.05 ± 4.4265.63 ± 5.0765.11 ± 4.0761.59 ± 5.6576.95 ± 4.2775.91 ± 4.3278.51 ± 5.12 (0.85)Credit86.08 ± 5.6583.20 ± 3.1585.22 ± 5.7080.73 ± 8.6957.83 ± 6.5283.63 ± 6.0984.93 ± 4.8985.64 ± 6.39 (1)Breast93.42 ± 2.3893.99 ± 1.9384.12 ± 2.0692.29 ± 5.0469.07 ± 6.7192.42 ± 3.5093.99 ± 3.9494.13 ± 2.67 (0.75)Hepatitis79.83 ± 7.5778.65 ± 2.1079.96 ± 8.5278.75 ± 6.4558.83 ± 7.3677.88 ± 13.0885.13 ± 8.2387.08 ± 7.09 (0.75)Average79.951178.487680.417872.161162.028981.187880.760686.1972Each bold value is the highest accuracy within each line.By observing the results in these tables, we can see that on small-size datasets, we have the following: (i) the performance of all methods declines after adding noise, and the higher the noise percentage, the worse the performance; (ii) compared with other methods, the proposed CPSVM is less affected by noise and is better than other methods on most datasets; (iii) the optimal1−q% may be different for different levels of outliers. In addition, more outliers may require a small 1−q%, because 1−q% is set to exclude the q percent outliers. On the data without noise and outliers, the percentage of 1−q% should be close to 1; (iv) the proposed CPSVM can deal with small-size data and noisy datasets well. To better compare various methods, we compute the rank of each method according to their accuracies, and the results are shown in Table 7. In the table, for each data and each method, four numbers represent the ranks of the method on this original data and corresponding three types of noise data. For example, 5/5/3/6 for BUPA data and classifier GEPSVM in the table means that GEPSVM has rank 5 on original BUPA data and has ranks 5, 3, and 6 on three noise BUPA data. It can be seen that CPSVM has the highest rank on most datasets and has certain comparability on other datasets. In addition, CPSVM has a higher average rank in the case of noise than that of the original data, which shows the robustness of CPSVM.Table 7
Classification ranks on original and noise small-size UCI datasets.
DatasetGEPSVML1-GEPSVMIGEPSVMPCCL1-NPSVMLpNPSVMGLpNPSVMCPSVMRankRankRankRankRankRankRankRankBUPA5/6/3/76/5/1/22/2/8/57/8/4/48/7/5/64/4/6/33/3/7/81/1/2/1Echo6/5/3/63/4/2/42/1/4/17/7/6/78/8/8/81/2/5/24/6/7/55/3/1/3Heartstatlog4/3/4/52/2/2/66/6/5/37/7/7/78/8/8/85/5/6/43/4/3/21/1/1/1Heartc5/5/5/52/6/4/61/2/2/27/7/7/78/8/8/86/4/6/34/3/3/43/1/1/1Diabetics3/2/5/44/4/4/26/6/6/77/7/7/68/8/8/85/3/3/32/5/2/51/1/1/1Spect3/2/2/52/6/4/67/3/5/25/4/3/38/8/8/84/5/7/46/7/6/71/1/1/1Hourse5/6/2/61/3/3/54/4/4/47/7/7/78/8/8/83/2/6/26/5/5/32/1/1/1TicTacToe5/6/4/51/1/5/72/2/3/17/7/7/68/8/8/83/3/6/24/4/2/36/5/1/4Ionosphere6/4/7/55/6/5/64/5/4/47/8/6/78/7/8/83/2/2/32/3/3/21/1/1/1Monks31/5/1/17/6/6/62/4/3/35/7/7/78/8/8/86/3/5/54/1/4/43/2/2/2House _ votes1/3/3/37/2/7/22/6/2/66/7/6/78/8/8/84/4/4/43/5/5/55/1/1/1Sonar5/5/3/32/2/4/26/6/6/68/8/8/87/7/7/73/4/2/44/3/5/51/1/1/1Australian2/2/2/26/4/1/45/3/3/37/7/7/78/8/8/81/1/4/53/5/6/64/6/5/1Haberman4/6/2/22/2/3/76/7/6/48/3/4/57/8/8/35/4/5/63/5/7/81/1/1/1PimaIndian2/5/3/43/3/4/36/6/6/67/7/8/78/8/7/85/1/5/24/4/2/51/2/1/1Credit5/3/4/12/5/6/63/2/3/37/7/7/78/8/8/81/1/2/54/6/5/46/4/1/2Breast4/3/2/43/4/6/27/7/7/75/5/1/68/8/8/86/6/5/52/2/4/31/1/3/1Hepatitis6/5/3/43/3/5/67/7/6/35/4/7/58/8/8/84/6/4/72/2/2/21/1/1/1Average4.0/4.2/3.4/3.8/4.3/4.4/6.6/6.5/7.9/7.8/3.8/3.3/3.5/4.1/2.4/1.9/3.2/4.04.0/4.64.6/3.96.1/6.37.7/7.64.6/3.84.3/4.51.4/1.4Each bold value is the highest accuracy within each line.


### 4.2.2. Large-Size UCI Datasets

To see the performance of the proposed CPSVM on relatively large-size data, we then test the performance of these methods on large-size UCI data shown in Table8. The classification accuracies along with standard deviations for all methods on original data are listed in Table 9. By observing the results in Table 9, we see that the proposed CPSVM outperforms other methods on some of the datasets. On other datasets, CPSVM performs comparably to the classier with the highest accuracy.Table 8
Large-size UCI datasets information.
DatasetSample no.Feature no.HTRU2178988Page-blocks547210Spanbas460157Yeast14848Occupancy205605Titanic22013Hypothyroid316325Waveform500021Two-norm740020Table 9
Classification results on original large-size UCI datasets.
DatasetGEPSVML1-GEPSVMIGEPSVMPCCL1-NPSVMLpNPSVMGLpNPSVMCPSVMAcc ± stdAcc ± stdAcc ± stdAcc ± stdAcc ± stdAcc ± stdAcc ± stdAcc ± std1−q%HTRU297.36 ± 0.3295.26 ± 0.5794.22 ± 0.3292.34 ± 0.1979.13 ± 13.2297.35 ± 0.2897.01 ± 0.2696.54 ± 0.31 (0.95)Page-blocks94.34 ± 0.8594.02 ± 0.6590.04 ± 0.2289.78 ± 0.0587.77 ± 7.6792.32 ± 1.0994.21 ± 0.7894.00 ± 1.04 (0.9)Spanbas76.42 ± 1.3782.13 ± 2.9788.55 ± 1.5160.66 ± 0.1458.03 ± 2.7587.50 ± 0.9587.69 ± 1.9688.87 ± 1.62 (0.8)Yeast90.70 ± 3.2688.96 ± 4.5990.90 ± 1.0189.02 ± 0.3089.02 ± 0.3091.44 ± 2.8391.73 ± 1.3293.28 ± 1.18 (0.85)Occupancy98.75 ± 0.3197.68 ± 0.4183.11 ± 0.7976.91 ± 0.0378.34 ± 3.3998.64 ± 0.7198.70 ± 0.2797.22 ± 0.30 (0.9)Titanic77.69 ± 3.9277.60 ± 3.0377.51 ± 3.7865.42 ± 5.5766.24 ± 6.1777.60 ± 3.8578.01 ± 3.6973.15 ± 4.75 (0.8)Hypothyroid95.35 ± 1.6695.23 ± 0.0995.19 ± 1.4795.23 ± 1.4495.23 ± 1.4496.23 ± 0.6796.04 ± 0.7095.23 ± 1.44 (1)Waveform82.41 ± 1.8178.12 ± 0.9684.82 ± 1.4366.86 ± 1.8663.76 ± 5.4982.56 ± 1.7983.89 ± 1.9983.42 ± 1.44 (1)Two-norm97.82 ± 0.3897.76 ± 0.6197.78 ± 0.3450.59 ± 3.0553.15 ± 2.7396.62 ± 0.6397.61 ± 0.3397.74 ± 0.55 (0.9)Average90.093389.640489.124476.312274.518991.140091.654491.0500Each bold value is the highest accuracy within each line.To further test the robustness of the proposed CPSVM on large-size datasets, we consider the noise-contaminated data the same as on the small-size datasets. The classification results on these noise data are listed in Tables10–12. The corresponding accuracy ranks are shown in Table 13.Table 10
Classification results on 10% noise large-size UCI datasets.
DatasetGEPSVML1-GEPSVMIGEPSVMPCCL1-NPSVMLpNPSVMGLpNPSVMCPSVMAcc ± stdAcc ± stdAcc ± stdAcc ± stdAcc ± stdAcc ± stdAcc ± stdAcc ± std1−q%HTRU297.20 ± 0.4095.63 ± 0.7993.97 ± 0.2691.70 ± 0.1486.76 ± 8.3796.93 ± 0.2096.79 ± 0.4297.10 ± 0.29 (0.85)Page-blocks92.82 ± 1.0093.91 ± 0.7790.04 ± 0.2489.78 ± 0.0586.83 ± 7.2392.18 ± 0.5493.89 ± 0.8394.15 ± 0.93 (0.9)Spanbas77.31 ± 1.4679.16 ± 2.2489.07 ± 1.5160.44 ± 0.3357.77 ± 2.3785.02 ± 2.4587.04 ± 2.1287.22 ± 0.79 (0.75)Yeast90.43 ± 2.9586.08 ± 8.6589.02 ± 0.3089.02 ± 0.3088.95 ± 0.3390.97 ± 0.8691.20 ± 2.1493.06 ± 0.81 (0.75)Occupancy98.75 ± 0.3197.77 ± 0.3583.11 ± 0.7976.91 ± 0.0380.56 ± 6.0398.36 ± 0.1698.37 ± 0.6197.39 ± 0.21 (0.9)Titanic69.92 ± 2.2576.46 ± 1.9266.97 ± 5.1267.70 ± 2.7161.56 ± 8.9877.70 ± 2.7173.65 ± 10.7475.23 ± 4.04 (0.9)Hypothyroid95.83 ± 1.8695.20 ± 0.1294.97 ± 1.3695.23 ± 1.4495.23 ± 1.4496.20 ± 0.7296.23 ± 0.6595.23 ± 1.44 (0.9)Waveform82.66 ± 1.8378.22 ± 1.9083.08 ± 1.6466.86 ± 1.8665.20 ± 3.1780.33 ± 0.6083.10 ± 1.4784.08 ± 2.32 (1)Two-norm97.11 ± 0.4596.97 ± 0.3597.04 ± 0.3850.66 ± 3.1052.96 ± 2.4294.73 ± 0.3896.97 ± 0.3997.22 ± 0.54 (0.8)Average89.114488.821887.474476.477875.091190.268990.804491.1867Each bold value is the highest accuracy within each line.Table 11
Classification results on 20% noise large-size UCI datasets.
DatasetGEPSVML1-GEPSVMIGEPSVMPCCL1-NPSVMLpNPSVMGLpNPSVMCPSVMAcc ± stdAcc ± stdAcc ± stdAcc ± stdAcc ± stdAcc ± stdAcc ± stdAcc ± std1−q%HTRU296.90 ± 0.3191.46 ± 1.2094.33 ± 0.3092.21 ± 0.1681.90 ± 11.7796.79 ± 0.2196.69 ± 0.4596.93 ± 0.28 (0.8)Page-blocks92.77 ± 0.8993.84 ± 0.7789.95 ± 0.1789.78 ± 0.0586.53 ± 12.6191.48 ± 0.3492.66 ± 0.3993.97 ± 0.98 (0.85)Spanbas75.96 ± 2.4378.72 ± 2.6188.76 ± 1.1060.62 ± 0.1259.29 ± 1.0385.38 ± 0.5083.93 ± 3.9286.36 ± 2.38 (0.75)Yeast90.63 ± 3.3685.52 ± 8.9189.02 ± 0.3089.02 ± 0.3089.02 ± 0.3090.65 ± 0.8090.37 ± 2.1093.81 ± 1.31 (0.8)Occupancy96.55 ± 0.3697.24 ± 0.3578.99 ± 0.3676.90 ± 0.0286.43 ± 8.3395.71 ± 1.8096.42 ± 4.3296.95 ± 0.37 (0.8)Titanic77.69 ± 3.9275.72 ± 3.2177.51 ± 3.7866.97 ± 2.9467.60 ± 7.9977.60 ± 3.8577.29 ± 4.2277.03 ± 3.86 (0.9)Hypothyroid95.73 ± 1.8995.20 ± 0.1295.26 ± 1.4995.23 ± 1.4488.50 ± 10.3796.20 ± 0.6796.07 ± 0.8296.14 ± 0.45 (0.8)Waveform82.97 ± 1.6078.32 ± 1.4282.81 ± 1.7066.86 ± 1.8661.69 ± 6.1179.98 ± 0.7679.70 ± 2.0584.21 ± 1.13 (0.95)Two-norm96.35 ± 0.6196.42 ± 0.5596.32 ± 0.6650.04 ± 2.7353.50 ± 2.7894.31 ± 0.3695.04 ± 1.7596.58 ± 0.63 (0.7)Average89.505688.047988.105676.403374.940089.788989.796791.3311Each bold value is the highest accuracy within each line.Table 12
Classification results on 30% noise large-size UCI datasets.
DatasetGEPSVML1-GEPSVMIGEPSVMPCCL1-NPSVMLpNPSVMGLpNPSVMCPSVMAcc ± stdAcc ± stdAcc ± stdAcc ± stdAcc ± stdAcc ± stdAcc ± stdAcc ± std1−q%HTRU296.87 ± 0.3594.98 ± 0.5495.13 ± 0.3091.47 ± 0.1588.98 ± 3.5996.80 ± 0.3496.61 ± 0.2796.90 ± 0.31 (0.75)Page-blocks92.66 ± 0.8893.64 ± 0.7189.91 ± 0.1689.78 ± 0.0588.94 ± 4.4891.94 ± 1.1092.64 ± 0.3593.90 ± 0.93 (0.7)Spanbas72.92 ± 1.7877.27 ± 6.485.09 ± 1.7960.60 ± 0.0856.82 ± 4.1377.59 ± 1.0283.99 ± 2.4585.37 ± 2.83 (0.75)Yeast88.88 ± 3.5683.95 ± 11.8489.02 ± 0.3089.02 ± 0.3088.14 ± 1.7787.60 ± 2.9587.99 ± 0.8993.34 ± 0.82 (0.85)Occupancy84.23 ± 0.7384.57 ± 0.9676.87 ± 0.7376.16 ± 0.3276.95 ± 0.5484.35 ± 0.9284.52 ± 1.0286.20 ± 0.71 (0.7)Titanic77.60 ± 3.8573.97 ± 3.377.60 ± 3.8577.60 ± 3.8569.24 ± 4.9767.70 ± 2.7172.28 ± 8.3477.60 ± 3.85 (0.85)Hypothyroid93.55 ± 0.9495.23 ± 0.0995.23 ± 1.5395.23 ± 1.4491.50 ± 4.2995.94 ± 0.8295.91 ± 0.7095.80 ± 0.98 (0.7)Waveform76.17 ± 1.5577.34 ± 1.3175.78 ± 1.2066.86 ± 1.8664.73 ± 4.1075.79 ± 1.2176.61 ± 1.2677.73 ± 1.53 (0.9)Two-norm95.31 ± 0.6695.43 ± 0.8595.35 ± 0.8150.04 ± 2.7353.72 ± 4.1293.53 ± 1.3294.26 ± 0.3195.53 ± 0.74 (0.7)Average86.465686.265586.664477.417875.446785.693387.201189.1522Each bold value is the highest accuracy within each line.Table 13
Classification ranks on original and noise large-size UCI datasets.
DatasetGEPSVML1-GEPSVMIGEPSVMPCCL1-NPSVMLpNPSVMGLpNPSVMCPSVMRankRankRankRankRankRankRankRankHTRU21/1/2/25/5/7/66/6/5/57/7/6/78/8/8/82/3/3/33/4/4/44/2/1/1Page-blocks1/4/3/33/2/2/26/6/6/67/7/7/78/8/8/85/5/5/52/3/4/44/1/1/1Spanbas6/6/6/65/5/5/52/1/1/27/7/7/78/8/8/84/4/3/43/3/4/31/2/2/1Yeast5/4/3/48/8/8/84/5/5/26/6/6/37/7/7/53/3/2/72/2/4/61/1/1/1Occupancy1/1/3/54/4/1/26/6/7/78/8/8/87/7/6/63/3/5/42/2/4/35/5/2/1Titanic2/5/1/13/2/6/55/7/3/28/6/8/37/8/7/74/1/2/81/4/4/66/3/5/4Hypothyroid3/3/4/74/7/7/48/8/5/55/4/6/66/5/8/81/2/1/12/1/3/27/6/2/3Waveform5/4/2/46/6/6/21/3/3/67/7/7/78/8/8/84/5/4/52/2/5/33/1/1/1Two-norm1/2/3/43/4/2/22/3/4/38/8/8/87/7/7/76/6/6/65/5/5/54/1/1/1Average2.8/3.3/4.6/4.8/4.4/5.0/7.0/6.7/7.3/7.3/3.6/3.6/2.4/2.9/3.9/2.4/3.0/4.04.9/4.04.3/4.27/6.27.4/7.23.4/4.84.1/4.01.8/1.6By observing the results in these tables, we can see that on relatively large-size datasets, CPSVM has similar behavior compared to small-size datasets. In specific, noise has an influence on all methods. However, CPSVM is influenced less compared to other methods. In addition, the optimal 1 −q% still plays an important role in CPSVM, which may vary on different data and different levels of noise. The results in Table 13 confirm the superiority of the proposed method.


## 4.2.1. Small-Size UCI Datasets

This subsection first compares each method on some small-size UCI datasets, whose information is listed in Table2. Classification accuracies along with standard deviations for all methods on these original datasets are listed in Table 3. “Acc” is short for accuracy %, and “Std” is short for standard deviation %. By observing the results in Table 3, we see that the proposed CPSVM outperforms other methods on most of these small-size datasets. On other small-size datasets, CPSVM performs comparably to the one with the highest accuracy for most cases.Table 2
Small-size UCI datasets information.
DatasetSample no.Feature no.BUPA3456Echo13110Heartstatlog27013Heartc30314Diabetics7688Spect26744Hourse30026TicTacToe95827Ionosphere35134Monks34326House _ votes43516Sonar20860Australian69014Haberman3063PimaIndian7688Credit69015Breast6999Hepatitis15519Table 3
Classification results on original small-size UCI datasets.
DatasetGEPSVML1-GEPSVMIGEPSVMPCCL1-NPSVMLpNPSVMGLpNPSVMCPSVMAcc ± stdAcc ± stdAcc ± stdAcc ± stdAcc ± stdAcc ± stdAcc ± stdAcc ± std1−q%BUPA59.99 ± 6.4359.97 ± 5.4966.08 ± 5.8057.96 ± 4.8957.37 ± 5.2063.00 ± 9.2363.68 ± 12.9273.77 ± 10.67 (0.8)Echo81.83 ± 17.6090.22 ± 7.4891.61 ± 9.0680.01 ± 9.0968.24 ± 9.9192.23 ± 7.9389.94 ± 10.7689.34 ± 8.95 (0.75)Heartstatlog82.96 ± 8.0483.70 ± 2.4681.48 ± 6.3066.30 ± 6.8664.81 ± 9.2881.85 ± 7.5083.70 ± 5.8495.93 ± 6.86 (0.85)Heartc88.87 ± 6.0589.48 ± 4.1396.12 ± 3.7476.64 ± 7.5663.97 ± 13.9987.88 ± 5.4789.02 ± 6.6789.17 ± 11.96 (0.95)Diabetics75.91 ± 4.0875.39 ± 3.9374.22 ± 5.4764.72 ± 6.8563.93 ± 7.1675.27 ± 5.7576.17 ± 4.7079.95 ± 5.33 (0.85)Spect79.37 ± 4.8580.03 ± 1.1578.26 ± 4.3079.00 ± 4.4968.59 ± 9.4879.37 ± 4.8578.63 ± 4.7580.88 ± 4.18 (0.9)Hourse80.34 ± 5.8882.78 ± 3.4180.64 ± 5.2667.31 ± 9.1657.95 ± 7.3181.30 ± 5.1279.36 ± 5.0081.68 ± 4.13 (1)TicTacToe91.25 ± 22.2298.33 ± 1.2598.33 ± 1.4167.75 ± 6.1256.90 ± 4.1298.33 ± 1.4198.33 ± 1.4189.79 ± 12.52 (1)Ionosphere72.39 ± 17.5273.30 ± 5.8879.46 ± 6.2866.97 ± 7.2964.40 ± 8.0383.46 ± 6.1784.32 ± 6.0597.12 ± 5.74 (0.8)Monks387.80 ± 6.4467.35 ± 7.0087.29 ± 4.9770.62 ± 6.4554.19 ± 3.7970.40 ± 3.9374.07 ± 4.3284.72 ± 3.26 (0.95)House_votes95.61 ± 2.5972.31 ± 27.8995.17 ± 2.2991.02 ± 3.0068.22 ± 10.4094.93 ± 2.4295.16 ± 2.0994.69 ± 3.49 (0.9)Sonar74.51 ± 8.9777.00 ± 9.5274.05 ± 8.0057.65 ± 9.1259.58 ± 10.1474.84 ± 12.2774.54 ± 7.1286.34 ± 7.96 (0.9)Australian85.67 ± 3.7284.63 ± 3.1884.94 ± 3.6275.11 ± 7.3569.90 ± 9.9086.68 ± 3.9085.38 ± 3.5585.23 ± 3.98 (1)Haberman72.91 ± 10.4375.84 ± 5.8171.26 ± 7.8368.68 ± 10.7769.00 ± 8.2472.23 ± 7.9673.25 ± 8.9176.88 ± 12.04 (0.85)PimaIndian76.69 ± 5.5575.27 ± 4.2574.35 ± 3.7764.98 ± 4.5764.43 ± 7.1874.61 ± 5.9474.99 ± 5.7077.06 ± 6.17 (0.95)Credit84.64 ± 6.2385.65 ± 3.4385.37 ± 5.2573.00 ± 9.8562.00 ± 8.4186.38 ± 4.2884.93 ± 4.5782.02 ± 8.31 (1)Breast95.28 ± 10.0595.57 ± 3.2291.14 ± 6.4892.99 ± 2.3869.39 ± 6.9591.85 ± 5.9095.72 ± 7.0596.14 ± 6.34 (0.8)Hepatitis79.17 ± 0.3181.34 ± 3.0778.63 ± 0.7979.38 ± 0.0360.75 ± 3.3981.25 ± 0.7183.21 ± 0.3189.08 ± 0.30 (0.9)Average81.399580.453982.688972.227463.534281.992182.466886.0995In order to test the robustness of the proposed CPSVM, we consider the noise-contaminated data. Specifically, random 30% features of random 10%, 20%, and 30% percent samples are assigned with 0. The classification results on noise data are listed in Tables4–6.Table 4
Classification results on 10% noise small-size UCI datasets.
DatasetGEPSVML1-GEPSVMIGEPSVMPCCL1-NPSVMLpNPSVMGLpNPSVMCPSVMAcc ± stdAcc ± stdAcc ± stdAcc ± stdAcc ± stdAcc ± stdAcc ± stdAcc ± std1−q%BUPA60.35 ± 6.7661.15 ± 5.0667.89 ± 5.9058.38 ± 6.0558.53 ± 5.8362.77 ± 10.1967.55 ± 9.1974.66 ± 8.04 (0.85)Echo86.53 ± 9.6388.49 ± 7.9492.23 ± 7.9381.65 ± 14.2469.07 ± 10.8592.23 ± 7.9385.72 ± 11.6591.39 ± 5.62 (0.7)Heartstatlog83.70 ± 7.2484.07 ± 3.3381.85 ± 7.9074.44 ± 10.5471.11 ± 11.0182.22 ± 7.9682.59 ± 8.7493.70 ± 4.64 (0.8)Heartc89.12 ± 4.7388.22 ± 3.9690.19 ± 5.0678.20 ± 11.9766.27 ± 12.1689.21 ± 5.0189.78 ± 5.4093.08 ± 10.48 (0.95)Diabetics75.91 ± 4.1375.25 ± 3.7774.22 ± 5.4765.37 ± 5.7963.29 ± 7.2075.40 ± 5.5674.87 ± 4.3580.06 ± 5.78 (0.8)Spect79.37 ± 4.8579.35 ± 1.4379.37 ± 4.8579.37 ± 4.8567.35 ± 7.3679.37 ± 4.8577.85 ± 6.4780.88 ± 4.18 (0.95)Hourse79.06 ± 5.6280.78 ± 7.6680.32 4.0469.62 ± 5.4958.99 ± 8.0180.97 ± 5.5580.02 ± 4.8382.36 ± 4.32 (0.95)TicTacToe75.52 ± 30.3698.33 ± 0.6898.33 ± 1.4169.33 ± 9.8054.80 ± 3.5698.33 ± 1.4198.33 ± 1.4191.97 ± 7.89 (1)Ionosphere77.77 ± 7.0873.84 ± 8.3976.62 ± 7.3863.54 ± 7.5468.69 ± 7.3783.48 ± 5.4882.62 ± 5.2696.86 ± 6.93 (0.75)Monks368.84 ± 8.7866.69 ± 6.4770.86 ± 4.6363.27 ± 15.6555.73 ± 6.7272.03 ± 6.4974.28 ± 4.5473.85 ± 7.89 (1)House _ votes95.61 ± 2.5995.67 ± 3.7293.32 ± 3.7288.96 ± 5.2565.22 ± 11.9995.61 ± 3.3695.16 ± 2.5896.78 ± 1.60 (1)Sonar73.19 ± 7.8176.86 ± 9.4072.81 ± 9.9258.21 ± 11.9863.89 ± 8.0173.26 ± 11.6276.41 ± 7.2480.56 ± 16.44 (0.7)Australian85.96 ± 3.5885.35 ± 2.8585.39 ± 4.3376.40 ± 5.0161.86 ± 8.8986.82 ± 3.1984.78 ± 3.7784.66 ± 4.10 (1)Haberman73.26 ± 11.2774.52 ± 4.5070.94 ± 9.7873.52 ± 6.8168.32 ± 9.7273.52 ± 6.2873.52 ± 6.5177.87 ± 10.01 (0.75)PimaIndian75.39 ± 3.5375.64 ± 4.9665.63 ± 5.0764.07 ± 4.6764.06 ± 4.6076.56 ± 4.1775.64 ± 5.6576.41 ± 5.72 (1)Credit85.07 ± 5.6483.79 ± 3.8485.81 ± 5.4878.85 ± 3.6059.28 ± 4.7186.38 ± 4.8983.48 ± 4.4384.48 ± 8.72 (1)Breast95.14 ± 1.5395.14 ± 3.0890.85 ± 2.4093.41 ± 5.1763.79 ± 4.1391.84 ± 3.7196.00 ± 2.4096.29 ± 1.59 (0.7)Hepatitis78.46 ± 12.9881.92 ± 4.8777.38 ± 4.8180.00 ± 1.8662.75 ± 8.7478.04 ± 8.8683.88 ± 4.5988.96 ± 7.30 (0.8)Average79.902881.393380.778373.143963.500082.113382.360085.8233Each bold value is the highest accuracy within each line.Table 5
Classification results on 20% noise small-size UCI datasets.
DatasetGEPSVML1-GEPSVMIGEPSVMPCCL1-NPSVMLpNPSVMGLpNPSVMCPSVMAcc ± stdAcc ± stdAcc ± stdAcc ± stdAcc ± stdAcc ± stdAcc ± stdAcc ± std1−q%BUPA57.96 ± 3.9561.12 ± 7.7152.06 ± 6.8457.96 ± 4.8957.37 ± 5.0256.70 ± 5.8855.89 ± 5.9760.16 ± 7.96 (0.7)Echo85.98 ± 9.2889.45 ± 6.0085.98 ± 9.2883.64 ± 9.0970.75 ± 9.5985.98 ± 9.2881.18 ± 13.5794.51 ± 6.76 (0.75)Heartstatlog82.96 ± 7.8583.33 ± 5.5682.59 ± 5.8070.37 ± 10.6258.52 ± 5.1875.19 ± 6.9983.33 ± 4.3692.22 ± 8.63 (0.75)Heartc87.56 ± 6.8787.86 ± 5.3490.47 ± 5.2470.38 ± 16.1960.77 ± 11.3487.20 ± 5.8489.46 ± 4.3992.22 ± 9.50 (0.9)Diabetics73.44 ± 3.7773.45 ± 4.0765.89 ± 5.6764.45 ± 5.6463.54 ± 7.6073.96 ± 4.8074.35 ± 4.2279.30 ± 5.48 (0.8)Spect79.37 ± 4.8579.12 ± 1.6679.00 ± 4.4979.37 ± 4.8571.92 ± 11.7078.99 ± 5.2179.00 ± 5.1279.76 ± 4.43 (0.7)Hourse82.35 ± 6.7082.33 ± 5.7880.97 ± 5.5564.67 ± 10.7160.66 ± 6.0279.96 ± 5.3579.99 ± 5.0182.71 ± 7.07 (1)TicTacToe69.84 ± 4.8268.12 ± 1.0172.14 ± 4.8665.66 ± 7.9557.62 ± 5.4466.92 ± 4.5473.18 ± 4.9773.80 ± 5.56 (1)Ionosphere62.23 ± 25.8473.24 ± 9.1275.23 ± 8.5064.13 ± 6.0761.80 ± 7.2582.36 ± 5.8082.08 ± 8.0097.45 ± 5.76 (0.8)Monks388.92 ± 4.8766.47 ± 9.6182.76 ± 9.5966.39 ± 9.7458.60 ± 5.6871.34 ± 5.0174.78 ± 4.8384.95 ± 4.62 (0.8)House _ votes88.27 ± 3.3581.64 ± 15.2789.88 ± 3.5083.88 ± 6.0358.45 ± 7.5187.82 ± 4.4687.82 ± 3.7589.91 ± 4.32 (0.9)Sonar73.42 ± 9.2473.14 ± 9.4071.75 ± 10.2755.18 ± 10.4461.49 ± 8.7673.50 ± 10.1472.86 ± 9.0192.96 ± 9.70 (0.75)Australian76.38 ± 4.7884.34 ± 3.5475.95 ± 5.3971.76 ± 8.6859.86 ± 7.1274.80 ± 5.0773.77 ± 3.6574.36 ± 3.76 (1)Haberman74.90 ± 8.3374.55 ± 5.8473.19 ± 6.1473.84 ± 6.2267.98 ± 8.3973.52 ± 6.2870.70 ± 14.2575.25 ± 9.74 (0.8)PimaIndian75.00 ± 5.2874.74 ± 4.6465.63 ± 5.2264.06 ± 4.8464.46 ± 4.7474.48 ± 4.1575.12 ± 5.5278.90 ± 4.32 (0.9)Credit85.50 ± 5.2583.76 ± 4.8585.66 ± 5.2976.09 ± 6.5560.00 ± 7.9685.79 ± 3.4284.20 ± 5.9387.09 ± 6.52 (1)Breast95.57 ± 2.1794.71 ± 2.8689.99 ± 2.5596.29 ± 2.5168.12 ± 6.8195.00 ± 2.1595.43 ± 2.3095.57 ± 2.24 (0.7)Hepatitis81.83 ± 6.3380.58 ± 5.5380.04 ± 5.4476.75 ± 4.6558.63 ± 7.7881.25 ± 6.6983.83 ± 6.4188.33 ± 5.66 (0.75)Average78.971178.442177.732271.381762.252278.042278.720684.4139Each bold value is the highest accuracy within each line.Table 6
Classification results on 30% noise small-size UCI datasets.
DatasetGEPSVML1-GEPSVMIGEPSVMPCCL1-NPSVMLpNPSVMGLpNPSVMCPSVMAcc stdAcc stdAcc stdAcc stdAcc stdAcc stdAcc stdAcc std1−q%BUPA56.24 ± 6.4158.79 ± 2.5157.44 ± 5.7857.96 ± 4.8957.21 ± 6.0458.59 ± 5.4653.16 ± 7.9662.90 ± 7.91 (0.75)Echo86.98 ± 9.1088.30 ± 5.6692.23 ± 7.9383.82 ± 12.1069.63 ± 10.9392.23 ± 7.9387.72 ± 10.0990.11 ± 11.90 (0.75)Heartstatlog81.85 ± 7.0881.85 ± 5.3582.22 ± 7.7772.59 ± 13.9959.26 ± 7.4182.22 ± 8.3484.07 ± 7.6289.63 ± 8.15 (0.7)Heartc88.01 ± 4.4087.46 ± 4.1390.04 ± 5.7265.91 ± 8.8259.98 ± 6.7489.72 ± 5.2588.75 ± 5.2497.64 ± 1.64 (0.75)Diabetics66.01 ± 6.5367.18 ± 3.5065.11 ± 6.0265.38 ± 6.4462.38 ± 5.1166.54 ± 6.8666.01 ± 6.7972.39 ± 6.41 (0.85)Spect79.00 ± 4.4978.83 ± 1.4679.37 ± 4.8579.37 ± 4.8572.25 ± 8.9279.36 ± 5.5574.83 ± 9.7379.76 ± 4.43 (0.8)Hourse80.01 ± 5.9480.15 ± 5.6680.68 ± 5.9767.38 ± 14.0860.33 ± 7.1381.31 ± 4.2780.97 ± 4.0081.65 ± 4.28 (0.95)TicTacToe71.49 ± 30.2767.92 ± 1.2398.33 ± 1.4168.18 ± 5.5455.53 ± 4.1998.33 ± 1.4198.33 ± 1.5798.33 ± 1.41 (1)Ionosphere73.79 ± 17.8572.43 ± 6.4581.18 ± 5.7966.42 ± 9.8565.48 ± 9.4581.79 ± 5.2682.93 ± 6.5898.57 ± 3.09 (0.8)Monks388.69 ± 4.9466.66 ± 4.9082.29 ± 10.0565.53 ± 8.1253.94 ± 2.7971.55 ± 3.8672.69 ± 6.0684.97 ± 6.34 (0.9)House _ votes95.61 ± 2.5995.66 ± 2.3892.86 ± 3.3885.46 ± 8.4263.53 ± 9.8995.61 ± 3.0095.38 ± 2.7097.00 ± 2.18 (0.75)Sonar76.28 ± 8.6777.00 ± 9.5271.79 ± 10.4453.67 ± 9.1559.32 ± 5.6375.22 ± 11.8674.06 ± 10.8888.83 ± 15.47 (0.75)Australian85.67 ± 3.6085.51 ± 3.8785.53 ± 3.4576.83 ± 5.0556.51 ± 5.0784.51 ± 3.6382.89 ± 7.5387.26 ± 2.97 (1)Haberman74.25 ± 8.9373.16 ± 7.5173.52 ± 6.2873.52 ± 6.2873.85 ± 6.3673.52 ± 6.2871.94 ± 5.5477.15 ± 7.79 (0.75)PimaIndian75.91 ± 4.4176.05 ± 4.4265.63 ± 5.0765.11 ± 4.0761.59 ± 5.6576.95 ± 4.2775.91 ± 4.3278.51 ± 5.12 (0.85)Credit86.08 ± 5.6583.20 ± 3.1585.22 ± 5.7080.73 ± 8.6957.83 ± 6.5283.63 ± 6.0984.93 ± 4.8985.64 ± 6.39 (1)Breast93.42 ± 2.3893.99 ± 1.9384.12 ± 2.0692.29 ± 5.0469.07 ± 6.7192.42 ± 3.5093.99 ± 3.9494.13 ± 2.67 (0.75)Hepatitis79.83 ± 7.5778.65 ± 2.1079.96 ± 8.5278.75 ± 6.4558.83 ± 7.3677.88 ± 13.0885.13 ± 8.2387.08 ± 7.09 (0.75)Average79.951178.487680.417872.161162.028981.187880.760686.1972Each bold value is the highest accuracy within each line.By observing the results in these tables, we can see that on small-size datasets, we have the following: (i) the performance of all methods declines after adding noise, and the higher the noise percentage, the worse the performance; (ii) compared with other methods, the proposed CPSVM is less affected by noise and is better than other methods on most datasets; (iii) the optimal1−q% may be different for different levels of outliers. In addition, more outliers may require a small 1−q%, because 1−q% is set to exclude the q percent outliers. On the data without noise and outliers, the percentage of 1−q% should be close to 1; (iv) the proposed CPSVM can deal with small-size data and noisy datasets well. To better compare various methods, we compute the rank of each method according to their accuracies, and the results are shown in Table 7. In the table, for each data and each method, four numbers represent the ranks of the method on this original data and corresponding three types of noise data. For example, 5/5/3/6 for BUPA data and classifier GEPSVM in the table means that GEPSVM has rank 5 on original BUPA data and has ranks 5, 3, and 6 on three noise BUPA data. It can be seen that CPSVM has the highest rank on most datasets and has certain comparability on other datasets. In addition, CPSVM has a higher average rank in the case of noise than that of the original data, which shows the robustness of CPSVM.Table 7
Classification ranks on original and noise small-size UCI datasets.
DatasetGEPSVML1-GEPSVMIGEPSVMPCCL1-NPSVMLpNPSVMGLpNPSVMCPSVMRankRankRankRankRankRankRankRankBUPA5/6/3/76/5/1/22/2/8/57/8/4/48/7/5/64/4/6/33/3/7/81/1/2/1Echo6/5/3/63/4/2/42/1/4/17/7/6/78/8/8/81/2/5/24/6/7/55/3/1/3Heartstatlog4/3/4/52/2/2/66/6/5/37/7/7/78/8/8/85/5/6/43/4/3/21/1/1/1Heartc5/5/5/52/6/4/61/2/2/27/7/7/78/8/8/86/4/6/34/3/3/43/1/1/1Diabetics3/2/5/44/4/4/26/6/6/77/7/7/68/8/8/85/3/3/32/5/2/51/1/1/1Spect3/2/2/52/6/4/67/3/5/25/4/3/38/8/8/84/5/7/46/7/6/71/1/1/1Hourse5/6/2/61/3/3/54/4/4/47/7/7/78/8/8/83/2/6/26/5/5/32/1/1/1TicTacToe5/6/4/51/1/5/72/2/3/17/7/7/68/8/8/83/3/6/24/4/2/36/5/1/4Ionosphere6/4/7/55/6/5/64/5/4/47/8/6/78/7/8/83/2/2/32/3/3/21/1/1/1Monks31/5/1/17/6/6/62/4/3/35/7/7/78/8/8/86/3/5/54/1/4/43/2/2/2House _ votes1/3/3/37/2/7/22/6/2/66/7/6/78/8/8/84/4/4/43/5/5/55/1/1/1Sonar5/5/3/32/2/4/26/6/6/68/8/8/87/7/7/73/4/2/44/3/5/51/1/1/1Australian2/2/2/26/4/1/45/3/3/37/7/7/78/8/8/81/1/4/53/5/6/64/6/5/1Haberman4/6/2/22/2/3/76/7/6/48/3/4/57/8/8/35/4/5/63/5/7/81/1/1/1PimaIndian2/5/3/43/3/4/36/6/6/67/7/8/78/8/7/85/1/5/24/4/2/51/2/1/1Credit5/3/4/12/5/6/63/2/3/37/7/7/78/8/8/81/1/2/54/6/5/46/4/1/2Breast4/3/2/43/4/6/27/7/7/75/5/1/68/8/8/86/6/5/52/2/4/31/1/3/1Hepatitis6/5/3/43/3/5/67/7/6/35/4/7/58/8/8/84/6/4/72/2/2/21/1/1/1Average4.0/4.2/3.4/3.8/4.3/4.4/6.6/6.5/7.9/7.8/3.8/3.3/3.5/4.1/2.4/1.9/3.2/4.04.0/4.64.6/3.96.1/6.37.7/7.64.6/3.84.3/4.51.4/1.4Each bold value is the highest accuracy within each line.


## 4.2.2. Large-Size UCI Datasets

To see the performance of the proposed CPSVM on relatively large-size data, we then test the performance of these methods on large-size UCI data shown in Table8. The classification accuracies along with standard deviations for all methods on original data are listed in Table 9. By observing the results in Table 9, we see that the proposed CPSVM outperforms other methods on some of the datasets. On other datasets, CPSVM performs comparably to the classier with the highest accuracy.Table 8
Large-size UCI datasets information.
DatasetSample no.Feature no.HTRU2178988Page-blocks547210Spanbas460157Yeast14848Occupancy205605Titanic22013Hypothyroid316325Waveform500021Two-norm740020Table 9
Classification results on original large-size UCI datasets.
DatasetGEPSVML1-GEPSVMIGEPSVMPCCL1-NPSVMLpNPSVMGLpNPSVMCPSVMAcc ± stdAcc ± stdAcc ± stdAcc ± stdAcc ± stdAcc ± stdAcc ± stdAcc ± std1−q%HTRU297.36 ± 0.3295.26 ± 0.5794.22 ± 0.3292.34 ± 0.1979.13 ± 13.2297.35 ± 0.2897.01 ± 0.2696.54 ± 0.31 (0.95)Page-blocks94.34 ± 0.8594.02 ± 0.6590.04 ± 0.2289.78 ± 0.0587.77 ± 7.6792.32 ± 1.0994.21 ± 0.7894.00 ± 1.04 (0.9)Spanbas76.42 ± 1.3782.13 ± 2.9788.55 ± 1.5160.66 ± 0.1458.03 ± 2.7587.50 ± 0.9587.69 ± 1.9688.87 ± 1.62 (0.8)Yeast90.70 ± 3.2688.96 ± 4.5990.90 ± 1.0189.02 ± 0.3089.02 ± 0.3091.44 ± 2.8391.73 ± 1.3293.28 ± 1.18 (0.85)Occupancy98.75 ± 0.3197.68 ± 0.4183.11 ± 0.7976.91 ± 0.0378.34 ± 3.3998.64 ± 0.7198.70 ± 0.2797.22 ± 0.30 (0.9)Titanic77.69 ± 3.9277.60 ± 3.0377.51 ± 3.7865.42 ± 5.5766.24 ± 6.1777.60 ± 3.8578.01 ± 3.6973.15 ± 4.75 (0.8)Hypothyroid95.35 ± 1.6695.23 ± 0.0995.19 ± 1.4795.23 ± 1.4495.23 ± 1.4496.23 ± 0.6796.04 ± 0.7095.23 ± 1.44 (1)Waveform82.41 ± 1.8178.12 ± 0.9684.82 ± 1.4366.86 ± 1.8663.76 ± 5.4982.56 ± 1.7983.89 ± 1.9983.42 ± 1.44 (1)Two-norm97.82 ± 0.3897.76 ± 0.6197.78 ± 0.3450.59 ± 3.0553.15 ± 2.7396.62 ± 0.6397.61 ± 0.3397.74 ± 0.55 (0.9)Average90.093389.640489.124476.312274.518991.140091.654491.0500Each bold value is the highest accuracy within each line.To further test the robustness of the proposed CPSVM on large-size datasets, we consider the noise-contaminated data the same as on the small-size datasets. The classification results on these noise data are listed in Tables10–12. The corresponding accuracy ranks are shown in Table 13.Table 10
Classification results on 10% noise large-size UCI datasets.
DatasetGEPSVML1-GEPSVMIGEPSVMPCCL1-NPSVMLpNPSVMGLpNPSVMCPSVMAcc ± stdAcc ± stdAcc ± stdAcc ± stdAcc ± stdAcc ± stdAcc ± stdAcc ± std1−q%HTRU297.20 ± 0.4095.63 ± 0.7993.97 ± 0.2691.70 ± 0.1486.76 ± 8.3796.93 ± 0.2096.79 ± 0.4297.10 ± 0.29 (0.85)Page-blocks92.82 ± 1.0093.91 ± 0.7790.04 ± 0.2489.78 ± 0.0586.83 ± 7.2392.18 ± 0.5493.89 ± 0.8394.15 ± 0.93 (0.9)Spanbas77.31 ± 1.4679.16 ± 2.2489.07 ± 1.5160.44 ± 0.3357.77 ± 2.3785.02 ± 2.4587.04 ± 2.1287.22 ± 0.79 (0.75)Yeast90.43 ± 2.9586.08 ± 8.6589.02 ± 0.3089.02 ± 0.3088.95 ± 0.3390.97 ± 0.8691.20 ± 2.1493.06 ± 0.81 (0.75)Occupancy98.75 ± 0.3197.77 ± 0.3583.11 ± 0.7976.91 ± 0.0380.56 ± 6.0398.36 ± 0.1698.37 ± 0.6197.39 ± 0.21 (0.9)Titanic69.92 ± 2.2576.46 ± 1.9266.97 ± 5.1267.70 ± 2.7161.56 ± 8.9877.70 ± 2.7173.65 ± 10.7475.23 ± 4.04 (0.9)Hypothyroid95.83 ± 1.8695.20 ± 0.1294.97 ± 1.3695.23 ± 1.4495.23 ± 1.4496.20 ± 0.7296.23 ± 0.6595.23 ± 1.44 (0.9)Waveform82.66 ± 1.8378.22 ± 1.9083.08 ± 1.6466.86 ± 1.8665.20 ± 3.1780.33 ± 0.6083.10 ± 1.4784.08 ± 2.32 (1)Two-norm97.11 ± 0.4596.97 ± 0.3597.04 ± 0.3850.66 ± 3.1052.96 ± 2.4294.73 ± 0.3896.97 ± 0.3997.22 ± 0.54 (0.8)Average89.114488.821887.474476.477875.091190.268990.804491.1867Each bold value is the highest accuracy within each line.Table 11
Classification results on 20% noise large-size UCI datasets.
DatasetGEPSVML1-GEPSVMIGEPSVMPCCL1-NPSVMLpNPSVMGLpNPSVMCPSVMAcc ± stdAcc ± stdAcc ± stdAcc ± stdAcc ± stdAcc ± stdAcc ± stdAcc ± std1−q%HTRU296.90 ± 0.3191.46 ± 1.2094.33 ± 0.3092.21 ± 0.1681.90 ± 11.7796.79 ± 0.2196.69 ± 0.4596.93 ± 0.28 (0.8)Page-blocks92.77 ± 0.8993.84 ± 0.7789.95 ± 0.1789.78 ± 0.0586.53 ± 12.6191.48 ± 0.3492.66 ± 0.3993.97 ± 0.98 (0.85)Spanbas75.96 ± 2.4378.72 ± 2.6188.76 ± 1.1060.62 ± 0.1259.29 ± 1.0385.38 ± 0.5083.93 ± 3.9286.36 ± 2.38 (0.75)Yeast90.63 ± 3.3685.52 ± 8.9189.02 ± 0.3089.02 ± 0.3089.02 ± 0.3090.65 ± 0.8090.37 ± 2.1093.81 ± 1.31 (0.8)Occupancy96.55 ± 0.3697.24 ± 0.3578.99 ± 0.3676.90 ± 0.0286.43 ± 8.3395.71 ± 1.8096.42 ± 4.3296.95 ± 0.37 (0.8)Titanic77.69 ± 3.9275.72 ± 3.2177.51 ± 3.7866.97 ± 2.9467.60 ± 7.9977.60 ± 3.8577.29 ± 4.2277.03 ± 3.86 (0.9)Hypothyroid95.73 ± 1.8995.20 ± 0.1295.26 ± 1.4995.23 ± 1.4488.50 ± 10.3796.20 ± 0.6796.07 ± 0.8296.14 ± 0.45 (0.8)Waveform82.97 ± 1.6078.32 ± 1.4282.81 ± 1.7066.86 ± 1.8661.69 ± 6.1179.98 ± 0.7679.70 ± 2.0584.21 ± 1.13 (0.95)Two-norm96.35 ± 0.6196.42 ± 0.5596.32 ± 0.6650.04 ± 2.7353.50 ± 2.7894.31 ± 0.3695.04 ± 1.7596.58 ± 0.63 (0.7)Average89.505688.047988.105676.403374.940089.788989.796791.3311Each bold value is the highest accuracy within each line.Table 12
Classification results on 30% noise large-size UCI datasets.
DatasetGEPSVML1-GEPSVMIGEPSVMPCCL1-NPSVMLpNPSVMGLpNPSVMCPSVMAcc ± stdAcc ± stdAcc ± stdAcc ± stdAcc ± stdAcc ± stdAcc ± stdAcc ± std1−q%HTRU296.87 ± 0.3594.98 ± 0.5495.13 ± 0.3091.47 ± 0.1588.98 ± 3.5996.80 ± 0.3496.61 ± 0.2796.90 ± 0.31 (0.75)Page-blocks92.66 ± 0.8893.64 ± 0.7189.91 ± 0.1689.78 ± 0.0588.94 ± 4.4891.94 ± 1.1092.64 ± 0.3593.90 ± 0.93 (0.7)Spanbas72.92 ± 1.7877.27 ± 6.485.09 ± 1.7960.60 ± 0.0856.82 ± 4.1377.59 ± 1.0283.99 ± 2.4585.37 ± 2.83 (0.75)Yeast88.88 ± 3.5683.95 ± 11.8489.02 ± 0.3089.02 ± 0.3088.14 ± 1.7787.60 ± 2.9587.99 ± 0.8993.34 ± 0.82 (0.85)Occupancy84.23 ± 0.7384.57 ± 0.9676.87 ± 0.7376.16 ± 0.3276.95 ± 0.5484.35 ± 0.9284.52 ± 1.0286.20 ± 0.71 (0.7)Titanic77.60 ± 3.8573.97 ± 3.377.60 ± 3.8577.60 ± 3.8569.24 ± 4.9767.70 ± 2.7172.28 ± 8.3477.60 ± 3.85 (0.85)Hypothyroid93.55 ± 0.9495.23 ± 0.0995.23 ± 1.5395.23 ± 1.4491.50 ± 4.2995.94 ± 0.8295.91 ± 0.7095.80 ± 0.98 (0.7)Waveform76.17 ± 1.5577.34 ± 1.3175.78 ± 1.2066.86 ± 1.8664.73 ± 4.1075.79 ± 1.2176.61 ± 1.2677.73 ± 1.53 (0.9)Two-norm95.31 ± 0.6695.43 ± 0.8595.35 ± 0.8150.04 ± 2.7353.72 ± 4.1293.53 ± 1.3294.26 ± 0.3195.53 ± 0.74 (0.7)Average86.465686.265586.664477.417875.446785.693387.201189.1522Each bold value is the highest accuracy within each line.Table 13
Classification ranks on original and noise large-size UCI datasets.
DatasetGEPSVML1-GEPSVMIGEPSVMPCCL1-NPSVMLpNPSVMGLpNPSVMCPSVMRankRankRankRankRankRankRankRankHTRU21/1/2/25/5/7/66/6/5/57/7/6/78/8/8/82/3/3/33/4/4/44/2/1/1Page-blocks1/4/3/33/2/2/26/6/6/67/7/7/78/8/8/85/5/5/52/3/4/44/1/1/1Spanbas6/6/6/65/5/5/52/1/1/27/7/7/78/8/8/84/4/3/43/3/4/31/2/2/1Yeast5/4/3/48/8/8/84/5/5/26/6/6/37/7/7/53/3/2/72/2/4/61/1/1/1Occupancy1/1/3/54/4/1/26/6/7/78/8/8/87/7/6/63/3/5/42/2/4/35/5/2/1Titanic2/5/1/13/2/6/55/7/3/28/6/8/37/8/7/74/1/2/81/4/4/66/3/5/4Hypothyroid3/3/4/74/7/7/48/8/5/55/4/6/66/5/8/81/2/1/12/1/3/27/6/2/3Waveform5/4/2/46/6/6/21/3/3/67/7/7/78/8/8/84/5/4/52/2/5/33/1/1/1Two-norm1/2/3/43/4/2/22/3/4/38/8/8/87/7/7/76/6/6/65/5/5/54/1/1/1Average2.8/3.3/4.6/4.8/4.4/5.0/7.0/6.7/7.3/7.3/3.6/3.6/2.4/2.9/3.9/2.4/3.0/4.04.9/4.04.3/4.27/6.27.4/7.23.4/4.84.1/4.01.8/1.6By observing the results in these tables, we can see that on relatively large-size datasets, CPSVM has similar behavior compared to small-size datasets. In specific, noise has an influence on all methods. However, CPSVM is influenced less compared to other methods. In addition, the optimal 1 −q% still plays an important role in CPSVM, which may vary on different data and different levels of noise. The results in Table 13 confirm the superiority of the proposed method.


## 4.3. MNIST Handwritten Dataset

In this subsection, we apply our proposed CPSVM to MNIST handwritten symbol recognition. To test the robustness, we also consider polluting the data by adding black or Gaussian rectangular block to training data. The MNIST database consists of grayscale images of handwritten digits from 0 to 9, and the size of each image is 16× 16 pixels with 256 gray levels, as shown in Figure 6(a). We choose three pairs of numbers (0 vs 6, 0 vs 9, and 6 vs 9) for our comparisons because these groups of numbers are similar and not easy to distinguish. Each dataset is randomly divided into two subsets: 50% images for training and 50% images for testing. Then, to generate outliers, we randomly select 10% images from the training set and add black block noise or Gaussian block noise of 10%, 20%, 30%, and 40% area on it. Figure 6(b) shows sample images of black block noise of 30% area outliers, and Figure 6(c) shows sample images of Gaussian block noise of 30% area. These datasets are denoted as “Ori,” “B10,” “B20,” “B30,” “B40,” “G10,” “G20,” “G30,” and “G40,” where 10, 20, 30, and 40 represent the percentage of blocks, and B and G represent black block and Gaussian block, respectively.Figure 6
Training samples from MNIST  data. (a) Original training data. (b) Training data with 30% area black block noise. (c) Training data with 30% area Gaussian block noise.
(a)(b)(c)The recognition results of all methods on these datasets are listed in Tables14–19. For the black block noise case, the results of L1-GEPSVM, IGEPSVM, LpNPSVM, GLpNPSVM, and CPSVM are relatively close, while they are all not better than CPSVM for most cases. When gradually increasing the area of black block noise, L1-GEPSVM, IGEPSVM, LpNPSVM, and GLpNPSVM are affected by noise more seriously than CPSVM, which shows the robustness of CPSVM. For the Gaussian noise experiment, the performance of CPSVM is the best or is comparable to the best method for most cases. In addition, with the increase of noise level, the performance of CPSVM is stable, while the noise has a greater impact on other methods.Table 14
Classification results on 0 vs 6 data with black block noise.
DatasetGEPSVML1-GEPSVMIGEPSVMPCCL1-NPSVMLpNPSVMGLpNPSVMCPSVMAccAccAccAccAccAccAccAcc1−q%Ori92.0697.5897.7492.1469.0693.0698.4999.00 (0.9)B1094.9897.1697.6692.6464.7298.7599.0098.91 (0.9)B2089.9796.7496.8391.6561.9098.3396.9198.25 (0.85)B3091.8295.5897.6692.2466.6196.9996.7498.08 (0.85)B4094.1395.2296.9092.7173.7696.996.0698.08 (0.7)Each bold value is the highest accuracy within each line.Table 15
Classification results on 0 vs 9 data with black block noise.
DatasetGEPSVML1-GEPSVMIGEPSVMPCCL1-NPSVMLpNPSVMGLpNPSVMCPSVMAccAccAccAccAccAccAccAcc1−q%Ori98.4098.3299.1696.6260.9099.5899.1699.58 (0.9)B1097.7298.1699.1696.7177.8799.7599.0799.58 (0.85)B2096.8097.7599.0796.2190.6693.8699.1699.41 (0.75)B3098.8297.4998.8296.2083.0495.8699.1699.41 (0.7)B4094.3197.7597.5796.8280.3391.4699.0899.41 (0.75)Each bold value is the highest accuracy within each line.Table 16
Classification results on 6 vs 9 data with black block noise.
DatasetGEPSVML1-GEPSVMIGEPSVMPCCL1-NPSVMLpNPSVMGLpNPSVMCPSVMAccAccAccAccAccAccAccAcc1−q%Ori92.6994.1597.2182.0565.2998.0198.2798.01 (0.95)B1092.1593.9397.6178.1986.8497.8797.2198.54 (0.85)B2084.7493.4396.4578.9581.9795.9296.8497.24 (0.85)B3093.0493.2797.1176.7790.4296.0696.7297.24 (0.7)B4093.4793.0496.4879.7766.8495.3096.3497.24 (0.7)Each bold value is the highest accuracy within each line.Table 17
Classification results on 0 vs 6 data with Gaussian block noise.
DatasetGEPSVML1-GEPSVMIGEPSVMPCCL1-NPSVMLpNPSVMGLpNPSVMCPSVMAccAccAccAccAccAccAccAcc1−q%Ori97.4998.4196.1692.4074.5299.0899.0899.16 (0.85)G1097.6697.4995.9192.1581.5498.3399.0098.75 (0.85)G2097.91970795.9992.1570.3497.7498.7599.08 (0.8)G3097.0897.1695.9192.1572.8597.9998.9198.91 (0.75)G4097.0895.5795.9192.2371.0997.9198.8398.91 (0.7)Each bold value is the highest accuracy within each line.Table 18
Classification results on 0 vs 9 data with Gaussian block noise.
DatasetGEPSVML1-GEPSVMIGEPSVMPCCL1-NPSVMLpNPSVMGLpNPSVMCPSVMAccAccAccAccAccAccAccAcc1−q%Ori99.2498.1698.5796.9887.3299.5099.6699.50 (1)G1098.6698.3398.4996.8976.6699.2499.5899.50 (0.9)G2098.8298.1698.4096.8991.0298.9998.1599.41 (0.85)G3098.3298.1698.3296.8164.8298.7498.2499.16 (0.75)G4098.3298.0898.4096.8184.0598.1598.3299.16 (0.75)Table 19
Classification results on 6 vs 9 data with Gaussian block noise.
DatasetGEPSVML1-GEPSVMIGEPSVMPCCL1-NPSVMLpNPSVMGLpNPSVMCPSVMAccAccAccAccAccAccAccAcc1−q%Ori95.0597.3996.7482.5373.5395.1898.7098.04 (1)G1095.7097.0096.8781.6280.4493.4897.7898.17 (0.95)G2094.1396.4896.7482.2761.5494.0097.7898.04 (0.85)G3094.7896.3596.7481.7572.2392.0596.6197.39 (0.8)G4093.8796.2296.3582.6679.4093.8796.2297.39 (0.7)Each bold value is the highest accuracy within each line.


## 5. Conclusion

In this paper, we propose a cappedL1-norm proximal support vector machine (CPSVM). Capped L1-norm brings robustness to CPSVM, and CPSVM can be viewed as a weighted SVM. Experimental results show that compared to related SVMs, CPSVM can effectively remove extreme outliers and suppress the influence of noise data. However, CPSVM requires solving a series of generalized eigenvalue problems, which may be very slow on very large-scale data. Therefore, how to design a more efficient algorithm is one of our future works. In addition, extending CPSVM to its kernel case or matrix and tensor data is also interesting. Another interesting area is its potential improvement by the type-3 fuzzy logic systems [35]. The fuzzy technique balances the importance of the samples and hence generates robustness. Therefore, the robustness of CPSVM can be further improved by incorporating type-3 fuzzy logic systems, and a fuzzy CPSVM should be considered.



---

*Source: 3082657-2022-09-30.xml*