# From Linear Programming Approach to Metaheuristic Approach: Scaling Techniques

Authors: Elsayed Badr; Mustafa Abdul Salam; Sultan Almotairi; Hagar Ahmed
Journal: Complexity
 (2021)

---

## Abstract

The objective of this work is to propose ten efficient scaling techniques for the Wisconsin Diagnosis Breast Cancer (WDBC) dataset using the support vector machine (SVM). These scaling techniques are efficient for the linear programming approach. SVM with proposed scaling techniques was applied on the WDBC dataset. The scaling techniques are, namely, arithmetic mean, de Buchet for three casesp=1,2,and∞, equilibration, geometric mean, IBM MPSX, and Lp-norm for three cases p=1,2,and∞. The experimental results show that the equilibration scaling technique overcomes the benchmark normalization scaling technique used in many commercial solvers. Finally, the experimental results also show the effectiveness of the grid search technique which gets the optimal parameters (C and gamma) for the SVM classifier.


---

## Body

## 1. Introduction

Scaling techniques play an important role in the convergence speed of machine learning algorithms, specially in classification and regression tasks. Using an efficient scaling technique makes training of algorithms faster. There is an integrative relationship between the linear programming approach and metaheuristic approach according to scaling techniques. A scaling technique is defined as a mathematical formula which makes these elements have similar magnitudes. In linear programming, the scaling techniques are applied on the objective function, the coefficient matrix of the inequalities, and the coefficient of constants. On the contrary, in the metaheuristic approach, the scaling techniques are applied on the matrix in which its rows represent the observations and its columns are the attributes of the dataset.A dataset contains mostly nonzero elements which are of different values. Such representation is called a bad representation for this matrix. Scaling techniques can be used to handle this issue. Scaling techniques are used before applying the classifier in order to improve the classification accuracy on the dataset.The comparison among the following scaling techniques, Curtis and Reid [1] scaling technique, arithmetic mean scaling technique, Wolfe [2] scaling technique, geometric mean scaling technique, and equilibration scaling technique, was proposed by Tomlin [3] on 6 test linear programming problems of different sizes. Another study was proposed by Larsson [4] by proposing and comparing entropy, de Buchet scaling technique [5], and Lp-norm scaling technique [6] on one-hundred thirty-five randomly generated problems of different dimensions. He deduced that the entropy scaling method outperforms the other scaling techniques. Elble and Sahinidis [7] proposed new experimental results from the comparison among the following scaling techniques: IBM MPSX, entropy, arithmetic mean, binormalization, geometric mean, Lp-norm, equilibration, and de Buchet on benchmark problems from Netlib. Scaling and solution times, the number of iterations for the solution, and the maximum condition number were the evaluation metrics for their study. They deduced that the equilibration method outperformed other techniques. Ploskas and Samaras [8] introduced experimental results for three algorithms: MATLAB’s revised simplex method, exterior point simplex method, and interior point algorithm using geometric mean scaling technique, equilibration scaling technique, and arithmetic mean scaling technique. They deduced that the equilibration scaling technique overcame the other techniques and that the effectiveness of scaling is important to both the interior point algorithm and revised simplex method; on the contrary, the exterior point simplex method is scaling invariant [9]. Ploskas and Samaras [10] proposed new experimental results comparing arithmetic mean, de Buchet for three cases p=1,2,and∞, equilibration, geometric mean, IBM MPSX, and Lp-norm for three cases p=1,2,and∞. Ploskas and Samaras [10] deduced that arithmetic mean, equilibration, and geometric mean overcame the other scaling techniques according to the execution time. In [11], Ploskas and Samaras in the chapter “Scaling Techniques” present a complete list of the scaling techniques plus illustrative examples. Ploskas and Samaras [12] state clearly that MATLAB’s GPU environment (in 2014) did not offer sparse utilities. Also, they were the first to present a GPU-based simplex implementation that showed speedups in benchmark instances. In their implementation, they used the most efficient scaling techniques.In this work, ten efficient scaling techniques were proposed for the Wisconsin Diagnosis Breast Cancer (WDBC) dataset using the support vector machine (SVM). The SVM with proposed scaling techniques was applied on the WDBC dataset. The experimental results show that the equilibration scaling technique overcomes the benchmark normalization scaling technique.The rest of this paper is organized as follows. The support vector machine classifier is described in section2. In section 3, detailed descriptions of new scaling techniques are presented. The experimental design which has data description, experimental setup, measure for performance evaluation, and grid search method is introduced in section 4. In section 5, experimental results and discussions are discussed. In section 6, conclusions and future works are introduced.


## 2. Support Vector Machine Classifier

The support vector machine (SVM) is considered as a machine learning model originally developed by Vapnik [13, 14]. The SVM is based on the Vapnik–Chervonenkis (VC) theory and structural risk minimization (SRM) principle [13, 15]. The main objective of the SVM is finding a hyperplane in an N-dimensional space (N: the number of features) that distinctly classifies the data points, as shown in Figure 1. The convex quadratic programming is used for the SVM in order to avoid the local minima [13, 16].Figure 1
The SVM hyperplane in the linear classification.In the linear classification, the hyperplane is placed in the largest distance between two vectors. In case of the nonlinear classification, it is mapped to the linear classification problem in a high-dimensional space [17], as shown in Figure 2.Figure 2
Mapping from nonlinear to linear classification.Let us consider a binary classification task: suppose thatx1,y1,…,xn,yn:xi∈Rd and yi∈−1,1 is a labeled training dataset such that xi is a representation of the feature vector and yi is the class label (negative or positive) of a training compound i. The optimal hyperplane can then be defined as follows:(1)wxT+b=0.Such thatw is the weight vector, x is the input feature vector, and b is the bias. w and b would satisfy both inequality (2) and inequality (3) for all elements of the training set:(2)wxiT+b≥1,ifyi=1,(3)wxiT+b≤1,ifyi=−1.The aim of training an SVM classifier is to determinew and b so that the hyperplane separates the data and maximizes the margin 1/w2. Vectors, xi for which yiwxiT+b=1, will be termed the support vector.There are cases in which we can linearly separate between the two classes, and there are other cases in which we cannot linearly separate between the two classes. We can overcome this problem by transforming the original input space into some higher-dimensional feature space where the two classes can be linearly separable. An alternative use for the SVM is the kernel method, which enables us to model higher-dimensional, nonlinear models [18]. In a nonlinear problem, a kernel function could be used to add additional dimensions to the raw data and thus making it a linear problem in the resulting higher-dimensional space. On the contrary, the kernel functions could help do certain calculations faster which would need computations in the high-dimensional space. There are many kernel functions, for example, but not limited to, the linear kernel and the Gaussian kernel, which are defined as shown in the following equations:(4)Kxi,xj=1+xiTxjp,(5)Kxi,xj=exp−γxi−xj2,where p is the order of polynomial and γ is the predefined parameter controlling the width of the Gaussian kernel. The SVM classification accuracy is improved by the proper model parameters’ setting [19]. It is important to choose the parameters in advance. These parameters are C, (γ or p), and kernel function.C parameter is considered a regularization or generalization parameter. It governs the trade-off between having a minimum training error and minimizing the weight’s norm. C parameter tuning is a very important step in optimizing of the SVM. The parameter C imposes an upper bound on the weight’s norm, which implies that there are multiple hypothesis classes indexed by C. Increasing the C parameter leads to increasing the complexity of the hypothesis class. If we increase C slightly, we can still form all of the linear models [19]. Determining how to set C is not very well developed, so most researchers use cross-validation.


## 3. Scaling Techniques

Here, we introduce the mathematical notations of ten scaling techniques in addition to the normalization scaling techniques with ranges [0, 1)] and [−1, 1]. First of all, we introduce the following mathematical preliminaries, as shown in Table1.Table 1
Mathematical preliminaries for scaling techniques.
SymbolDescriptionAaij:m×n matrix (with m rows (observations) and n columns (attributes))r1:The scaling agent of rowisj:The scaling agent of columnjR:Diagonal matrix, such thatR=diagr1,…,rmS:Diagonal matrix, such thatS=diagr1,…,rnNi:Ni=j:Aij≠0, such that 1≤i≤m and Mj=i:Aij≠0, such that 1≤j≤nMj: ni:The number of elements for the setNimj:The number of elements for the setMjARaijR:The scaled matrix by the rowR scaling agentARSaijRS:The final scaled matrixThe scaled matrix is expressed as RAS, such thatR=diagr1,…,rm and S=diags1,…,sn. All scaling techniques proposed in this section first apply row scaling and after that column scaling. Then, the matrix after full scaling (row and column) is given by the following:(6)AR=RA,ARS=ARS.


### 3.1. Arithmetic Scaling Technique [11]

First, equation (7) represents the rows’ scaling such that each row (instance) is divided by the arithmetic mean of the absolute value of the nonzero elements in that row (instance):(7)ri=ni∑j∈Niaij:aij≠0.Second, equation (8) represents the columns’ scaling such that each column (attribute) is divided by the arithmetic mean of the absolute value of the nonzero elements in that column (attribute):(8)sj=mj∑i∈MjaijRaijR≠0.


### 3.2. de Buchet Scaling Technique [4]

Equation (9) formulates the de Buchet scaling method which is based on the relative divergence:(9)minr,s>0∑i,j∈Z¯aijrisj+1aijrisjp1/p,where the number of the nonzero elements of A is dented by Z¯ and the parameter p is a positive integer. Here, there are the following three cases:Casep = 1: in this case, equation (9) approaches to the following equation:(10)minr,s>0∑i,j∈Z¯aijrisj+1aijrisj.Equation (11) represents the row scaling factor of the matrix A:(11)ri=∑j∈Ni1/aij∑j∈Niaij1/2.Equation (12) represents the column scaling factor of the scaled matrix A by ri:(12)sj=∑i∈Mj1/aijR∑i∈MjaijR1/2.Casep = 2: in this case, equation (9) approaches to the following equation:(13)minr,s>0∑i,j∈Z¯aijrisj+1aijrisj21/2.Equation (14) represents the row scaling factor of the matrix A:(14)ri=∑j∈Ni1/aij2∑j∈Niaij21/4.Equation (15) represents the column scaling factor of the scaled matrix A by ri:(15)sj=∑i∈Mj1/aijR2∑i∈MjaijR21/4.Casep = ∞: in this case, equation (9) approaches to the following equation:(16)minr,s>0maxi,j∈Z¯logaijrisj.Equation (17) represents the row scaling factor of the matrix A:(17)ri=1maxj∈Niaijminj∈Niaij1/2.Equation (18) represents the column scaling factor of the scaled matrix A by ri:(18)sj=1maxi∈MjaijRmini∈MjaijR1/2.The last case of the de Buchet (p = ∞) scaling technique is equivalent to the geometric mean scaling method that will be introduced later.


### 3.3. Equilibration Scaling Technique [11]

The largest element in the absolute value is the corner stone for this scaling method. Each row of the matrixA is divided by the largest element in the absolute value in that row. Then, each column of the scaled matrix A by the row factor is divided by the largest element in the absolute value in that column. The range of the final scaled matrix A is (−1, 1).


### 3.4. Geometric Mean Scaling Technique [11]

First, equation (19) represents the rows’ scaling such that each row (instance) is divided by the geometric mean of the absolute value of the nonzero elements in that row (instance):(19)ri=maxj∈Niaijminj∈Niaij−1/2.Second, equation (20) represents the columns’ scaling such that each column (attribute) is divided by the geometric mean of the absolute value of the nonzero elements in that column (attribute):(20)sj=maxj∈MjaijRminj∈MjaijR−1/2.


### 3.5. IBM MPSX Scaling Technique [11]

The IBM MPSX scaling method is a combination between the geometric mean and the equilibration scaling methods. First, the geometric mean is performed four times or until the relation (21) holds true:(21)1Z¯∑i,j∈Z¯aij2−∑i,j∈Z¯aij22Z¯<ε,where the number of the nonzero elements of A is denoted by Z¯ and the parameter εis a positive integer (which is often ε<10). Then, the equilibration scaling method is applied. The IBM MPSX scaling method was introduced by Benichou et al. [20].


### 3.6.Lp-Norm Scaling Technique [11]

Equation (22) formulates the Lp-norm scaling method: (22)minr,s>0∑i,j∈Z¯logaijrisjp1/p,where the number of the nonzero elements of A is dented by Z¯. Here, there are the following three cases:Casep = 1: in this case, equation (22) approaches to the following equation:(23)minr,s>0∑i,j∈Z¯logaijrisj.Equation (24) represents the row scaling factor of the matrix A:(24)ri=1medianaij|j∈Ni.Similarly, equation (25) represents the column scaling factor of the matrix A:(25)sj=1medianaijR|i∈Mj.Casep = 2: in this case, equation (22) approaches to the following equation:(26)minr,s>0∑i,j∈Z¯logaijrisj21/2.Equation (27) represents the row scaling factor of the matrix A:(27)ri=1∏j∈Niaij1/ni.Similarly, equation (28) represents the column scaling factor of the matrix A:(28)sj=1∏i∈MjaijR1/mj.Casep = ∞, the last case of the Lp-norm (p = ∞) scaling technique is equivalent to the geometric mean scaling method.


### 3.7. Normalization Scaling Technique [−1, 1] [21]

Equation (29) is used for the normalization scaling method with range [−1, 1] such that a,a′,maxk,andmink are the original value, the scaled value, the maximum value, and the minimum value of feature k, respectively:(29)a′=2a−minkmaxk−mink−1.The normalization scaling method avoids the numerical difficulties during the calculation.


### 3.8. Normalization Scaling Technique [0, 1] [21]

Another normalization scaling technique is formulated from the updated equation (29) as follows:(30)a′=a−minkmaxk−mink.


## 3.1. Arithmetic Scaling Technique [11]

First, equation (7) represents the rows’ scaling such that each row (instance) is divided by the arithmetic mean of the absolute value of the nonzero elements in that row (instance):(7)ri=ni∑j∈Niaij:aij≠0.Second, equation (8) represents the columns’ scaling such that each column (attribute) is divided by the arithmetic mean of the absolute value of the nonzero elements in that column (attribute):(8)sj=mj∑i∈MjaijRaijR≠0.


## 3.2. de Buchet Scaling Technique [4]

Equation (9) formulates the de Buchet scaling method which is based on the relative divergence:(9)minr,s>0∑i,j∈Z¯aijrisj+1aijrisjp1/p,where the number of the nonzero elements of A is dented by Z¯ and the parameter p is a positive integer. Here, there are the following three cases:Casep = 1: in this case, equation (9) approaches to the following equation:(10)minr,s>0∑i,j∈Z¯aijrisj+1aijrisj.Equation (11) represents the row scaling factor of the matrix A:(11)ri=∑j∈Ni1/aij∑j∈Niaij1/2.Equation (12) represents the column scaling factor of the scaled matrix A by ri:(12)sj=∑i∈Mj1/aijR∑i∈MjaijR1/2.Casep = 2: in this case, equation (9) approaches to the following equation:(13)minr,s>0∑i,j∈Z¯aijrisj+1aijrisj21/2.Equation (14) represents the row scaling factor of the matrix A:(14)ri=∑j∈Ni1/aij2∑j∈Niaij21/4.Equation (15) represents the column scaling factor of the scaled matrix A by ri:(15)sj=∑i∈Mj1/aijR2∑i∈MjaijR21/4.Casep = ∞: in this case, equation (9) approaches to the following equation:(16)minr,s>0maxi,j∈Z¯logaijrisj.Equation (17) represents the row scaling factor of the matrix A:(17)ri=1maxj∈Niaijminj∈Niaij1/2.Equation (18) represents the column scaling factor of the scaled matrix A by ri:(18)sj=1maxi∈MjaijRmini∈MjaijR1/2.The last case of the de Buchet (p = ∞) scaling technique is equivalent to the geometric mean scaling method that will be introduced later.


## 3.3. Equilibration Scaling Technique [11]

The largest element in the absolute value is the corner stone for this scaling method. Each row of the matrixA is divided by the largest element in the absolute value in that row. Then, each column of the scaled matrix A by the row factor is divided by the largest element in the absolute value in that column. The range of the final scaled matrix A is (−1, 1).


## 3.4. Geometric Mean Scaling Technique [11]

First, equation (19) represents the rows’ scaling such that each row (instance) is divided by the geometric mean of the absolute value of the nonzero elements in that row (instance):(19)ri=maxj∈Niaijminj∈Niaij−1/2.Second, equation (20) represents the columns’ scaling such that each column (attribute) is divided by the geometric mean of the absolute value of the nonzero elements in that column (attribute):(20)sj=maxj∈MjaijRminj∈MjaijR−1/2.


## 3.5. IBM MPSX Scaling Technique [11]

The IBM MPSX scaling method is a combination between the geometric mean and the equilibration scaling methods. First, the geometric mean is performed four times or until the relation (21) holds true:(21)1Z¯∑i,j∈Z¯aij2−∑i,j∈Z¯aij22Z¯<ε,where the number of the nonzero elements of A is denoted by Z¯ and the parameter εis a positive integer (which is often ε<10). Then, the equilibration scaling method is applied. The IBM MPSX scaling method was introduced by Benichou et al. [20].


## 3.6.Lp-Norm Scaling Technique [11]

Equation (22) formulates the Lp-norm scaling method: (22)minr,s>0∑i,j∈Z¯logaijrisjp1/p,where the number of the nonzero elements of A is dented by Z¯. Here, there are the following three cases:Casep = 1: in this case, equation (22) approaches to the following equation:(23)minr,s>0∑i,j∈Z¯logaijrisj.Equation (24) represents the row scaling factor of the matrix A:(24)ri=1medianaij|j∈Ni.Similarly, equation (25) represents the column scaling factor of the matrix A:(25)sj=1medianaijR|i∈Mj.Casep = 2: in this case, equation (22) approaches to the following equation:(26)minr,s>0∑i,j∈Z¯logaijrisj21/2.Equation (27) represents the row scaling factor of the matrix A:(27)ri=1∏j∈Niaij1/ni.Similarly, equation (28) represents the column scaling factor of the matrix A:(28)sj=1∏i∈MjaijR1/mj.Casep = ∞, the last case of the Lp-norm (p = ∞) scaling technique is equivalent to the geometric mean scaling method.


## 3.7. Normalization Scaling Technique [−1, 1] [21]

Equation (29) is used for the normalization scaling method with range [−1, 1] such that a,a′,maxk,andmink are the original value, the scaled value, the maximum value, and the minimum value of feature k, respectively:(29)a′=2a−minkmaxk−mink−1.The normalization scaling method avoids the numerical difficulties during the calculation.


## 3.8. Normalization Scaling Technique [0, 1] [21]

Another normalization scaling technique is formulated from the updated equation (29) as follows:(30)a′=a−minkmaxk−mink.


## 4. Experimental Design

In this section, we introduce data description, measure for performance evaluation, and the grid search method.


### 4.1. Data Description

In this work, we have run the proposed model on the Wisconsin Diagnosis Breast Cancer (WDBC) dataset that is available on the UCI Machine Learning Repository [22]. The dataset consists of 569 instances divided into two classes. The two classes malignant and benign have 357 and 212 cases, respectively. Every observation in the database has thirty-three attributes. These thirty-three attributes differ between benign and malignant samples.The MATLAB platform is used to implement the SVM diagnostic system. On the contrary, we use LIBSVM that is developed by Chang and Lin [23]. Table 2 describes the computing environment.Table 2
Description of the computing environment.
CPUIntel (R) core (TM) i5-7200 U CPU@ 2.70 GHzRAM size4 GB RAMMATLAB versionR2018a (9.4.0.813654)Salzberg [24] introduced the k-fold CV which is used to guarantee the valid results. In this paper, set k as 10, i.e., the data consists of 10 subsets. The most commonly used (default) value is k = 10 in k-fold CV, which is often a good choice [25]. Each time, one of the 10 subsets is utilized as the test set and the remaining 9 subsets are utilized as a training set.


### 4.2. Measure for Performance Evaluation

In order to test the performance of the SVM model, we use accuracy (ACC). Table3 shows the confusion matrix. TP, FN, TN, and FP are the number of true positives, the number of false negatives, the number of true negatives, and the number of false positives, respectively. According to the confusion matrix, the total classification accuracy (ACC) is defined as follows: ACC = (TP + TN)/(TP + FP + TN + FN) × 100%.Table 3
confusion matrix for breast cancer diagnosis.
Predictive positivePredictive negativeActualPositive(TP)(FN)Negative(FP)(TN)


### 4.3. Grid Search Method

In order to test the performance of the SVM system, we use the grid search method. The grid search method is used to determine the optimal parametersC and γ. Figure 3 shows the flowchart of the SVM training using the grid search. We utilize the searching space of C and γ as follows: 2−5,2−3,…,215and2−15,2−13,…,21, respectively.Figure 3
SVM using grid search.


## 4.1. Data Description

In this work, we have run the proposed model on the Wisconsin Diagnosis Breast Cancer (WDBC) dataset that is available on the UCI Machine Learning Repository [22]. The dataset consists of 569 instances divided into two classes. The two classes malignant and benign have 357 and 212 cases, respectively. Every observation in the database has thirty-three attributes. These thirty-three attributes differ between benign and malignant samples.The MATLAB platform is used to implement the SVM diagnostic system. On the contrary, we use LIBSVM that is developed by Chang and Lin [23]. Table 2 describes the computing environment.Table 2
Description of the computing environment.
CPUIntel (R) core (TM) i5-7200 U CPU@ 2.70 GHzRAM size4 GB RAMMATLAB versionR2018a (9.4.0.813654)Salzberg [24] introduced the k-fold CV which is used to guarantee the valid results. In this paper, set k as 10, i.e., the data consists of 10 subsets. The most commonly used (default) value is k = 10 in k-fold CV, which is often a good choice [25]. Each time, one of the 10 subsets is utilized as the test set and the remaining 9 subsets are utilized as a training set.


## 4.2. Measure for Performance Evaluation

In order to test the performance of the SVM model, we use accuracy (ACC). Table3 shows the confusion matrix. TP, FN, TN, and FP are the number of true positives, the number of false negatives, the number of true negatives, and the number of false positives, respectively. According to the confusion matrix, the total classification accuracy (ACC) is defined as follows: ACC = (TP + TN)/(TP + FP + TN + FN) × 100%.Table 3
confusion matrix for breast cancer diagnosis.
Predictive positivePredictive negativeActualPositive(TP)(FN)Negative(FP)(TN)


## 4.3. Grid Search Method

In order to test the performance of the SVM system, we use the grid search method. The grid search method is used to determine the optimal parametersC and γ. Figure 3 shows the flowchart of the SVM training using the grid search. We utilize the searching space of C and γ as follows: 2−5,2−3,…,215and2−15,2−13,…,21, respectively.Figure 3
SVM using grid search.


## 5. Experimental Results and Discussion

Here, the experimental results were applied, and an attempt is made to prove the validation of the proposed scaling techniques. The experiments were done on the WBCD dataset using the SVM to estimate the efficiency of the proposed scaling techniques for the breast cancer.Table4 shows the effectiveness of the grid search method which gets the best parameters C and γ for the SVM. The accuracy of normalization scaling techniques (S1) is better than that without the scaling technique (S0). Also, using scaling techniques (S1) speeds up the search and achieves dramatic decrease of CPU time. So, this result shows the effectiveness of the grid search method using the scaling technique (S1).Table 4
Accuracy of the WBCD database using the SVM withC and γ which were calculated by the grid search technique (without scaling, normalization scaling [0, 1], and normalization scaling [−1, 1]).
FoldWithout scaling (S0)Normalization scaling (0, 1) (S1)CγAccuracy %CγAccuracy %12^32^−1394.762^132^−710022^72^−1591.592^152^−998.2532^152^−131002^152^192.9842^52^−1397.182^152^−194.7452^12^−1196.232^152^−194.7462^−12^−991.292^152^196.4972^112^−1597.592^152^−398.2582^92^−1598.602^152^−1396.4992^92^−1597.592^152^194.74102^152^−996.232^152^198.25Avg.6877.90.000596.1030310.40.9196.49CPU time52.6216714.558410Tables5 and 6 show the average classification accuracy rates and CPU time of the SVM with four scaling techniques. These techniques are normalization between (−1, 1) (S2), the equilibration scaling (S3), the geometric mean scaling (S4), and the arithmetic mean scaling (S5). One can notice easily that S3 achieved the best accuracy with 98.95% outperforming the compared scaling techniques. S3 also achieved lowest CPU time with about 10.2 seconds.Table 5
Accuracy of the WBCD database using the SVM withC and γ which were calculated by the grid search technique (normalization between [−1, 1] and equilibration scaling).
FoldNormalization between (−1, 1) (S2)Equilibration scaling (S3)CγAccuracy%CγAccuracy %12^112^194.642^52^−1100.0022^152^192.982^32^198.2532^132^11002^52^−1100.0042^132^198.252^152^198.2552^152^196.492^12^−1100.0062^152^−196.492^92^−198.2572^132^11002^152^1100.0082^152^196.492^152^1100.0092^132^194.742^32^194.74102^132^−196.492^32^1100.00Avg.174081.796.669890.61.498.95CPU time19.20879710.175330Table 6
Accuracy of the WBCD database using the SVM withC and γ which were calculated by the grid search technique (de Buchet p = 1 scaling, de Buchet p = 2 scaling, and IBM MPSX scaling).
FoldGeometric mean scaling (S4)Arithmetic mean scaling (S5)CγAccuracy %CγAccuracy %12^112^−91002^32^−7100.0022^112^−998.252^152^−998.2532^112^−91002^92^−596.4942^152^−1398.252^−12^−596.4952^112^−996.492^92^−9100.0062^112^−91002^52^−598.2572^112^−998.252^72^−798.2582^52^−598.252^−12^−398.2592^112^−91002^92^−9100.00102^112^−996.492^152^−998.25Avg.4918.40.004798.606724.10.02498.42CPU time15.14307612.516496Tables7 and 8 show the average accuracy rates of the SVM with the de Buchet scaling technique with p = 1 (S6), de Buchet scaling technique with p = 2 (S7), and the IBM MPSX scaling technique (S8). It is clear that both S6 and S8 scaling techniques have the same accuracy of 98.59% which is better than the accuracy of S7 scaling technique.Table 7
Accuracy for the WBCD database using the SVM withC and γ which were calculated by the grid search technique (de Buchet p = 1 and p = 2).
Foldde Buchetp = 1 (S6)de Buchetp = 2 (S7)CγAccuracy %CγAccuracy %12^92^−796.432^152^−1194.4622^92^−1198.252^52^−596.4932^12^−51002^52^−510042^92^−796.492^92^−710052^52^−71002^32^−310062^92^−71002^32^−196.4972^92^−71002^52^−598.2582^92^−71002^52^−510092^92^−71002^52^−1100102^132^−794.742^52^−5100Avg.11810.009498.593348.80.12998.57CPU time12.9926413.28168Table 8
Accuracy for the WBCD database using the SVM withC and γ which were calculated by the grid search technique (IBM MPSX).
FoldIBM MPSX (S8)CγAccuracy%12^152^−994.6422^52^−110032^132^−798.2542^92^−598.2552^152^−910062^152^−910072^52^110082^152^−910092^152^−9100102^52^194.74Avg.172640.003198.59CPU time11.516754Table9 shows the average classification accuracy rates of the SVM with the Lp-norm scaling technique with p = 1 (S9) and Lp-norm scaling technique with p = 2 (S10). One can notice that S9 achieved 98.25 accuracy outperforming S10 scaling technique. But, S10 CPU time is slightly lower than S9.Table 9
Accuracy for the WBCD database using the SVM withC and γ which were calculated by the grid search technique (Lp-norm p = 1 and p = 2).
FoldLp-norm p = 1 (S9)Lp-norm p = 2 (S10)CγAccuracy %CγAccuracy %12^112^−71002^132^−598.2122^152^−1598.252^152^−794.7432^112^−91002^132^−596.4942^92^−798.252^132^−592.9852^152^−1596.492^132^−596.4962^152^−1596.492^132^−510072^92^−71002^92^−596.4982^152^−151002^112^−510092^152^−1592.982^132^−598.25102^152^−151002^132^−596.49Avg.20172.80.002698.259267.20.02997.01CPU time13.21700812.847815Tables10 and 11 summarize the accuracy and CPU time of all compared scaling techniques. The equilibration scaling technique (S3) achieved the best accuracy and the lowest CPU time outperforming all compared scaling techniques. Figures 4 and 5 show the superiority of S3 according to the accuracy rate and CPU time, respectively.Table 10
Accuracy for the WBCD database using the SVM for all scaling techniques.
No.SymbolScaling techniquesAccuracy1(S3)Equilibration scaling98.952(S4)Geometric mean scaling98.603(S8)IBM MPSX98.594(S6)de Buchetp = 198.595(S7)de Buchetp = 298.576(S5)Arithmetic mean scaling98.427(S9)Lp-norm p = 198.258(S10)Lp-norm p = 297.019(S2)Normalization (−1, 1)96.6610(S1)Normalization scaling (0,1)96.4911(S0)Without scaling96.10Table 11
CPU time for the WBCD database using the SVM for all scaling techniques.
No.SymbolScaling techniquesCPU time1(S3)Equilibration scaling10.1753302(S8)IBM MPSX11.5167543(S5)Arithmetic mean scaling12.5164964(S10)Lp-norm p = 212.8478155(S6)de Buchetp = 112.992646(S9)Lp-norm p = 113.2170087(S7)de Buchetp = 213.281688(S1)Normalization scaling (0,1)14.5584109(S4)Geometric mean scaling15.14307610(S2)Normalization (−1, 1)19.20879711(S0)Without scaling52.62167Figure 4
An accuracy comparison among all proposed scaling techniques against the normalization [0, 1], normalization [−1, 1], and without scaling technique S0.Figure 5
CPU time comparison among all proposed scaling techniques against the normalization [0, 1], normalization [−1, 1], and without scaling technique S0.Figure6 shows the superiority of equilibration scaling technique (S3) and achieved best accuracy in all 10-fold cross-validation.Figure 6
An accuracy comparison among the equilibration scaling technique against the normalization [0, 1], normalization [−1, 1], and without scaling technique S0 according to 10-fold.


## 6. Conclusions

In this work, we proposed ten efficient scaling techniques for the Wisconsin Diagnosis Breast Cancer (WDBC) dataset using the support vector machine (SVM). These scaling techniques can enhance classification accuracy, reduce CPU time, and make training faster. Also, grid search is used to select best free parameters of the SVM (C, gamma). Simulation results showed that equilibration scaling technique (S3) achieved best accuracy with 98.95% outperforming all compared scaling techniques. S3 also achieved lowest CPU time with about 10.2 seconds. Eight efficient scaling techniques outperformed the two benchmark scaling techniques according to the accuracy rate. These techniques are S3, S4, S5, S6, S7, and S8. Seven of the ten efficient scaling techniques outperformed two benchmark scaling techniques according to the CPU time. These techniques are S3, S5, S6, S7, and S8.In the future work, the proposed scaling techniques will be applied on other data sets with other classifiers in order to prove the superiority of these techniques on the benchmark normalization scaling technique that is used in MATLAB SOFTWARE. We can improve this work by using the different metaheuristic algorithms with other mathematical models [26–30]. Also, swarm intelligence techniques will be used to optimize the SVM instead of grid search [31–33].



---

*Source: 9384318-2021-02-11.xml*