# Local Similarity-Based Fuzzy Multiple Kernel One-Class Support Vector Machine

Authors: Qiang He; Qingshuo Zhang; Hengyou Wang; Changlun Zhang
Journal: Complexity
 (2020)

---

## Abstract

One-class support vector machine (OCSVM) is one of the most popular algorithms in the one-class classification problem, but it has one obvious disadvantage: it is sensitive to noise. In order to solve this problem, the fuzzy membership degree is introduced into OCSVM, which makes the samples with different importance have different influences on the determination of classification hyperplane and enhances the robustness. In this paper, a new calculation method of membership degree is proposed and introduced into the fuzzy multiple kernel OCSVM (FMKOCSVM). The combined kernel is used to measure the local similarity between samples, and then, the importance of samples is determined based on the local similarity between training samples, so as to determine the membership degree and reduce the impact of noise. The proposed membership requires only positive data in the calculation process, which is consistent with the training set of OCSVM. In this method, the noise has a smaller membership value, which can reduce the negative impact of noise on the classification boundary. Simultaneously, this method of calculating membership has a higher efficiency. The experimental results show that FMKOCSVM based on proposed local similarity membership is efficient and more robust to outliers than the ordinary multiple kernel OCSVMs.


---

## Body

## 1. Introduction

Anomaly detection is an important aspect of data mining. It is used to find objects in a data set that are significantly different from other data to achieve the purpose of preventing abnormal events. At present, the application of anomaly detection in the field of medicine and biological systems is of great significance, and it has been successfully applied to protein detection, [1] cancer screening, [2] and health monitoring [3]. The essence of anomaly detection is a classification algorithm suitable for processing data with an extremely imbalanced class. Complex biological systems usually have this feature. For example, the data of an infectious disease model may include characteristic data of patients and characteristic data of nonpatients. But, in real life, there are far more healthy people than patients. Timely and effective detection of patients with infectious diseases is an effective way to prevent the outbreak of infectious diseases.Support vector machine (SVM) [4, 5] is a classical classification algorithm, but its performance will deteriorate when dealing with a one-class classification problem or distribution imbalance data. Among the solutions to one-class classification problems, there are density estimation-based methods and support vector-based methods. The method based on support vector is popular because of its simplicity and high efficiency. There are two models for this method: (1) one-class support vector machine (OCSVM) [6]; (2) support vector data description (SVDD) [7]. The goal of SVDD is to find a minimum hypersphere that contains all target samples. The main idea of OCSVM is to take the origin of the feature space as a representative of the abnormal data and then separate the target sample from the origin at the maximum margin. This paper focuses on OCSVM.Like SVM, OCSVM is also sensitive to noise, which is due to the assumption that each sample has the same importance or weight during training. Introducing fuzzy membership in SVM and constructing the fuzzy support vector machine (FSVM) [8] is one of the effective methods to solve this problem. The calculation methods of fuzzy membership are mainly focused on two kinds of classification problems [9–11]. For example, the heuristic function derived from the centered kernel alignment is used to calculate the dependency relationship between the data point and its label to calculate the fuzzy membership [9] In [10], the membership degree of each sample point is determined by the lower approximation operator of a fuzzy rough set based on Gaussian kernel. As shown in [11], entropy is used to measure the class determinacy of samples. Samples with a higher class determinacy are assigned to a larger fuzzy membership. Generally, in order to improve the robustness of OCSVM, different weights are assigned to training samples, which are called weighted one-class support vector machine (WOCSVM) [12–14]. WOCSVM reduces the impact of noise by assigning lower weights to the noise [12]. As shown in [14], using prior knowledge to assign different weights to the samples, the weight is only related to the distribution knowledge of the neighbors, which is only determined by the k-nearest neighbors of the instance. In recent years, there are many studies on calculating sample weight in one-class classification problems [15–18]. As shown in [19], membership based on fuzzy rough set theory [20, 21] is used as a weight in OCSVM.The above method improves the robustness of OCSVM to a certain extent but has some limitations. For example, when the amount of data is too large, it [14] is too inefficient when calculating sample weights. The authors in [19] use abnormal data when calculating the membership degree of the sample. In this paper, a novel strategy is proposed to solve the problem of poor robustness of OCSVM; that is, membership degree is introduced into the model. Different from the above membership calculation method, this method only uses one category of data, which fully adapts to the characteristics of OCSVM. The membership calculation method proposed is related to the local density of the data, which is obtained by the local similarity of the training data. We take an S-type function based on local density as a membership function.OCSVM uses kernel trick to solve the nonlinear separability problem, but it also brings the problem of kernel selection. Multiple kernel learning method [22–26] is used to solve this problem, that is, the multiple kernel one-class support vector machine (MKOCSVM) [27].The main work of this paper is as follows:(1)
The multiple kernel learning and membership degree are introduced into OCSVM at the same time, and the fuzzy multiple kernel one-class support vector machine (FMKOCSVM) model is constructed to solve the problem of core selection and noise sensitivity.(2)
A novel method of membership calculation is proposed, which is based on local similarity.(3)
We illustrate the effectiveness of this degree of membership in the figure.(4)
According to the maximum similarity between the combination kernel and the ideal kernel, the weight coefficients of the multiple kernels are determined.(5)
It is proved by experiments that the method proposed in this paper performs better than the method of fuzzy membership and the model without membership.The combined kernel more fully characterizes the data than the single kernel. Using multiple kernel functions at the same time can solve the difficult problem of selecting the kernel function and its parameters, and this method can be applied to different sample information.The rest of this paper is organized as follows: Section2 introduces the knowledge of OCSVM, MKOCSVM, and FMKOCSVM. The formulation and algorithm of our FMKOCSVM based on local similarity are detailed in Section 3; and Section 4 reports the experimental results, followed by the conclusion in Section 5.


## 2. Related Information

### 2.1. One-Class Support Vector Machine

Compared with the SVM, OCSVM is suitable for dealing with the problem of data category imbalance or one-class classification. The main idea is to first map the data from the original space to the feature space through nonlinear mapping and then take the origin of the feature space as the representative of outliers to find an optimal classification hyperplane in the feature space, in which the image of normal data can be separated from the origin by the maximum margin. A graphical illustration is shown in Figure1. Figure 1(a) shows the description of the classification in the original space. Figure 1(b) shows the description of the classification in the feature space.Figure 1
A graphical illustration of the one-class support vector machine. The black dot represents the target sample, the box represents the abnormal sample, and the black solid line represents the boundary.
(a)(b)Supposed the training samplesx1,x2,…,xl∈ℜn (n is the dimension of xi), where l is the number of training samples. ϕ⋅ is a function that maps samples to feature spaces. Let ω,ρ denote normal vector and bias term for classification hyperplane in the feature space. The classification hyperplane is expressed as ωTϕx−ρ=0.The goal is to maximize the distance between the classification hyperplane and origin. Then, OCSVM needs to solve the following convex programming [6]:(1)minw,ρ,ξ12ω2+1vl∑i=1lξi−ρ,s.t.ωTϕxi≥ρ−ξi,ξi≥0,i=1,2,…,l.Here,ξi is the slack variable, which means that outliers are allowed to exist, and v∈0,1 is a parameter to control the proportion of support vector and error points. Using the Lagrange multiplier method, the dual problem of the above optimization problem can be written as follows:(2)mina12∑il∑jlαiαjkxi,xj,s.t.∑i=1lαi=1,0≤αi≤1vl,i=1,2,…,l.Because the above process needs to meet the KKT (Karush–Kuhn–Tucker) condition,(3)αi≥0,ωTϕxi−ρ+ξi≥0,αiωTϕxi−ρ+ξi=0,γi≥0,ξi≥0,γiξi=0.The solution of equation (3) corresponds to sample xi, and there is always αi=0 or ωTϕxi=ρ−ξi. When αi=0, the sample xi has no effect on the hyperplane. When αi>0, ωTϕxi=ρ−ξi must be true. In this case, this sample is called a support vector. If αi<1/vl, then γi>0; there must be ξi=0; that is, the sample is located on the maximum separation boundary; if αi=1/vl, then γi=0; in this case, when ξi>0, the sample xi is misclassified, which is called the boundary support vector. As shown in Figure 2, when different values are used, the positions of corresponding sample points are different.Figure 2
The definitions of support vectors. The black dot represents the target sample, and the black dot with a dotted circle represents support vector.LetNSV,NBSV represent the total number of support vectors and the total number of boundary support vectors, respectively. The maximum value of αi is 1/vl and has the following constraints:(4)∑i=1lαi=1,0≤αi≤1vl,i=1,2,…,l.So we can have the following inequality:(5)NBSV1vl≤∑i=1lαi≤NSV1vl,i=1,2,…,l.Multiplyingν on both sides of equation (5) gives(6)NBSV1l≤v≤NSV1l,i=1,2,…,l.It can be known from equation (6) that the value of ν determines the lower bound of the total support vector ratio and the upper bound of the boundary support vector ratio:(7)ω=∑i=1lαiϕxi.The normal vector of the hyperplane can be obtained by using equation (7).LetxtSV denote the tth support vector located on the maximum spacing plane 0<αt<1/vl and the bias term of the hyperplane obtained according to(8)ρ=1nsv∑t=1nsvωTϕxtSV=∑t=1nsvαikxi,xtSV.Thus, the decision function can be written as(9)fx=sgnωTϕx−ρ=sgn∑ilαikxi,x−ρ.For a given test samplex, after substituting x into equation (9), when fx returns +1, the sample is judged as a normal point; when fx returns −1, the sample is judged as an abnormal point.


### 2.2. Multiple Kernel One-Class Support Vector Machine

MKOCSVM replaces the single kernel function in the conventional OCSVM with a combined kernel, which can effectively avoid the difficulty in selecting the kernel function and its parameters.The forms of combined kernel function include linear combination and nonlinear combination, [28] expressed as(10)kηx,z=∑m=1Pηmkmx,z,ηm≥0,(11)kηx,z=∏m=1Pkmxm,zmηm.Here,ηm is the kernel weight of the mth basic kernel.The MKOCSVM model can be formulated as(12)minωη,ρ,ξ12ωη2+1vl∑i=1lξi−ρ,s.t.ωηTϕηxi≥ρ−ξi,ξi≥0,i=1,2,…,l.Here,kηxi,xj=ϕηxi,ϕηxj. The function fx can be written as(13)fx=sgnωηTϕx−ρ=sgn∑ilαikηxi,x−ρ.To seek the optimal combination weight for each basis kernel, the authors in [27] suggest optimizing the maximum kernel-target alignment value of the combination kernel and the ideal kernel, that is, solving the following objective function [29]:(14)maxηAKη,yyT=∑m=1PηmKm,yyTF∑m=1PηmKm,∑m=1PηmKmFyyT,yyTF,=∑m=1P∑i,j=1lηmkijml∑m,s=1PηmηsKm,KsF,s.t.ηm≥0,m=1,2,…,P.Here,Km is the kernel matrix. K1,K2 is the Frobenius inner product between two matrices, which is given by(15)K1,K2F=∑i=1l∑j=1lk1xi,xjk2xi,xj.Only solution equation (16) is needed to obtain the optimal combination weight:(16)maxη∑m=1P∑i,j=1lηmkijm−∑m,s=1PηmηsKm,KsF−δ∑m=1Pηm2,s.t.ηm≥0,m=1,2,…,P.Here,δ is a regularization coefficient.


### 2.3. Fuzzy Multiple Kernel One-Class Support Vector Machine

Letsi denote membership degree of the sample xi, then the training set can be expressed as x1,s1,x2,s2,…,xl,sl∈ℜn, where si∈0,1. The FMKOCSVM needs to solve the following optimal programming:(17)minω,ρ,ξ12ωη2+1vl∑i=1lsiξi−ρ,s.t.ωηTϕηxi≥ρ−ξi,ξi≥0,i=1,2,…,l.Whensi=1,i=1,2,…,l, the FMKOCSVM degenerates to the normal MKOCSVM.After introducing the Lagrange multiplierαi≥0,γi≥0, for each inequality constraint, the Lagrange function of equation (17) is(18)Lωη,ξ,ρ,α,γ=12ωη2+1vl∑i=1lsiξi−ρ−∑i=1lγiξi−∑i=1lαiωηTϕηxi−ρ+ξi.Setting the derivatives with respect toωη, ξ, and ρ to zero, then we can obtain(19)ωη=∑i=1lαiϕηxi,∑i=1lαi=1,αi=1vlsi−γi.Substituting equation (19) into equation (18), the dual form of equation (17) can be written as(20)minα12∑il∑jlαiαjkηxi,xj,s.t.∑i=1lαi=1,0≤αi≤1vlsi,i=1,2,…,l.Obviously, the only difference between the dual problems of MKOCSVM and FMKOCSVM is the upper bound ofαi. The upper bound of αi becomes si/vl in equation (20). The function fx can be written as(21)fx=sgn∑ilαikηxi,x−ρ.In FMKOCSVM, when the noise has a lower membership during training, the negative effect of noise on the classification hyperplane can be reduced.


## 2.1. One-Class Support Vector Machine

Compared with the SVM, OCSVM is suitable for dealing with the problem of data category imbalance or one-class classification. The main idea is to first map the data from the original space to the feature space through nonlinear mapping and then take the origin of the feature space as the representative of outliers to find an optimal classification hyperplane in the feature space, in which the image of normal data can be separated from the origin by the maximum margin. A graphical illustration is shown in Figure1. Figure 1(a) shows the description of the classification in the original space. Figure 1(b) shows the description of the classification in the feature space.Figure 1
A graphical illustration of the one-class support vector machine. The black dot represents the target sample, the box represents the abnormal sample, and the black solid line represents the boundary.
(a)(b)Supposed the training samplesx1,x2,…,xl∈ℜn (n is the dimension of xi), where l is the number of training samples. ϕ⋅ is a function that maps samples to feature spaces. Let ω,ρ denote normal vector and bias term for classification hyperplane in the feature space. The classification hyperplane is expressed as ωTϕx−ρ=0.The goal is to maximize the distance between the classification hyperplane and origin. Then, OCSVM needs to solve the following convex programming [6]:(1)minw,ρ,ξ12ω2+1vl∑i=1lξi−ρ,s.t.ωTϕxi≥ρ−ξi,ξi≥0,i=1,2,…,l.Here,ξi is the slack variable, which means that outliers are allowed to exist, and v∈0,1 is a parameter to control the proportion of support vector and error points. Using the Lagrange multiplier method, the dual problem of the above optimization problem can be written as follows:(2)mina12∑il∑jlαiαjkxi,xj,s.t.∑i=1lαi=1,0≤αi≤1vl,i=1,2,…,l.Because the above process needs to meet the KKT (Karush–Kuhn–Tucker) condition,(3)αi≥0,ωTϕxi−ρ+ξi≥0,αiωTϕxi−ρ+ξi=0,γi≥0,ξi≥0,γiξi=0.The solution of equation (3) corresponds to sample xi, and there is always αi=0 or ωTϕxi=ρ−ξi. When αi=0, the sample xi has no effect on the hyperplane. When αi>0, ωTϕxi=ρ−ξi must be true. In this case, this sample is called a support vector. If αi<1/vl, then γi>0; there must be ξi=0; that is, the sample is located on the maximum separation boundary; if αi=1/vl, then γi=0; in this case, when ξi>0, the sample xi is misclassified, which is called the boundary support vector. As shown in Figure 2, when different values are used, the positions of corresponding sample points are different.Figure 2
The definitions of support vectors. The black dot represents the target sample, and the black dot with a dotted circle represents support vector.LetNSV,NBSV represent the total number of support vectors and the total number of boundary support vectors, respectively. The maximum value of αi is 1/vl and has the following constraints:(4)∑i=1lαi=1,0≤αi≤1vl,i=1,2,…,l.So we can have the following inequality:(5)NBSV1vl≤∑i=1lαi≤NSV1vl,i=1,2,…,l.Multiplyingν on both sides of equation (5) gives(6)NBSV1l≤v≤NSV1l,i=1,2,…,l.It can be known from equation (6) that the value of ν determines the lower bound of the total support vector ratio and the upper bound of the boundary support vector ratio:(7)ω=∑i=1lαiϕxi.The normal vector of the hyperplane can be obtained by using equation (7).LetxtSV denote the tth support vector located on the maximum spacing plane 0<αt<1/vl and the bias term of the hyperplane obtained according to(8)ρ=1nsv∑t=1nsvωTϕxtSV=∑t=1nsvαikxi,xtSV.Thus, the decision function can be written as(9)fx=sgnωTϕx−ρ=sgn∑ilαikxi,x−ρ.For a given test samplex, after substituting x into equation (9), when fx returns +1, the sample is judged as a normal point; when fx returns −1, the sample is judged as an abnormal point.


## 2.2. Multiple Kernel One-Class Support Vector Machine

MKOCSVM replaces the single kernel function in the conventional OCSVM with a combined kernel, which can effectively avoid the difficulty in selecting the kernel function and its parameters.The forms of combined kernel function include linear combination and nonlinear combination, [28] expressed as(10)kηx,z=∑m=1Pηmkmx,z,ηm≥0,(11)kηx,z=∏m=1Pkmxm,zmηm.Here,ηm is the kernel weight of the mth basic kernel.The MKOCSVM model can be formulated as(12)minωη,ρ,ξ12ωη2+1vl∑i=1lξi−ρ,s.t.ωηTϕηxi≥ρ−ξi,ξi≥0,i=1,2,…,l.Here,kηxi,xj=ϕηxi,ϕηxj. The function fx can be written as(13)fx=sgnωηTϕx−ρ=sgn∑ilαikηxi,x−ρ.To seek the optimal combination weight for each basis kernel, the authors in [27] suggest optimizing the maximum kernel-target alignment value of the combination kernel and the ideal kernel, that is, solving the following objective function [29]:(14)maxηAKη,yyT=∑m=1PηmKm,yyTF∑m=1PηmKm,∑m=1PηmKmFyyT,yyTF,=∑m=1P∑i,j=1lηmkijml∑m,s=1PηmηsKm,KsF,s.t.ηm≥0,m=1,2,…,P.Here,Km is the kernel matrix. K1,K2 is the Frobenius inner product between two matrices, which is given by(15)K1,K2F=∑i=1l∑j=1lk1xi,xjk2xi,xj.Only solution equation (16) is needed to obtain the optimal combination weight:(16)maxη∑m=1P∑i,j=1lηmkijm−∑m,s=1PηmηsKm,KsF−δ∑m=1Pηm2,s.t.ηm≥0,m=1,2,…,P.Here,δ is a regularization coefficient.


## 2.3. Fuzzy Multiple Kernel One-Class Support Vector Machine

Letsi denote membership degree of the sample xi, then the training set can be expressed as x1,s1,x2,s2,…,xl,sl∈ℜn, where si∈0,1. The FMKOCSVM needs to solve the following optimal programming:(17)minω,ρ,ξ12ωη2+1vl∑i=1lsiξi−ρ,s.t.ωηTϕηxi≥ρ−ξi,ξi≥0,i=1,2,…,l.Whensi=1,i=1,2,…,l, the FMKOCSVM degenerates to the normal MKOCSVM.After introducing the Lagrange multiplierαi≥0,γi≥0, for each inequality constraint, the Lagrange function of equation (17) is(18)Lωη,ξ,ρ,α,γ=12ωη2+1vl∑i=1lsiξi−ρ−∑i=1lγiξi−∑i=1lαiωηTϕηxi−ρ+ξi.Setting the derivatives with respect toωη, ξ, and ρ to zero, then we can obtain(19)ωη=∑i=1lαiϕηxi,∑i=1lαi=1,αi=1vlsi−γi.Substituting equation (19) into equation (18), the dual form of equation (17) can be written as(20)minα12∑il∑jlαiαjkηxi,xj,s.t.∑i=1lαi=1,0≤αi≤1vlsi,i=1,2,…,l.Obviously, the only difference between the dual problems of MKOCSVM and FMKOCSVM is the upper bound ofαi. The upper bound of αi becomes si/vl in equation (20). The function fx can be written as(21)fx=sgn∑ilαikηxi,x−ρ.In FMKOCSVM, when the noise has a lower membership during training, the negative effect of noise on the classification hyperplane can be reduced.


## 3. Training FMKOCSVM with Local Similarity-Based Membership

Noise in the training set may not belong to any class at all. Therefore, if these samples with uncertainty are distributed near the edges of the target data, the model will overfit. To alleviate this phenomenon, this paper assigns membership to each training point, which makes the samples play a different role in training and reduces the negative impact of noise. In this section, we first introduce the calculation method of membership based on local similarity in detail and then propose the FMKOCSVM algorithm using membership based on local similarity.


### 3.1. Local Similarity-Based Memberships

Assume the target samplex1,x2,…,xl∈ℜn. Let Kη represent the multiple kernel matrix defined by Kη=kηxi,xjl×l, where the expression of multiple kernel function is equation (10).Let all the elements of the upper triangle of the multiple kernel be sorted from large to small, i.e.,Kη=kηxi,xji<j, and then write it as a vector G=g1,g2,…,gt, t=ll−1/2.Next, define a constantε∈0,1 and let h=εt, where εt is the integral part of εt. θ=gh is a threshold. For each sample x, let u represent the total number of kij≥θ,j=1,2,…,l. In other words, ui represents the number of samples in the target sample whose similarity with the sample xi is greater than or equal to the threshold.The kernelkηxi,xj measures the similarity between two target samples xi and xj, and a large kernel indicates a large similarity. If the membership degree of the sample xi to the target class is higher, it is obvious that more samples are similar to the sample xi in the input sample, i.e., the greater the value of ui. In other words, a sample with a higher value of u should have a greater contribution to the classification boundary, the penalty for misclassification of such samples is greater, and the noise will have a smaller value of u.Therefore, we takeu as a measure function, which measures the importance of the target sample to the classification hyperplane. Obviously, the value of u cannot be directly used as a membership degree of FMKOCSVM. We use an S-type function to map this measure into the membership degree in the unit interval. At the same time, this S-type function increases the difference between membership degrees of samples with different importance. The membership function is written as(22)si=11+exp−τui/U−1/2,where U=maxiui. τ>0 is a constant. The value range is 0,1. Figure 3 describes the distribution of membership values by τ taking different values. According to Figure 3, when τ=10, the distribution of membership value is the best. Algorithm 1 lists the detailed calculation process of membership based on local similarity.Figure 3
The distribution of membership with different values ofτ. The horizontal axis is the values of u/U. The vertical axis is the corresponding membership value. The black dashed line is the distribution of membership when τ=1. The red solid line is the distribution of membership when τ=10. The blue dash-dotted line is the distribution of membership when τ=20. The cyan dotted line is the distribution of membership degree when τ=30.Algorithm 1: Local similarity-based memberships.
Input: the training setx1,x2,…,xl, the kernel function set k1,k2,…,kP,The kernel weightηm,m=1,2,…,P,ε∈0,1Output: the membership vectorsi,i=1,2,…,l(1)
Preprocess the training set(2)
Calculate the combined kernel matrixKη=kηijl×l according to equation (10)(3)
SortKη=kηiji<j from large to small as G=g1,g2,…,gt,t=ll−1/2(4)
Calculate the constanth according to h=εt, and fixed threshold according to θ=gh(5)
fori = 1 : l do(6)
initializeui=0(7)
forj = 1 : l do(8)
ifkij≥gh then(9)
ui=ui+1(10)
end if(11)
end for(12)
end for(13)
CalculateU=maxiui(14)
fori = 1 : l do(15)
Calculate the membership degreesi of the sample xi according to equation (22)(16)
end for(17)
endSince noise has a low degree of membership to the target class, there are few similar instances in the input data. In other words, the noise will get a smaller membership value. Therefore, we proposed a method that can make the noise have less influence on the classification boundary. More importantly, because the training data of OCSVM only include target samples, the traditional method of calculating membership degree is not suitable for OCSVM. However, our method of membership based on local similarity only uses the features of the target data and does not involve the information of class, which is very suitable for a one-class classification problem. Furthermore, the proposed method has obvious high efficiency.We analyze the computational complexity of Algorithm1 with the O notation. First, the computational complexity of calculating the multiple kernel matrix in step 2 is Ol2. Second, the average computational complexity of multiple kernel matrix sorting in step 3 is Ot2. Third, it costs Ol2 to calculate u in steps 4 to 12. Finally, calculating the memberships in steps 13 to 16 costs Ol. Hence, the total computational complexity of local similarity-based membership degree is(23)Ol2+Ot2+Ol2+Ol=Ol2+Ol4+Ol2+Ol=Ol4.Compared with classification performance,Ol4 is also acceptable.


### 3.2. Overall Procedure of FMKOCSVM Based on Local Similarity

The detailed process of FMKOCSVM based on membership degree of local similarity is listed in Algorithm2. In the following part of this paper, we use FMKOCSVM_LS to represent the proposed algorithm.Algorithm 2: FMKOCSVM_LS.
Input: the training setx1,x2,…,xl, the kernel function set k1,k2,…,kP, the kernel parameter set σ1,σ2,…,σP, the test set TOutput: the classification results ofT:y(1)
Preprocess the training set(2)
form = 1 : P do(3)
Calculate the corresponding kernel matrixKm of σm(4)
end for(5)
Substituting the kernel matrixKm,m=1,2,…,P into equation (16)(6)
Calculate the kernel weight vectorηm,m=1,2,⋯,P according to equation (16)(7)
Calculate the combined kernel matrixKη according to equation (10)(8)
for eachxi∈x1,x2,⋯,xl do(9)
Calculate the membership degreesi of the sample xi according to equation (22)(10)
end for(11)
Train the FMKOCSVM with the fuzzy membership values.(12)
Calculate the classification results of test set according to equation (21)(13)
endIn Figure4, the classification performance of MKOCSVM is shown in Figure 4(a) and the classification performance of FMKOCSVM_LS proposed by us is shown in Figure 4(b). The combined kernel function is composed of seven Gaussian kernels with a width of 2−5,2−4,2−3,2−2,2−1,20,21. Parameter v is set to 0.02. The value of the regularization coefficient δ in equation (16) is set to 100. In order to reduce the cost of tuning extraparameter, we set ε to 0.2 and τ to 10 directly. Obviously, FMKOCSVM_LS has a tighter boundary than MKOCSVM. In Figure 4(b), some outliers are identified by FMKOCSVM_LS. However, MKOCSVM does not identify any outliers, and there are a lot of gaps inside the boundary.Figure 4
The classification performance of MKOCSVM and FMKOCSVM_LS without noises. The magenta solid line is the boundary. The blue dots are the target samples.
(a)(b)After adding 10% Gaussian noise to the training set, the results are shown in Figure5.Figure 5
The classification performance of MKOCSVM and FMKOCSVM_LS. 10% of Gaussian noise is added. The magenta solid line is the boundary. The blue dots are the target samples. The black crosses are the noises with Gaussian distribution. The number at bottom of the noise represents its membership.
(a)(b)The parameter settings in Figure5 are the same as in Figure 4. As we can see, when there are noises in the training set, the classification ability of FMKOCSVM_LS is much better than that of MKOCSVM. In Figure 5(a), MKOCSVM distinguishes all noises as target data, which makes the performance of MKOCSVM very bad. In Figure 5(b), we can see that most of the noises have a very small membership value, and the negative effect of noise on the boundary is weak. Therefore, FMKOCSVM_LS can improve the robustness of MKOCSVM.In the next section, we will further prove through experiments that our proposed method of calculating membership is better than the previous method.


## 3.1. Local Similarity-Based Memberships

Assume the target samplex1,x2,…,xl∈ℜn. Let Kη represent the multiple kernel matrix defined by Kη=kηxi,xjl×l, where the expression of multiple kernel function is equation (10).Let all the elements of the upper triangle of the multiple kernel be sorted from large to small, i.e.,Kη=kηxi,xji<j, and then write it as a vector G=g1,g2,…,gt, t=ll−1/2.Next, define a constantε∈0,1 and let h=εt, where εt is the integral part of εt. θ=gh is a threshold. For each sample x, let u represent the total number of kij≥θ,j=1,2,…,l. In other words, ui represents the number of samples in the target sample whose similarity with the sample xi is greater than or equal to the threshold.The kernelkηxi,xj measures the similarity between two target samples xi and xj, and a large kernel indicates a large similarity. If the membership degree of the sample xi to the target class is higher, it is obvious that more samples are similar to the sample xi in the input sample, i.e., the greater the value of ui. In other words, a sample with a higher value of u should have a greater contribution to the classification boundary, the penalty for misclassification of such samples is greater, and the noise will have a smaller value of u.Therefore, we takeu as a measure function, which measures the importance of the target sample to the classification hyperplane. Obviously, the value of u cannot be directly used as a membership degree of FMKOCSVM. We use an S-type function to map this measure into the membership degree in the unit interval. At the same time, this S-type function increases the difference between membership degrees of samples with different importance. The membership function is written as(22)si=11+exp−τui/U−1/2,where U=maxiui. τ>0 is a constant. The value range is 0,1. Figure 3 describes the distribution of membership values by τ taking different values. According to Figure 3, when τ=10, the distribution of membership value is the best. Algorithm 1 lists the detailed calculation process of membership based on local similarity.Figure 3
The distribution of membership with different values ofτ. The horizontal axis is the values of u/U. The vertical axis is the corresponding membership value. The black dashed line is the distribution of membership when τ=1. The red solid line is the distribution of membership when τ=10. The blue dash-dotted line is the distribution of membership when τ=20. The cyan dotted line is the distribution of membership degree when τ=30.Algorithm 1: Local similarity-based memberships.
Input: the training setx1,x2,…,xl, the kernel function set k1,k2,…,kP,The kernel weightηm,m=1,2,…,P,ε∈0,1Output: the membership vectorsi,i=1,2,…,l(1)
Preprocess the training set(2)
Calculate the combined kernel matrixKη=kηijl×l according to equation (10)(3)
SortKη=kηiji<j from large to small as G=g1,g2,…,gt,t=ll−1/2(4)
Calculate the constanth according to h=εt, and fixed threshold according to θ=gh(5)
fori = 1 : l do(6)
initializeui=0(7)
forj = 1 : l do(8)
ifkij≥gh then(9)
ui=ui+1(10)
end if(11)
end for(12)
end for(13)
CalculateU=maxiui(14)
fori = 1 : l do(15)
Calculate the membership degreesi of the sample xi according to equation (22)(16)
end for(17)
endSince noise has a low degree of membership to the target class, there are few similar instances in the input data. In other words, the noise will get a smaller membership value. Therefore, we proposed a method that can make the noise have less influence on the classification boundary. More importantly, because the training data of OCSVM only include target samples, the traditional method of calculating membership degree is not suitable for OCSVM. However, our method of membership based on local similarity only uses the features of the target data and does not involve the information of class, which is very suitable for a one-class classification problem. Furthermore, the proposed method has obvious high efficiency.We analyze the computational complexity of Algorithm1 with the O notation. First, the computational complexity of calculating the multiple kernel matrix in step 2 is Ol2. Second, the average computational complexity of multiple kernel matrix sorting in step 3 is Ot2. Third, it costs Ol2 to calculate u in steps 4 to 12. Finally, calculating the memberships in steps 13 to 16 costs Ol. Hence, the total computational complexity of local similarity-based membership degree is(23)Ol2+Ot2+Ol2+Ol=Ol2+Ol4+Ol2+Ol=Ol4.Compared with classification performance,Ol4 is also acceptable.


## 3.2. Overall Procedure of FMKOCSVM Based on Local Similarity

The detailed process of FMKOCSVM based on membership degree of local similarity is listed in Algorithm2. In the following part of this paper, we use FMKOCSVM_LS to represent the proposed algorithm.Algorithm 2: FMKOCSVM_LS.
Input: the training setx1,x2,…,xl, the kernel function set k1,k2,…,kP, the kernel parameter set σ1,σ2,…,σP, the test set TOutput: the classification results ofT:y(1)
Preprocess the training set(2)
form = 1 : P do(3)
Calculate the corresponding kernel matrixKm of σm(4)
end for(5)
Substituting the kernel matrixKm,m=1,2,…,P into equation (16)(6)
Calculate the kernel weight vectorηm,m=1,2,⋯,P according to equation (16)(7)
Calculate the combined kernel matrixKη according to equation (10)(8)
for eachxi∈x1,x2,⋯,xl do(9)
Calculate the membership degreesi of the sample xi according to equation (22)(10)
end for(11)
Train the FMKOCSVM with the fuzzy membership values.(12)
Calculate the classification results of test set according to equation (21)(13)
endIn Figure4, the classification performance of MKOCSVM is shown in Figure 4(a) and the classification performance of FMKOCSVM_LS proposed by us is shown in Figure 4(b). The combined kernel function is composed of seven Gaussian kernels with a width of 2−5,2−4,2−3,2−2,2−1,20,21. Parameter v is set to 0.02. The value of the regularization coefficient δ in equation (16) is set to 100. In order to reduce the cost of tuning extraparameter, we set ε to 0.2 and τ to 10 directly. Obviously, FMKOCSVM_LS has a tighter boundary than MKOCSVM. In Figure 4(b), some outliers are identified by FMKOCSVM_LS. However, MKOCSVM does not identify any outliers, and there are a lot of gaps inside the boundary.Figure 4
The classification performance of MKOCSVM and FMKOCSVM_LS without noises. The magenta solid line is the boundary. The blue dots are the target samples.
(a)(b)After adding 10% Gaussian noise to the training set, the results are shown in Figure5.Figure 5
The classification performance of MKOCSVM and FMKOCSVM_LS. 10% of Gaussian noise is added. The magenta solid line is the boundary. The blue dots are the target samples. The black crosses are the noises with Gaussian distribution. The number at bottom of the noise represents its membership.
(a)(b)The parameter settings in Figure5 are the same as in Figure 4. As we can see, when there are noises in the training set, the classification ability of FMKOCSVM_LS is much better than that of MKOCSVM. In Figure 5(a), MKOCSVM distinguishes all noises as target data, which makes the performance of MKOCSVM very bad. In Figure 5(b), we can see that most of the noises have a very small membership value, and the negative effect of noise on the boundary is weak. Therefore, FMKOCSVM_LS can improve the robustness of MKOCSVM.In the next section, we will further prove through experiments that our proposed method of calculating membership is better than the previous method.


## 4. Experiments

### 4.1. Experiment Setup

#### 4.1.1. Approaches

We compared FMKOCSVM_LS with the following methods: (1) MKOCSVM: the ordinary multiple kernel one-class support vector machine [27]; (2) WMKOCSVM: the weighted one-class support vector machine is formed by WOCSVM [14] combined with the multiple kernel function; (3) FMKOCSVM: the fuzzy multiple kernel one-class support vector machine, in which membership is calculated based on a rough set [19]. Because two classes of samples are needed to calculate membership, the training set contains negative class samples. These negative samples are only used to calculate membership.In MKOCSVM, the parameterv is determined by 10-fold cross validation, and the value range is 0.01,0.04,0.07,0.10,0.13,0.16,0.19. The basic kernel of the multiple kernel function is seven Gauss kernel functions with kernel width 2−6,2−5,2−4,2−3,2−2,2−1,20. The parameter δ in the multiple kernel learning algorithm is set to 100. WMKOCSVM, FMKOCSVM, and FMKOCSVM_LS also use these parameters during training. The number of nearest neighbors in WMKOCSVM is set to 10, which is the same as in [14]. In order to avoid increasing the time due to the adjustment of parameters, when calculating the membership based on local similarity in FMKOCSVM_LS, we directly set ε=0.5 and τ=10.


#### 4.1.2. Metrics

In this paper, the performance of different approaches is evaluated by three popular metrics, namely, g-mean, AUC, and training time. According to the confusion matrix in Table1, we can get the true positive rate (TPR) and the false positive rate (FPR). In one-class classification problems, using g-mean and AUC as measures is more accurate than using accuracy:(24)TPR=TPTP+FN,FPR=FPFP+TN,g−mean=a+×a−,AUC=1+TPR−FPR2.Table 1
Confusion matrix.
Predicted positivePredicted negativePositiveTPFNNegativeFPTN


#### 4.1.3. Data Sets

In this section, we selected 14 benchmark data sets, 13 of which are from the UCI machine learning repository. There are three experiments on biological systems. The Heart data set is a data set used for heart disease diagnosis. The Breast data set is a data set used to diagnose whether a patient’s breast cancer is benign or malignant. The Biomed data set is used to screen whether it is a carrier. Creditcard_cut is a part of the data set of creditcard fraud detection on Kaggle. Because the original Creditcard data set is too large, we only randomly selected 729 transaction data (483 normal transactions and 249 fraudulent transactions) for the experiment. Table2 lists the details of these data sets.Table 2
Data information.
NumberData setPositive samplesNegative samplesSamplesFeatures1Australia383307690142Balancescaleleft28833762543Biomed1276719454Glass7014421495Heart160137297136Vowel48480528107Wine48130178138Creditcard_cut483249729309Japan3572946511510Iris50100150411Breast458241699912Wdbc3572125693013Pima500268768814Waveform30060090021For each data set, we use 70% of the positive data as the training set. Then, we randomly selected a part of negative data as the noise in the training set, and the proportion of noise was 10%. The rest of the data is used as the test set. The training set is normalized before training. And the test set is processed according to the standard of the training set.


### 4.2. Results

In order to obtain stable results, each method has done 20 independent experiments on each data set. The result used in the comparison is the average of the 20 results. Table3 shows the optimal value of v obtained through the 10-fold cross validation. In order to get the best results of the four algorithms, v in Table 3 is used in each experiment.Table 3
The optimal value ofv.
NumberData setMKOCSVMWMKOCSVMFMKOCSVMFMKOCSVM_LS1Australia0.190.10.190.072Balancescaleleft0.190.10.070.043Biomed0.160.040.070.014Glass0.070.070.040.015Heart0.190.070.190.046Vowel0.010.010.010.017Wine0.010.010.010.018Creditcard_cut0.130.040.070.019Japan0.190.10.190.1310Iris0.010.010.010.0111Breast0.040.010.040.0112Wdbc0.070.010.040.0113Pima0.190.10.10.0714Waveform0.160.040.070.01Table4 is a comparison of g-mean. Table 5 shows a comparison of AUC values. Table 6 shows the average training time of MKOCSVM, WMKOCSVM, FMKOCSVM, and FMKOCSVM_LS on each data set. Figure 6 is the total training time of 14 data sets of each method.Table 4
Comparison of g-mean.
NumberData setg-mean (mean ± std (%))MKOCSVMWMKOCSVMFMKOCSVMFMKOCSVM_LS1Australia59.19 ± 2.2265.08 ± 2.3974.18 ± 2.6874.74 ± 2.512Balancescaleleft61.38 ± 2.0566.17 ± 2.2966.73 ± 3.2269.99 ± 3.093Biomed72.43 ± 5.5976.80 ± 4.5378.71 ± 2.9881.07 ± 4.154Glass67.74 ± 5.4174.02 ± 4.3870.20 ± 5.3472.51 ± 6.645Heart63.90 ± 1.7963.76 ± 3.9067.82 ± 5.0570.51 ± 3.456Vowel79.46 ± 8.2679.02 ± 4.7980.42 ± 5.7284.88 ± 6.247Wine81.49 ± 7.5981.20 ± 7.1782.87 ± 6.7185.68 ± 9.078Creditcard_cut72.00 ± 3.3774.26 ± 3.9081.36 ± 2.1883.66 ± 1.919Japan58.23 ± 4.0864.59 ± 2.7773.10 ± 2.9168.87 ± 3.5210Iris63.87 ± 15.2369.67 ± 6.9575.48 ± 13.6593.87 ± 4.7611Breast60.71 ± 4.8565.72 ± 5.2881.22 ± 4.5492.00 ± 1.9312Wdbc59.06 ± 5.0355.31 ± 6.7873.08 ± 4.2486.41 ± 1.8313Pima51.92 ± 1.9161.03 ± 2.1862.39 ± 2.1864.92 ± 2.0914Waveform68.99 ± 4.3967.65 ± 3.4974.13 ± 2.5278.23 ± 1.87Table 5
Comparison of AUC value.
NumberData setAUC (mean ± std (%))MKOCSVMWMKOCSVMFMKOCSVMFMKOCSVM_LS1Australia62.25 ± 1.9965.43 ± 2.3274.35 ± 2.6274.88 ± 2.492Balancescaleleft64.21 ± 1.9166.88 ± 2.2768.10 ± 3.1770.64 ± 2.843Biomed73.27 ± 5.1777.10 ± 4.4179.05 ± 2.8081.22 ± 4.114Glass71.26 ± 4.3174.29 ± 4.1371.26 ± 5.3373.63 ± 6.375Heart64.81 ± 1.6664.08 ± 3.8968.00 ± 5.1071.09 ± 2.996Vowel80.17 ± 7.3279.49 ± 4.6580.78 ± 5.6485.32 ± 5.717Wine82.23 ± 6.8481.74 ± 7.0283.62 ± 1.9587.04 ± 7.178Creditcard_cut73.86 ± 2.6775.25 ± 3.1881.44 ± 2.2083.70 ± 1.909Japan60.84 ± 3.4464.86 ± 2.7973. 24 ± 2.8469.03 ± 3.4810Iris69.90 ± 10.5471.75 ± 6.3278.24 ± 9.8294.17 ± 4.4411Breast67.87 ± 2.9571.04 ± 3.6182.53 ± 3.7692.34 ± 1.7612Wdbc65.65 ± 3.2563.44 ± 4.0275.17 ± 3.4486.50 ± 1.7313Pima57.33 ± 1.8461.82 ± 2.0762.75 ± 2.0365.09 ± 2.1814Waveform70.70 ± 3.8669.80 ± 2.9474.35 ± 2.5378.80 ± 1.82Table 6
Average training time.
NumberData setTraining time (s)MKOCSVMWMKOCSVMFMKOCSVMFMKOCSVM_LS1Australia20.6048.6522.6728.222Balancescaleleft13.1330.3016.8519.343Biomed5.4312.756.167.684Glass1.525.313.172.925Heart7.7517.877.969.646Vowel3.575.114.582.957Wine1.314.891.443.218Creditcard_cut38.5175.9347.1954.969Japan18.8144.1820.3024.4210Iris1.224.221.701.8811Breast32.8651.2733.2447.5812Wdbc18.7145.8923.8628.2613Pima38.4968.9451.4257.0814Waveform14.9137.2020.8421.97Figure 6
The total training time of 14 data sets of MKOCSVM, WMKOCSVM, FMKOCSVM, and FMKOCSVM_LS.From Tables4 and 5, we can find that the performance of FMKOCSVM_LS is the best among the four algorithms, which proves that our membership method can improve the robustness of MKOCSVM. More importantly, WMKOCSVM and FMKOCSVM have only one best result on 14 data sets, respectively. However, our proposed method has twelve optimal performances.On the Iris, Breast, and Wdbc data sets, FMKOCSVM_LS shows great advantages. Its g-mean is 27%∼32% higher than that of MKOCSVM, 10%∼18% higher than that of FMKOCSVM, and 23%∼31% higher than that of WMKOCSVM. In the corresponding data set, the AUC value of FMKOCSVM_LS also increased significantly. On the Japan data set, although the g-mean of FMKOCSVM LS is lower than the g-mean of FMKOCSVM, it is still 10% higher than the g-mean of MKOCSVM and 4% higher than the g-mean of WMKOCSVM. AUC value is the same as g-mean. On the Glass data set, WMKOCSVM has the best result, and its result is only about 2% higher than that of FMKOCSVM_LS. However, on the Glass data set, the result of FMKOCSVM_LS is 5% higher than that of MKOCSVM, which proves that our membership calculation method can reduce the impact of noise on classification ability. In the remaining 9 data, the results of FMKOCSVM_LS are the best and have obvious advantages. For example, on the Waveform data set, the g-mean of FMKOCSVM_LS is 10% higher than that of WMKOCSVM and 4% higher than that of FMKOCSVM.In terms of training time, although our method is not the fastest, FMKOCSVM_LS is still faster than WMKOCSVM. The training time of WMKOCSVM is 1.5 times of that of FMKOCSVM_LS on average. Compared with the training time of MKOCSVM, the increased training time of FMKOCSVM_LS is within the acceptable range.All of the above proves that the MKOCSVM with membership is more robust. Moreover, our proposed membership based on local similarity is the best.


## 4.1. Experiment Setup

### 4.1.1. Approaches

We compared FMKOCSVM_LS with the following methods: (1) MKOCSVM: the ordinary multiple kernel one-class support vector machine [27]; (2) WMKOCSVM: the weighted one-class support vector machine is formed by WOCSVM [14] combined with the multiple kernel function; (3) FMKOCSVM: the fuzzy multiple kernel one-class support vector machine, in which membership is calculated based on a rough set [19]. Because two classes of samples are needed to calculate membership, the training set contains negative class samples. These negative samples are only used to calculate membership.In MKOCSVM, the parameterv is determined by 10-fold cross validation, and the value range is 0.01,0.04,0.07,0.10,0.13,0.16,0.19. The basic kernel of the multiple kernel function is seven Gauss kernel functions with kernel width 2−6,2−5,2−4,2−3,2−2,2−1,20. The parameter δ in the multiple kernel learning algorithm is set to 100. WMKOCSVM, FMKOCSVM, and FMKOCSVM_LS also use these parameters during training. The number of nearest neighbors in WMKOCSVM is set to 10, which is the same as in [14]. In order to avoid increasing the time due to the adjustment of parameters, when calculating the membership based on local similarity in FMKOCSVM_LS, we directly set ε=0.5 and τ=10.


### 4.1.2. Metrics

In this paper, the performance of different approaches is evaluated by three popular metrics, namely, g-mean, AUC, and training time. According to the confusion matrix in Table1, we can get the true positive rate (TPR) and the false positive rate (FPR). In one-class classification problems, using g-mean and AUC as measures is more accurate than using accuracy:(24)TPR=TPTP+FN,FPR=FPFP+TN,g−mean=a+×a−,AUC=1+TPR−FPR2.Table 1
Confusion matrix.
Predicted positivePredicted negativePositiveTPFNNegativeFPTN


### 4.1.3. Data Sets

In this section, we selected 14 benchmark data sets, 13 of which are from the UCI machine learning repository. There are three experiments on biological systems. The Heart data set is a data set used for heart disease diagnosis. The Breast data set is a data set used to diagnose whether a patient’s breast cancer is benign or malignant. The Biomed data set is used to screen whether it is a carrier. Creditcard_cut is a part of the data set of creditcard fraud detection on Kaggle. Because the original Creditcard data set is too large, we only randomly selected 729 transaction data (483 normal transactions and 249 fraudulent transactions) for the experiment. Table2 lists the details of these data sets.Table 2
Data information.
NumberData setPositive samplesNegative samplesSamplesFeatures1Australia383307690142Balancescaleleft28833762543Biomed1276719454Glass7014421495Heart160137297136Vowel48480528107Wine48130178138Creditcard_cut483249729309Japan3572946511510Iris50100150411Breast458241699912Wdbc3572125693013Pima500268768814Waveform30060090021For each data set, we use 70% of the positive data as the training set. Then, we randomly selected a part of negative data as the noise in the training set, and the proportion of noise was 10%. The rest of the data is used as the test set. The training set is normalized before training. And the test set is processed according to the standard of the training set.


## 4.1.1. Approaches

We compared FMKOCSVM_LS with the following methods: (1) MKOCSVM: the ordinary multiple kernel one-class support vector machine [27]; (2) WMKOCSVM: the weighted one-class support vector machine is formed by WOCSVM [14] combined with the multiple kernel function; (3) FMKOCSVM: the fuzzy multiple kernel one-class support vector machine, in which membership is calculated based on a rough set [19]. Because two classes of samples are needed to calculate membership, the training set contains negative class samples. These negative samples are only used to calculate membership.In MKOCSVM, the parameterv is determined by 10-fold cross validation, and the value range is 0.01,0.04,0.07,0.10,0.13,0.16,0.19. The basic kernel of the multiple kernel function is seven Gauss kernel functions with kernel width 2−6,2−5,2−4,2−3,2−2,2−1,20. The parameter δ in the multiple kernel learning algorithm is set to 100. WMKOCSVM, FMKOCSVM, and FMKOCSVM_LS also use these parameters during training. The number of nearest neighbors in WMKOCSVM is set to 10, which is the same as in [14]. In order to avoid increasing the time due to the adjustment of parameters, when calculating the membership based on local similarity in FMKOCSVM_LS, we directly set ε=0.5 and τ=10.


## 4.1.2. Metrics

In this paper, the performance of different approaches is evaluated by three popular metrics, namely, g-mean, AUC, and training time. According to the confusion matrix in Table1, we can get the true positive rate (TPR) and the false positive rate (FPR). In one-class classification problems, using g-mean and AUC as measures is more accurate than using accuracy:(24)TPR=TPTP+FN,FPR=FPFP+TN,g−mean=a+×a−,AUC=1+TPR−FPR2.Table 1
Confusion matrix.
Predicted positivePredicted negativePositiveTPFNNegativeFPTN


## 4.1.3. Data Sets

In this section, we selected 14 benchmark data sets, 13 of which are from the UCI machine learning repository. There are three experiments on biological systems. The Heart data set is a data set used for heart disease diagnosis. The Breast data set is a data set used to diagnose whether a patient’s breast cancer is benign or malignant. The Biomed data set is used to screen whether it is a carrier. Creditcard_cut is a part of the data set of creditcard fraud detection on Kaggle. Because the original Creditcard data set is too large, we only randomly selected 729 transaction data (483 normal transactions and 249 fraudulent transactions) for the experiment. Table2 lists the details of these data sets.Table 2
Data information.
NumberData setPositive samplesNegative samplesSamplesFeatures1Australia383307690142Balancescaleleft28833762543Biomed1276719454Glass7014421495Heart160137297136Vowel48480528107Wine48130178138Creditcard_cut483249729309Japan3572946511510Iris50100150411Breast458241699912Wdbc3572125693013Pima500268768814Waveform30060090021For each data set, we use 70% of the positive data as the training set. Then, we randomly selected a part of negative data as the noise in the training set, and the proportion of noise was 10%. The rest of the data is used as the test set. The training set is normalized before training. And the test set is processed according to the standard of the training set.


## 4.2. Results

In order to obtain stable results, each method has done 20 independent experiments on each data set. The result used in the comparison is the average of the 20 results. Table3 shows the optimal value of v obtained through the 10-fold cross validation. In order to get the best results of the four algorithms, v in Table 3 is used in each experiment.Table 3
The optimal value ofv.
NumberData setMKOCSVMWMKOCSVMFMKOCSVMFMKOCSVM_LS1Australia0.190.10.190.072Balancescaleleft0.190.10.070.043Biomed0.160.040.070.014Glass0.070.070.040.015Heart0.190.070.190.046Vowel0.010.010.010.017Wine0.010.010.010.018Creditcard_cut0.130.040.070.019Japan0.190.10.190.1310Iris0.010.010.010.0111Breast0.040.010.040.0112Wdbc0.070.010.040.0113Pima0.190.10.10.0714Waveform0.160.040.070.01Table4 is a comparison of g-mean. Table 5 shows a comparison of AUC values. Table 6 shows the average training time of MKOCSVM, WMKOCSVM, FMKOCSVM, and FMKOCSVM_LS on each data set. Figure 6 is the total training time of 14 data sets of each method.Table 4
Comparison of g-mean.
NumberData setg-mean (mean ± std (%))MKOCSVMWMKOCSVMFMKOCSVMFMKOCSVM_LS1Australia59.19 ± 2.2265.08 ± 2.3974.18 ± 2.6874.74 ± 2.512Balancescaleleft61.38 ± 2.0566.17 ± 2.2966.73 ± 3.2269.99 ± 3.093Biomed72.43 ± 5.5976.80 ± 4.5378.71 ± 2.9881.07 ± 4.154Glass67.74 ± 5.4174.02 ± 4.3870.20 ± 5.3472.51 ± 6.645Heart63.90 ± 1.7963.76 ± 3.9067.82 ± 5.0570.51 ± 3.456Vowel79.46 ± 8.2679.02 ± 4.7980.42 ± 5.7284.88 ± 6.247Wine81.49 ± 7.5981.20 ± 7.1782.87 ± 6.7185.68 ± 9.078Creditcard_cut72.00 ± 3.3774.26 ± 3.9081.36 ± 2.1883.66 ± 1.919Japan58.23 ± 4.0864.59 ± 2.7773.10 ± 2.9168.87 ± 3.5210Iris63.87 ± 15.2369.67 ± 6.9575.48 ± 13.6593.87 ± 4.7611Breast60.71 ± 4.8565.72 ± 5.2881.22 ± 4.5492.00 ± 1.9312Wdbc59.06 ± 5.0355.31 ± 6.7873.08 ± 4.2486.41 ± 1.8313Pima51.92 ± 1.9161.03 ± 2.1862.39 ± 2.1864.92 ± 2.0914Waveform68.99 ± 4.3967.65 ± 3.4974.13 ± 2.5278.23 ± 1.87Table 5
Comparison of AUC value.
NumberData setAUC (mean ± std (%))MKOCSVMWMKOCSVMFMKOCSVMFMKOCSVM_LS1Australia62.25 ± 1.9965.43 ± 2.3274.35 ± 2.6274.88 ± 2.492Balancescaleleft64.21 ± 1.9166.88 ± 2.2768.10 ± 3.1770.64 ± 2.843Biomed73.27 ± 5.1777.10 ± 4.4179.05 ± 2.8081.22 ± 4.114Glass71.26 ± 4.3174.29 ± 4.1371.26 ± 5.3373.63 ± 6.375Heart64.81 ± 1.6664.08 ± 3.8968.00 ± 5.1071.09 ± 2.996Vowel80.17 ± 7.3279.49 ± 4.6580.78 ± 5.6485.32 ± 5.717Wine82.23 ± 6.8481.74 ± 7.0283.62 ± 1.9587.04 ± 7.178Creditcard_cut73.86 ± 2.6775.25 ± 3.1881.44 ± 2.2083.70 ± 1.909Japan60.84 ± 3.4464.86 ± 2.7973. 24 ± 2.8469.03 ± 3.4810Iris69.90 ± 10.5471.75 ± 6.3278.24 ± 9.8294.17 ± 4.4411Breast67.87 ± 2.9571.04 ± 3.6182.53 ± 3.7692.34 ± 1.7612Wdbc65.65 ± 3.2563.44 ± 4.0275.17 ± 3.4486.50 ± 1.7313Pima57.33 ± 1.8461.82 ± 2.0762.75 ± 2.0365.09 ± 2.1814Waveform70.70 ± 3.8669.80 ± 2.9474.35 ± 2.5378.80 ± 1.82Table 6
Average training time.
NumberData setTraining time (s)MKOCSVMWMKOCSVMFMKOCSVMFMKOCSVM_LS1Australia20.6048.6522.6728.222Balancescaleleft13.1330.3016.8519.343Biomed5.4312.756.167.684Glass1.525.313.172.925Heart7.7517.877.969.646Vowel3.575.114.582.957Wine1.314.891.443.218Creditcard_cut38.5175.9347.1954.969Japan18.8144.1820.3024.4210Iris1.224.221.701.8811Breast32.8651.2733.2447.5812Wdbc18.7145.8923.8628.2613Pima38.4968.9451.4257.0814Waveform14.9137.2020.8421.97Figure 6
The total training time of 14 data sets of MKOCSVM, WMKOCSVM, FMKOCSVM, and FMKOCSVM_LS.From Tables4 and 5, we can find that the performance of FMKOCSVM_LS is the best among the four algorithms, which proves that our membership method can improve the robustness of MKOCSVM. More importantly, WMKOCSVM and FMKOCSVM have only one best result on 14 data sets, respectively. However, our proposed method has twelve optimal performances.On the Iris, Breast, and Wdbc data sets, FMKOCSVM_LS shows great advantages. Its g-mean is 27%∼32% higher than that of MKOCSVM, 10%∼18% higher than that of FMKOCSVM, and 23%∼31% higher than that of WMKOCSVM. In the corresponding data set, the AUC value of FMKOCSVM_LS also increased significantly. On the Japan data set, although the g-mean of FMKOCSVM LS is lower than the g-mean of FMKOCSVM, it is still 10% higher than the g-mean of MKOCSVM and 4% higher than the g-mean of WMKOCSVM. AUC value is the same as g-mean. On the Glass data set, WMKOCSVM has the best result, and its result is only about 2% higher than that of FMKOCSVM_LS. However, on the Glass data set, the result of FMKOCSVM_LS is 5% higher than that of MKOCSVM, which proves that our membership calculation method can reduce the impact of noise on classification ability. In the remaining 9 data, the results of FMKOCSVM_LS are the best and have obvious advantages. For example, on the Waveform data set, the g-mean of FMKOCSVM_LS is 10% higher than that of WMKOCSVM and 4% higher than that of FMKOCSVM.In terms of training time, although our method is not the fastest, FMKOCSVM_LS is still faster than WMKOCSVM. The training time of WMKOCSVM is 1.5 times of that of FMKOCSVM_LS on average. Compared with the training time of MKOCSVM, the increased training time of FMKOCSVM_LS is within the acceptable range.All of the above proves that the MKOCSVM with membership is more robust. Moreover, our proposed membership based on local similarity is the best.


## 5. Conclusions

In order to solve the problem of poor robustness of MKOCSVM, this paper proposes a fuzzy multiple kernel one-class support vector machine based on local similarity, in which membership is based on the local similarity of the training data. Firstly, the similarity between samples is measured by combining the kernel matrix. Then, according to the selected threshold, the local similarity of each sample is determined. Finally, an S-type function is used to map the local similarity to the unit interval, and the function value is taken as the membership value. Experiments show that the membership method proposed in this paper can improve the robustness of MKOCSVM. Moreover, compared with the other two methods, our method is optimal.The difficulty in fuzzy multiple kernel one-class support vector machine lies in how to determine the effective membership. Compared with the previous membership calculation method, only the target data are needed to calculate the membership based on the local similarity of the data, which is consistent with the OCSVM training set. In this method, the noise or outliers are assigned a small membership value, which makes the noise have the weakest impact on the classification boundary. Therefore, the membership method in this paper helps improve the robustness of MKOCSVM. In the next step, we will research the optimization method of parameters in the process of membership calculation based on local similarity.



---

*Source: 8853277-2020-10-28.xml*