# A Corporate Credit Rating Model Using Support Vector Domain Combined with Fuzzy Clustering Algorithm

Authors: Xuesong Guo; Zhengwei Zhu; Jia Shi
Journal: Mathematical Problems in Engineering
 (2012)

---

## Abstract

Corporate credit-rating prediction using statistical and artificial intelligence techniques has received considerable attentions in the literature. Different from the thoughts of various techniques for adopting support vector machines as binary classifiers originally, a new method, based on support vector domain combined with fuzzy clustering algorithm for multiclassification, is proposed in the paper to accomplish corporate credit rating. By data preprocessing using fuzzy clustering algorithm, only the boundary data points are selected as training samples to accomplish support vector domain specification to reduce computational cost and also achieve better performance. To validate the proposed methodology, real-world cases are used for experiments, with results compared with conventional multiclassification support vector machine approaches and other artificial intelligence techniques. The results show that the proposed model improves the performance of corporate credit-rating with less computational consumption.


---

## Body

## 1. Introduction

Techniques of credit ratings have been applied by bond investors, debt issuers, and governmental officials as one of the most efficient measures of risk management. However, company credit ratings are too costly to obtain, because agencies including Standard and Poor’s (S&P), and Moody’s are required to invest lots of time and human resources to accomplish critical analysis based on various aspects ranging from strategic competitiveness to operational level in detail [1–3]. Moreover, from a technical perspective, credit rating constitutes a typical multiclassification problem, because the agencies generally have much more than two categories of ratings. For example, ratings from S&P range from AAA for the highest-quality bonds to D for the lowest-quality ones.The final objective of credit rating prediction is to develop the models, by which knowledge of credit risk evaluation can be extracted from experiences of experts and to be applied in much broader scope. Besides prediction, the studies can also help users capture fundamental characteristics of different financial markets by analyzing the information applied by experts.Although rating agencies take emphasis on experts’ subjective judgment in obtaining ratings, many promising results on credit rating prediction based on different statistical and Artificial Intelligence (AI) methods have been proposed, with a grand assumption that financial variables extracted from general statements, such as financial ratios, contain lots of information about company’s credit risk, embedded in their valuable experiences [4, 5].Among the technologies based on AI applied in credit rating prediction, the Artificial Neural Networks (ANNs) have been applied in the domain of finance because of the ability to learn from training samples. Moreover, in terms of defects of ANN such as overfitting, Support Vector Machine (SVM) has been regarded as one of the popular alternative solutions to the problems, because of its much better performance than traditional approaches such as ANN [6–11]. That is, an SVM’s solution can be globally optimal because the models seek to minimize the structural risk [12]. Conversely, the solutions found by ANN tend to fall into local optimum because of seeking to minimize the empirical risk.However, SVM, which was originally developed for binary classification, is not naturally modified for multiclassification of many problems including credit ratings. Thus, researchers have tried to extend original SVM to multiclassification problems [13], with some techniques of multiclassification SVM (MSVM) proposed, which include approaches that construct and combine several binary classifiers as well as the ones that directly consider all the data in a single optimization formulation.In terms of multiclassification in the domain of credit rating containing lots of data, current approaches applied in MSVM still have some drawbacks in integration of multiple binary classifiers as follows.(1)
Some unclassifiable regions may exist if a data point belongs to more than one class or to none.(2)
Training binary classifiers based on two-class SVM multiple times for the same data set often result in a highly intensive time complexity for large-scale problems including credit ratings prediction to improve computational consumption.To overcome the drawbacks associated with current MSVM in credit rating prediction, a novel model based on support vector domain combined with kernel-based fuzzy clustering is proposed in the paper to accomplish multiclassification involved in credit ratings prediction.


## 2. Literature Review

### 2.1. Credit Rating Using Data Mining Techniques

Major researches applying data mining techniques for bond rating prediction can be found in the literature.Early investigations of credit rating techniques mainly focused on the applicability of statistical techniques including multiple discriminant analysis (MDA) [14, 15] and logistic regression analysis (LRA) [16], and so forth, while typical techniques of AI including ANN [17, 18] and case-based reasoning (CBR) [19], and so forth are applied in the second phase of research.The important researches applying AI techniques in bond-rating prediction are listed in Table1. In summary, the most prior ones accomplish prediction using ANN with comparison to other statistical methods, with general conclusions that neural networks outperformed conventional statistical methods in the domain of bond rating prediction.Table 1
Prior bond rating prediction using AI techniques.
Research
Number of categories
AI methods applied
Data source
Samples size
[20]
2
BP
U.S
30/17
[21]
2
BP
U.S
126
[22]
3
BP
U.S (S&P)
797
[17]
6
BP, RPS
U.S (S&P)
110/60
[23]
6
BP
U.S (S&P)
N/A
[24]
6
BP
U.S (Moody’s)
299
[25]
5
BP with OPP
Korea
126
[26]
6
BP, RBF
U.S (S&P)
60/60
[27]
5
CBR, GA
Korea
3886
[28]
5
SVM
U.S (S&P)
N/A
[29]
5
BP, SVM
Taiwan, U.S
N/AOn the other hand, to overcome the limitations such as overfitting of ANN, techniques based on MSVM are applied in credit rating in recent years. Among the models based on MSVM in credit rating, method of Grammar and Singer was early proposed by Huang et al., with experiments based on different parameters so as to find the optimal model [29]. Moreover, methodologies based on One-Against-All, One-Against-One, and DAGSVM are also proposed to accomplish S&P’s bond ratings prediction, with kernel function of Gaussian RBF applied and the optimal parameters derived form a grid-search strategy [28]. Another automatic-classification model for credit rating prediction based on One-Against-One approach was also applied [30]. And Lee applied MSVM in corporate credit rating prediction [31], with experiments showing that model based on MSVM outperformed other AI techniques such as ANN, MDA, and CBR.


### 2.2. Multiclassification by Support Vector Domain Description

Support Vector Domain Description (SVDD), proposed by Tax and Duin in 1999 [32] and extended in 2004 [33], is a method for classification with the aim to accomplish accurate estimation of a set of data points originally. The methods based on SVDD differ from two or multiclass classification in that a single object type is interested rather than to be separated from other classes. The SVDD is a nonparametric method in the sense that it does not assume any particular form of distribution of the data points. The support of unknown distribution of data points is modeled by a boundary function. And the boundary is “soft” in the sense that atypical points are allowed outside it.The boundary function of SVDD is modeled by a hypersphere rather than a hyperplane applied in standard SVM, which can be made with less constrains by mapping the data points to a high-dimensional space using methodology known as kernel trick, where the classification is performed.SVDD has been applied in a wide range as a basis for new methodologies in statistical and machine learning, whose application in anomaly detection showed that the model based on it can improve accuracy and reduce computational complexity [34]. Moreover, ideas of improving the original SVDD through weighting each data point by an estimate of its corresponding density were also proposed [35] and applied in area of breast cancer, leukemia, and hepatitis, and so forth. Other applications including pump failure detection [36], face recognition [37], speaker recognition [38], and image retrieval [39] are argued by researchers.The capability of SVDD in modeling makes it one of the alternative to large-margin classifiers such as SVM. And some novel methods applied in multiclass classification were proposed based on SVDD [40] combined with other algorithms such as fuzzy theories [41, 42] and Bayesian decision [36].


## 2.1. Credit Rating Using Data Mining Techniques

Major researches applying data mining techniques for bond rating prediction can be found in the literature.Early investigations of credit rating techniques mainly focused on the applicability of statistical techniques including multiple discriminant analysis (MDA) [14, 15] and logistic regression analysis (LRA) [16], and so forth, while typical techniques of AI including ANN [17, 18] and case-based reasoning (CBR) [19], and so forth are applied in the second phase of research.The important researches applying AI techniques in bond-rating prediction are listed in Table1. In summary, the most prior ones accomplish prediction using ANN with comparison to other statistical methods, with general conclusions that neural networks outperformed conventional statistical methods in the domain of bond rating prediction.Table 1
Prior bond rating prediction using AI techniques.
Research
Number of categories
AI methods applied
Data source
Samples size
[20]
2
BP
U.S
30/17
[21]
2
BP
U.S
126
[22]
3
BP
U.S (S&P)
797
[17]
6
BP, RPS
U.S (S&P)
110/60
[23]
6
BP
U.S (S&P)
N/A
[24]
6
BP
U.S (Moody’s)
299
[25]
5
BP with OPP
Korea
126
[26]
6
BP, RBF
U.S (S&P)
60/60
[27]
5
CBR, GA
Korea
3886
[28]
5
SVM
U.S (S&P)
N/A
[29]
5
BP, SVM
Taiwan, U.S
N/AOn the other hand, to overcome the limitations such as overfitting of ANN, techniques based on MSVM are applied in credit rating in recent years. Among the models based on MSVM in credit rating, method of Grammar and Singer was early proposed by Huang et al., with experiments based on different parameters so as to find the optimal model [29]. Moreover, methodologies based on One-Against-All, One-Against-One, and DAGSVM are also proposed to accomplish S&P’s bond ratings prediction, with kernel function of Gaussian RBF applied and the optimal parameters derived form a grid-search strategy [28]. Another automatic-classification model for credit rating prediction based on One-Against-One approach was also applied [30]. And Lee applied MSVM in corporate credit rating prediction [31], with experiments showing that model based on MSVM outperformed other AI techniques such as ANN, MDA, and CBR.


## 2.2. Multiclassification by Support Vector Domain Description

Support Vector Domain Description (SVDD), proposed by Tax and Duin in 1999 [32] and extended in 2004 [33], is a method for classification with the aim to accomplish accurate estimation of a set of data points originally. The methods based on SVDD differ from two or multiclass classification in that a single object type is interested rather than to be separated from other classes. The SVDD is a nonparametric method in the sense that it does not assume any particular form of distribution of the data points. The support of unknown distribution of data points is modeled by a boundary function. And the boundary is “soft” in the sense that atypical points are allowed outside it.The boundary function of SVDD is modeled by a hypersphere rather than a hyperplane applied in standard SVM, which can be made with less constrains by mapping the data points to a high-dimensional space using methodology known as kernel trick, where the classification is performed.SVDD has been applied in a wide range as a basis for new methodologies in statistical and machine learning, whose application in anomaly detection showed that the model based on it can improve accuracy and reduce computational complexity [34]. Moreover, ideas of improving the original SVDD through weighting each data point by an estimate of its corresponding density were also proposed [35] and applied in area of breast cancer, leukemia, and hepatitis, and so forth. Other applications including pump failure detection [36], face recognition [37], speaker recognition [38], and image retrieval [39] are argued by researchers.The capability of SVDD in modeling makes it one of the alternative to large-margin classifiers such as SVM. And some novel methods applied in multiclass classification were proposed based on SVDD [40] combined with other algorithms such as fuzzy theories [41, 42] and Bayesian decision [36].


## 3. The Proposed Methodology

In terms of SVDD, which is a boundary-based method for data description, it needs more boundary samples to construct a closely fit boundary. Unfortunately, more boundary ones usually imply that more target objects have to be rejected with the overfitting problem arising and computational consumption increased. To accomplish multiclassification in corporate credit rating, a method using Fuzzy SVDD combined with fuzzy clustering algorithm is proposed in the paper. By mapping data points to a high-dimensional space by Kernel Trick, the hypersphere applied to every category is specified by training samples selected as boundary ones, which are more likely to be candidates of support vectors. After preprocessing using fuzzy clustering algorithm, rather than by original ones directly in standard SVDD [32, 33], one can improve accuracy and reduce computational consumption. Thus, testing samples are classified by the classification rules based on hyperspheres specified for every class. And the thoughts and framework of the proposed methodology can be illustrated in Figures 1 and 2, respectively.Figure 1
Multiclassification Based on SVDD.Figure 2
Framework of the Proposed Methodology.


### 3.1. Fuzzy SVDD

#### 3.1.1. Introduction to Hypersphere Specification Algorithm

The hypersphere, by which SVDD models data points, is specified by its centera and radius R. Let X=(x1,x2,x3,…) denote the data matrix with n data points and p variables, which implies that a is p-dimensional while R is scalar. The geometry of one solution to SVDD in two dimensions is illustrated in Figure 3, where ωi represents the perpendicular distance from the boundary to an exterior points xi. In terms of interior points, and the ones positioned on the boundary, ωi is to be assigned as 0. Hence, ωi can be calculated using the following equation:
(3.1)ωi=max⁡{0,‖xi-a‖-R}.Figure 3
Geometry of the SVDD in two dimensions.In the following, another closely related measure can be obtianed in (3.2) in terms of exterior points
(3.2)ξi=‖xi-a‖2-R2⟹‖xi-a‖2=R2+ξi.To obtain an exact and compact representation of the data points, the minimization of both the hypersphere radius andξi to any exterior point is required. Moreover, inspired by fuzzy set theory, matrix X can be extended to X=((x1,s1),(x2,s2),(x3,s3),…) with coefficients si  representing fuzzy membership associated with xi introduced. So, the data domain description can be formulated as (3.3), where nonnegative slack variables ξi are a measure of error in SVDD, and the term siξi is the one with different weights based on fuzzy set theory
(3.3)min⁡a,R,ζR2+C∑i=1lsiξi,s.t‖xi-a‖2≤R2+ξiξi≥0,i=1,…,l.To solve the problem, the Lagrange Function is introduced, whereαi,βi≥0 are Lagrange Multipliers shown as follows:
(3.4)L(R,a,ξ,α,β)=R2+C∑i=1lsiξi-∑i=1lαi(R2+ξi-‖xi-a‖2)-∑i=1lβiξi.Setting (3.4) to 0, the partial derivates of L leads to the following equations:
(3.5)∂L∂R=2R-2R∑i=1lαi=0,∂L∂a=∑i=1lαi(xi-a)=0,∂L∂ξi=siC-αi-βi=0.
That is,
(3.6)∑i=1lαi=1,a=∑i=1lαixi,βi=siC-αi.The Karush-Kuhn-Tucker complementarities conditions result in the following equations:(3.7)αi(R2+ξi-‖xi-a‖2)=0,βiξi=0.Therefore, the dual form of the objective function can be obtained as follows:(3.8)LD(α,β)=∑i=1lαixi⋅xi-∑i=1l∑i=1lαiαjxi⋅xj.And the problem can be formulated as follows:(3.9)max⁡⁡∑i=1lαixi⋅xi-∑i=1l∑i=1lαiαjxi⋅xjs.t0≤αi≤siC,i=1,2,…,l,∑i=1lαi=1.The center of the hypersphere is a linear combination of data points with weighting factorsαi obtained by optimizing (3.9). And the coefficients αi, which are nonzero, are thus selected as support vectors, only by which the hypersphere is specified and described. Hence, to judge whether a data point is within a hypersphere, the distance to the center should be calculated with (3.10) in order to judge whether it is smaller than the radius R. And the decision function shown as (3.12) can be concluded from
(3.10)‖x-∑i=1lαixi‖2≤R2,(3.11)R2=‖xi0-∑i=1lαixi‖=xi0⋅xi0-2∑i=1lαi(xi0⋅xi)+∑i=1l∑i=1lαiαjxi⋅xj,(3.12)x⋅x-2∑i=1lαi(x⋅xi)≤xi0⋅xi0-2∑i=1lαi(xi0⋅xi).


#### 3.1.2. Introduction to Fuzzy SVDD Based on Kernel Trick

Similarly to the methodology based on kernel function proposed by Vapnik [12], the Fuzzy SVDD can also be generalized to high-dimensional space by replacing its inner products by kernel functions K(·,·)=Φ(·)•Φ(·).For example, Kernel function of RBF can be introduced to SVDD algorithm, just as shown as follows:(3.13)max⁡⁡1-∑i=1lαi2-∑i=1l∑i=1lαiαjK(xi⋅xj)s.t0≤αi≤siC,i=1,2,…,l,∑i=1lαi=1.And it can be determined whether a testing data pointx is within the hypersphere with (3.14) by introducing kernel function based on (3.12)
(3.14)∑i=1lαiK(x,xi)≥∑i=1lαiK(xi0,xi).


### 3.2. Kernel-Based Fuzzy Clustering Algorithm

#### 3.2.1. Introduction to Fuzzy AttributeC-Means Clustering

Based on fuzzy clustering algorithm [42], Fuzzy Attribute C-means Clustering (FAMC) [43] was proposed as extension of Attribute Means Clustering (AMC) and Fuzzy C-means (FCM).Supposeχ⊂Rd denote any finite sample set, where χ={x1,x2,…,xn}, and each sample is defined as xn=(x1n,x2n,…,xdn)(1≤n≤N). The category of attribute space is F={C1,C2,…,Cc}, where c is the cluster number. For ∀x∈χ, let μx(Ck)  denote the attribute measure of x, with ∑k=1cμx(Ck)=1.Letpk=(pk1,pk2,…,pkd) denote the kth prototype of cluster Ck, where 1≤k≤c.Letμkn denote the attribute measure of the nth sample belonging to the kth cluster. That is, μkn=μn(pk),  U=(μkn), p=(p1,p2,…,pk). The task of fuzzy clustering is to calculate the attribute measure μkn, and determine the cluster which xn belongs to according to the maximum cluster index.FuzzyC-means (FCM) is an inner-product-induced distance based on the least-squared error criterion. A brief review of FCM can be found in Appendix based on coefficients definitions mentioned above.Attribute Means Clustering (AMC) is an iterative algorithm by introducing the stable function [44]. Suppose ρ(t) is a positive differential function in [0,∞). Let ω(t)=ρ′(t)/2t, if ω(t), called as weight function, is a positive nonincreasing function, ρ(t) is called as stable function. And ρ(t) can be adopted as follows:
(3.15)ρ(t)=∫0t2sω(s)ds.Hence, the relationship of objective functionρ(t) and its weight function is described by sable function, which was introduced to propose AMC.According to current researches, some alternative functions including squared stable function, Cauchy stable function, and Exponential stable function are recommended.Based on previous researches, AMC and FCM are extended to FAMC, which is also an iterative algorithm to minimize the following objective function shown as (3.16), where m>1, which is a coefficient of FCM introduced in Appendix
(3.16)P(U,p)=∑k=1c∑n=1Nρ(μknm/2‖xn-pk‖).Moreover, procedure of minimizing (3.16) can be converted to an iterative objective function shown as (3.17) [43]
(3.17)Q(i)(U,p)=∑k=1c∑n=1Nω((μkni)m/2‖xn-pki‖)((μkn)m‖xn-pk‖2).And the following equations can be obtained by minimizingQ(i)(Ui,p), Q(i)(U,p(i+1)), respectively, which can be seen in [43, 45] in detail
(3.18)pk(i+1)=∑n=1Nω((μkn(i))m/2‖xn-pk(i)‖)(μkn(i))mxn∑n=1Nω((μkn(i))m/2‖xn-pk(i)‖)(μkn(i))m,μkn(i+1)=ω((μkni)m/2‖xn-pk(i)‖)(‖xn-pk(i+1)‖2)-1/(m-1)∑k=1cω((μkni)m/2‖xn-pk(i)‖)(‖xn-pk(i+1)‖2)-1/(m-1).


#### 3.2.2. Introduction to Kernel-Based Fuzzy Clustering

To gain a high-dimensional discriminant, FAMC can be extended to Kernel-based Fuzzy AttributeC-means Clustering (KFAMC). That is, the training samples can be first mapped into high-dimensional space by the mapping Φ using kernel function methods addressed in Section 3.1.2.Since(3.19)‖Φ(xn)-Φ(pk)‖=(Φ(xn)-Φ(pk))T(Φ(xn)-Φ(pk))=Φ(xn)TΦ(xn)-Φ(xn)TΦ(pk)-Φ(pk)TΦ(xn)+Φ(pk)TΦ(pk)=K(xn,xn)+K(pk,pk)-2K(xn,pk)
when Kernel function of RBF is introduced, (3.19) can be given as follows
(3.20)‖Φ(xn)-Φ(pk)‖2=2(1-K(xn,pk)).And parameters in KFAMC can be estimated by(3.21)μkn=(1-K(xn,pk))-1/(m-1)∑k=1c(1-K(xn,pk))-1/(m-1),pk=∑n=1NμknmK(xn,pk)xn∑n=1NμknmK(xn,pk),
where n=1,2,…,N,k=1,2,…,c.Moreover, the objective function of KFAMC can be obtained by substituting (3.16), (3.17) with (3.22), (3.23), respectively,(3.22)P(U,p)=∑k=1c∑n=1Nρ(‖μknm/2(Φ(xn)-Φ(pk))‖),(3.23)Q(i)(U,p)=∑k=1c∑n=1Nω((μkni)m/2(1-K(xn,pk(i))1/2))((μkn)m(1-K(xn,pk))).


#### 3.2.3. Algorithms of Kernel-Based Fuzzy AttributeC-Means Clustering

Based on theorem proved in [45], the updating procedure of KFAMC can be summarized in the following iterative scheme.Step 1.
Setc, m, ε and tmax, and initialize U(0), W(0).Step 2.
Fori=1, calculate fuzzy cluster centers P(i), U(i). and W(i).Step 3.
If|Q(i)(U,P)-Q(i+1)(U,P)|<ε or i>tmax, stop, else go to Step 4.Step 4.
For stepi=i+1, update P(i+1), U(i+1), and W(i), turn to Step 3,
wherei denotes iterate step, tmax represents the maximum iteration times, and W(i) denotes the weighting matrix, respectively, which can be seen in [45] in detail.


### 3.3. The Proposed Algorithm

#### 3.3.1. Classifier Establishment

In terms of SVDD, only support vectors are necessary to specify hyperspheres. But in the original algorithms [32, 33, 41], all the training samples are analyzed and thus computational cost is high consumption. Hence, if the data points, which are more likely to be candidates of support vectors, can be selected as training samples, the hypersphere will be specified with much less computational consumption.Just as illustrated in Figure4, only the data points, such as M, N positioned in fuzzy areas, which are more likely to be candidates of support vectors, are necessary to be classified with SVDD, while the ones in deterministic areas can be regarded as data points belonging to certain class.Figure 4
Thoughts of proposed methodology.So, the new methodology applied in SVDD is proposed as follows.(1)
Preprocess data points using FAMC to reduce amount of training samples. That is, if fuzzy membership of a data point to a class is great enough, the data point can be ranked to the class directly. Just as shown in Figure5, the data points positioned in deterministic area (shadow area A) are to be regarded as samples belonging to the class, while the other ones are selected as training samples.(2)
Accomplish SVDD specification with training samples positioned in fuzzy areas, which has been selected using KFAMC. That is, among the whole data points, only the ones in fuzzy area, rather than all the data points, are treated as candidates of support vectors. And the classifier applied in multiclassification can be developed based on Fuzzy SVDD by specifying hypersphere according to every class. Hence, the main thoughts of Fuzzy SVDD establishment combined with KFAMC can be illustrated in Figure 6.Figure 5
Data points selection using FAMC.Fuzzy SVDD establishment.
(a)
Training data points obtained by preprocessing
(b)
Hypersphere specification after data points preprocessingThe process of methods proposed in the paper can be depicted as follows.In high-dimensional space, the training samples are selected according to their fuzzy memberships to clustering centers. Based on preprocessing with KFAMC, a set of training samples is given, which is represented byX0m={(x1,μ1m),(x2,μ2m),…,(xl,μlm)}, where l∈N,xi∈Rn, and μlm∈[0,1] denote the number of training data, input pattern, and membership to class m, respectively.Hence, the process of Fuzzy SVDD specification can be summarized as follows.Step 1.
Set a thresholdθ>0, and apply KFAMC to calculate the membership of each xi,i=1,2,…,l, to each class. If  μim≥θ, μim  is to be set as 1 and  μit,t≠m, is to be set as 0.Step 2.
Survey the membership of eachxi,i=1,2,…,l. If μim=1, xi is to be ranked to class m directly and removed from the training set. And an updated training set can be obtained.Step 3.
With hypersphere specified for each class using the updated training set obtained in Step2, classifier for credit rating can be established using the algorithm of Fuzzy SVDD, just as illustrated in Figure 6.


#### 3.3.2. Classification Rules for Testing Data Points

To accomplish multiclassification for testing data points using hyperspheres specified in Section3.3.1, the following two factors should be taken into consideration, just as illustrated in Figure 7:(1)
distances from the data point to centers of the hyperspheres;(2)
density of the data points belonging to the class implied with values of radius of each hypersphere.Figure 7
Classification of testing data point.Just as shown in Figure7, D(x,A), D(x,B) denote the distances from data point x to center of class A and class B, respectively. Even if D(x,A)=D(x,B), data point x is expected more likely to belong to class A rather than class B because of difference in distributions of data points. That is, data points circled by hypersphere of class A are sparser than the ones circled by hypersphere of class B since Ra is greater than Rb.So, classification rules can be concluded as follows.Letd denote the numbers of hyperspheres containing the data point.Case I (d=1).
Data point belongs to the class represented by the hypersphere.Case II (d=0 or d>1).
Calculate the index of membership of the data point to each hypersphere using (3.24), where Rc   denotes the radius of hypersphere c,D(xi,c) denotes the distance from data point xi to the center of hypersphere c(3.24)φ(xi,c)={λ(1-(D(xi,c)/Rc)1+(D(xi,c)/Rc))+γ,0≤D(xi,c)≤Rc,γ(RcD(xi,c)),D(xi,c)>Rc,λ,γ∈R+,λ+γ=1.And the testing data points can be classified according to the following rules represented with(3.25)F(xi)=argmaxcφ(xi,c).


## 3.1. Fuzzy SVDD

### 3.1.1. Introduction to Hypersphere Specification Algorithm

The hypersphere, by which SVDD models data points, is specified by its centera and radius R. Let X=(x1,x2,x3,…) denote the data matrix with n data points and p variables, which implies that a is p-dimensional while R is scalar. The geometry of one solution to SVDD in two dimensions is illustrated in Figure 3, where ωi represents the perpendicular distance from the boundary to an exterior points xi. In terms of interior points, and the ones positioned on the boundary, ωi is to be assigned as 0. Hence, ωi can be calculated using the following equation:
(3.1)ωi=max⁡{0,‖xi-a‖-R}.Figure 3
Geometry of the SVDD in two dimensions.In the following, another closely related measure can be obtianed in (3.2) in terms of exterior points
(3.2)ξi=‖xi-a‖2-R2⟹‖xi-a‖2=R2+ξi.To obtain an exact and compact representation of the data points, the minimization of both the hypersphere radius andξi to any exterior point is required. Moreover, inspired by fuzzy set theory, matrix X can be extended to X=((x1,s1),(x2,s2),(x3,s3),…) with coefficients si  representing fuzzy membership associated with xi introduced. So, the data domain description can be formulated as (3.3), where nonnegative slack variables ξi are a measure of error in SVDD, and the term siξi is the one with different weights based on fuzzy set theory
(3.3)min⁡a,R,ζR2+C∑i=1lsiξi,s.t‖xi-a‖2≤R2+ξiξi≥0,i=1,…,l.To solve the problem, the Lagrange Function is introduced, whereαi,βi≥0 are Lagrange Multipliers shown as follows:
(3.4)L(R,a,ξ,α,β)=R2+C∑i=1lsiξi-∑i=1lαi(R2+ξi-‖xi-a‖2)-∑i=1lβiξi.Setting (3.4) to 0, the partial derivates of L leads to the following equations:
(3.5)∂L∂R=2R-2R∑i=1lαi=0,∂L∂a=∑i=1lαi(xi-a)=0,∂L∂ξi=siC-αi-βi=0.
That is,
(3.6)∑i=1lαi=1,a=∑i=1lαixi,βi=siC-αi.The Karush-Kuhn-Tucker complementarities conditions result in the following equations:(3.7)αi(R2+ξi-‖xi-a‖2)=0,βiξi=0.Therefore, the dual form of the objective function can be obtained as follows:(3.8)LD(α,β)=∑i=1lαixi⋅xi-∑i=1l∑i=1lαiαjxi⋅xj.And the problem can be formulated as follows:(3.9)max⁡⁡∑i=1lαixi⋅xi-∑i=1l∑i=1lαiαjxi⋅xjs.t0≤αi≤siC,i=1,2,…,l,∑i=1lαi=1.The center of the hypersphere is a linear combination of data points with weighting factorsαi obtained by optimizing (3.9). And the coefficients αi, which are nonzero, are thus selected as support vectors, only by which the hypersphere is specified and described. Hence, to judge whether a data point is within a hypersphere, the distance to the center should be calculated with (3.10) in order to judge whether it is smaller than the radius R. And the decision function shown as (3.12) can be concluded from
(3.10)‖x-∑i=1lαixi‖2≤R2,(3.11)R2=‖xi0-∑i=1lαixi‖=xi0⋅xi0-2∑i=1lαi(xi0⋅xi)+∑i=1l∑i=1lαiαjxi⋅xj,(3.12)x⋅x-2∑i=1lαi(x⋅xi)≤xi0⋅xi0-2∑i=1lαi(xi0⋅xi).


### 3.1.2. Introduction to Fuzzy SVDD Based on Kernel Trick

Similarly to the methodology based on kernel function proposed by Vapnik [12], the Fuzzy SVDD can also be generalized to high-dimensional space by replacing its inner products by kernel functions K(·,·)=Φ(·)•Φ(·).For example, Kernel function of RBF can be introduced to SVDD algorithm, just as shown as follows:(3.13)max⁡⁡1-∑i=1lαi2-∑i=1l∑i=1lαiαjK(xi⋅xj)s.t0≤αi≤siC,i=1,2,…,l,∑i=1lαi=1.And it can be determined whether a testing data pointx is within the hypersphere with (3.14) by introducing kernel function based on (3.12)
(3.14)∑i=1lαiK(x,xi)≥∑i=1lαiK(xi0,xi).


## 3.1.1. Introduction to Hypersphere Specification Algorithm

The hypersphere, by which SVDD models data points, is specified by its centera and radius R. Let X=(x1,x2,x3,…) denote the data matrix with n data points and p variables, which implies that a is p-dimensional while R is scalar. The geometry of one solution to SVDD in two dimensions is illustrated in Figure 3, where ωi represents the perpendicular distance from the boundary to an exterior points xi. In terms of interior points, and the ones positioned on the boundary, ωi is to be assigned as 0. Hence, ωi can be calculated using the following equation:
(3.1)ωi=max⁡{0,‖xi-a‖-R}.Figure 3
Geometry of the SVDD in two dimensions.In the following, another closely related measure can be obtianed in (3.2) in terms of exterior points
(3.2)ξi=‖xi-a‖2-R2⟹‖xi-a‖2=R2+ξi.To obtain an exact and compact representation of the data points, the minimization of both the hypersphere radius andξi to any exterior point is required. Moreover, inspired by fuzzy set theory, matrix X can be extended to X=((x1,s1),(x2,s2),(x3,s3),…) with coefficients si  representing fuzzy membership associated with xi introduced. So, the data domain description can be formulated as (3.3), where nonnegative slack variables ξi are a measure of error in SVDD, and the term siξi is the one with different weights based on fuzzy set theory
(3.3)min⁡a,R,ζR2+C∑i=1lsiξi,s.t‖xi-a‖2≤R2+ξiξi≥0,i=1,…,l.To solve the problem, the Lagrange Function is introduced, whereαi,βi≥0 are Lagrange Multipliers shown as follows:
(3.4)L(R,a,ξ,α,β)=R2+C∑i=1lsiξi-∑i=1lαi(R2+ξi-‖xi-a‖2)-∑i=1lβiξi.Setting (3.4) to 0, the partial derivates of L leads to the following equations:
(3.5)∂L∂R=2R-2R∑i=1lαi=0,∂L∂a=∑i=1lαi(xi-a)=0,∂L∂ξi=siC-αi-βi=0.
That is,
(3.6)∑i=1lαi=1,a=∑i=1lαixi,βi=siC-αi.The Karush-Kuhn-Tucker complementarities conditions result in the following equations:(3.7)αi(R2+ξi-‖xi-a‖2)=0,βiξi=0.Therefore, the dual form of the objective function can be obtained as follows:(3.8)LD(α,β)=∑i=1lαixi⋅xi-∑i=1l∑i=1lαiαjxi⋅xj.And the problem can be formulated as follows:(3.9)max⁡⁡∑i=1lαixi⋅xi-∑i=1l∑i=1lαiαjxi⋅xjs.t0≤αi≤siC,i=1,2,…,l,∑i=1lαi=1.The center of the hypersphere is a linear combination of data points with weighting factorsαi obtained by optimizing (3.9). And the coefficients αi, which are nonzero, are thus selected as support vectors, only by which the hypersphere is specified and described. Hence, to judge whether a data point is within a hypersphere, the distance to the center should be calculated with (3.10) in order to judge whether it is smaller than the radius R. And the decision function shown as (3.12) can be concluded from
(3.10)‖x-∑i=1lαixi‖2≤R2,(3.11)R2=‖xi0-∑i=1lαixi‖=xi0⋅xi0-2∑i=1lαi(xi0⋅xi)+∑i=1l∑i=1lαiαjxi⋅xj,(3.12)x⋅x-2∑i=1lαi(x⋅xi)≤xi0⋅xi0-2∑i=1lαi(xi0⋅xi).


## 3.1.2. Introduction to Fuzzy SVDD Based on Kernel Trick

Similarly to the methodology based on kernel function proposed by Vapnik [12], the Fuzzy SVDD can also be generalized to high-dimensional space by replacing its inner products by kernel functions K(·,·)=Φ(·)•Φ(·).For example, Kernel function of RBF can be introduced to SVDD algorithm, just as shown as follows:(3.13)max⁡⁡1-∑i=1lαi2-∑i=1l∑i=1lαiαjK(xi⋅xj)s.t0≤αi≤siC,i=1,2,…,l,∑i=1lαi=1.And it can be determined whether a testing data pointx is within the hypersphere with (3.14) by introducing kernel function based on (3.12)
(3.14)∑i=1lαiK(x,xi)≥∑i=1lαiK(xi0,xi).


## 3.2. Kernel-Based Fuzzy Clustering Algorithm

### 3.2.1. Introduction to Fuzzy AttributeC-Means Clustering

Based on fuzzy clustering algorithm [42], Fuzzy Attribute C-means Clustering (FAMC) [43] was proposed as extension of Attribute Means Clustering (AMC) and Fuzzy C-means (FCM).Supposeχ⊂Rd denote any finite sample set, where χ={x1,x2,…,xn}, and each sample is defined as xn=(x1n,x2n,…,xdn)(1≤n≤N). The category of attribute space is F={C1,C2,…,Cc}, where c is the cluster number. For ∀x∈χ, let μx(Ck)  denote the attribute measure of x, with ∑k=1cμx(Ck)=1.Letpk=(pk1,pk2,…,pkd) denote the kth prototype of cluster Ck, where 1≤k≤c.Letμkn denote the attribute measure of the nth sample belonging to the kth cluster. That is, μkn=μn(pk),  U=(μkn), p=(p1,p2,…,pk). The task of fuzzy clustering is to calculate the attribute measure μkn, and determine the cluster which xn belongs to according to the maximum cluster index.FuzzyC-means (FCM) is an inner-product-induced distance based on the least-squared error criterion. A brief review of FCM can be found in Appendix based on coefficients definitions mentioned above.Attribute Means Clustering (AMC) is an iterative algorithm by introducing the stable function [44]. Suppose ρ(t) is a positive differential function in [0,∞). Let ω(t)=ρ′(t)/2t, if ω(t), called as weight function, is a positive nonincreasing function, ρ(t) is called as stable function. And ρ(t) can be adopted as follows:
(3.15)ρ(t)=∫0t2sω(s)ds.Hence, the relationship of objective functionρ(t) and its weight function is described by sable function, which was introduced to propose AMC.According to current researches, some alternative functions including squared stable function, Cauchy stable function, and Exponential stable function are recommended.Based on previous researches, AMC and FCM are extended to FAMC, which is also an iterative algorithm to minimize the following objective function shown as (3.16), where m>1, which is a coefficient of FCM introduced in Appendix
(3.16)P(U,p)=∑k=1c∑n=1Nρ(μknm/2‖xn-pk‖).Moreover, procedure of minimizing (3.16) can be converted to an iterative objective function shown as (3.17) [43]
(3.17)Q(i)(U,p)=∑k=1c∑n=1Nω((μkni)m/2‖xn-pki‖)((μkn)m‖xn-pk‖2).And the following equations can be obtained by minimizingQ(i)(Ui,p), Q(i)(U,p(i+1)), respectively, which can be seen in [43, 45] in detail
(3.18)pk(i+1)=∑n=1Nω((μkn(i))m/2‖xn-pk(i)‖)(μkn(i))mxn∑n=1Nω((μkn(i))m/2‖xn-pk(i)‖)(μkn(i))m,μkn(i+1)=ω((μkni)m/2‖xn-pk(i)‖)(‖xn-pk(i+1)‖2)-1/(m-1)∑k=1cω((μkni)m/2‖xn-pk(i)‖)(‖xn-pk(i+1)‖2)-1/(m-1).


### 3.2.2. Introduction to Kernel-Based Fuzzy Clustering

To gain a high-dimensional discriminant, FAMC can be extended to Kernel-based Fuzzy AttributeC-means Clustering (KFAMC). That is, the training samples can be first mapped into high-dimensional space by the mapping Φ using kernel function methods addressed in Section 3.1.2.Since(3.19)‖Φ(xn)-Φ(pk)‖=(Φ(xn)-Φ(pk))T(Φ(xn)-Φ(pk))=Φ(xn)TΦ(xn)-Φ(xn)TΦ(pk)-Φ(pk)TΦ(xn)+Φ(pk)TΦ(pk)=K(xn,xn)+K(pk,pk)-2K(xn,pk)
when Kernel function of RBF is introduced, (3.19) can be given as follows
(3.20)‖Φ(xn)-Φ(pk)‖2=2(1-K(xn,pk)).And parameters in KFAMC can be estimated by(3.21)μkn=(1-K(xn,pk))-1/(m-1)∑k=1c(1-K(xn,pk))-1/(m-1),pk=∑n=1NμknmK(xn,pk)xn∑n=1NμknmK(xn,pk),
where n=1,2,…,N,k=1,2,…,c.Moreover, the objective function of KFAMC can be obtained by substituting (3.16), (3.17) with (3.22), (3.23), respectively,(3.22)P(U,p)=∑k=1c∑n=1Nρ(‖μknm/2(Φ(xn)-Φ(pk))‖),(3.23)Q(i)(U,p)=∑k=1c∑n=1Nω((μkni)m/2(1-K(xn,pk(i))1/2))((μkn)m(1-K(xn,pk))).


### 3.2.3. Algorithms of Kernel-Based Fuzzy AttributeC-Means Clustering

Based on theorem proved in [45], the updating procedure of KFAMC can be summarized in the following iterative scheme.Step 1.
Setc, m, ε and tmax, and initialize U(0), W(0).Step 2.
Fori=1, calculate fuzzy cluster centers P(i), U(i). and W(i).Step 3.
If|Q(i)(U,P)-Q(i+1)(U,P)|<ε or i>tmax, stop, else go to Step 4.Step 4.
For stepi=i+1, update P(i+1), U(i+1), and W(i), turn to Step 3,
wherei denotes iterate step, tmax represents the maximum iteration times, and W(i) denotes the weighting matrix, respectively, which can be seen in [45] in detail.


## 3.2.1. Introduction to Fuzzy AttributeC-Means Clustering

Based on fuzzy clustering algorithm [42], Fuzzy Attribute C-means Clustering (FAMC) [43] was proposed as extension of Attribute Means Clustering (AMC) and Fuzzy C-means (FCM).Supposeχ⊂Rd denote any finite sample set, where χ={x1,x2,…,xn}, and each sample is defined as xn=(x1n,x2n,…,xdn)(1≤n≤N). The category of attribute space is F={C1,C2,…,Cc}, where c is the cluster number. For ∀x∈χ, let μx(Ck)  denote the attribute measure of x, with ∑k=1cμx(Ck)=1.Letpk=(pk1,pk2,…,pkd) denote the kth prototype of cluster Ck, where 1≤k≤c.Letμkn denote the attribute measure of the nth sample belonging to the kth cluster. That is, μkn=μn(pk),  U=(μkn), p=(p1,p2,…,pk). The task of fuzzy clustering is to calculate the attribute measure μkn, and determine the cluster which xn belongs to according to the maximum cluster index.FuzzyC-means (FCM) is an inner-product-induced distance based on the least-squared error criterion. A brief review of FCM can be found in Appendix based on coefficients definitions mentioned above.Attribute Means Clustering (AMC) is an iterative algorithm by introducing the stable function [44]. Suppose ρ(t) is a positive differential function in [0,∞). Let ω(t)=ρ′(t)/2t, if ω(t), called as weight function, is a positive nonincreasing function, ρ(t) is called as stable function. And ρ(t) can be adopted as follows:
(3.15)ρ(t)=∫0t2sω(s)ds.Hence, the relationship of objective functionρ(t) and its weight function is described by sable function, which was introduced to propose AMC.According to current researches, some alternative functions including squared stable function, Cauchy stable function, and Exponential stable function are recommended.Based on previous researches, AMC and FCM are extended to FAMC, which is also an iterative algorithm to minimize the following objective function shown as (3.16), where m>1, which is a coefficient of FCM introduced in Appendix
(3.16)P(U,p)=∑k=1c∑n=1Nρ(μknm/2‖xn-pk‖).Moreover, procedure of minimizing (3.16) can be converted to an iterative objective function shown as (3.17) [43]
(3.17)Q(i)(U,p)=∑k=1c∑n=1Nω((μkni)m/2‖xn-pki‖)((μkn)m‖xn-pk‖2).And the following equations can be obtained by minimizingQ(i)(Ui,p), Q(i)(U,p(i+1)), respectively, which can be seen in [43, 45] in detail
(3.18)pk(i+1)=∑n=1Nω((μkn(i))m/2‖xn-pk(i)‖)(μkn(i))mxn∑n=1Nω((μkn(i))m/2‖xn-pk(i)‖)(μkn(i))m,μkn(i+1)=ω((μkni)m/2‖xn-pk(i)‖)(‖xn-pk(i+1)‖2)-1/(m-1)∑k=1cω((μkni)m/2‖xn-pk(i)‖)(‖xn-pk(i+1)‖2)-1/(m-1).


## 3.2.2. Introduction to Kernel-Based Fuzzy Clustering

To gain a high-dimensional discriminant, FAMC can be extended to Kernel-based Fuzzy AttributeC-means Clustering (KFAMC). That is, the training samples can be first mapped into high-dimensional space by the mapping Φ using kernel function methods addressed in Section 3.1.2.Since(3.19)‖Φ(xn)-Φ(pk)‖=(Φ(xn)-Φ(pk))T(Φ(xn)-Φ(pk))=Φ(xn)TΦ(xn)-Φ(xn)TΦ(pk)-Φ(pk)TΦ(xn)+Φ(pk)TΦ(pk)=K(xn,xn)+K(pk,pk)-2K(xn,pk)
when Kernel function of RBF is introduced, (3.19) can be given as follows
(3.20)‖Φ(xn)-Φ(pk)‖2=2(1-K(xn,pk)).And parameters in KFAMC can be estimated by(3.21)μkn=(1-K(xn,pk))-1/(m-1)∑k=1c(1-K(xn,pk))-1/(m-1),pk=∑n=1NμknmK(xn,pk)xn∑n=1NμknmK(xn,pk),
where n=1,2,…,N,k=1,2,…,c.Moreover, the objective function of KFAMC can be obtained by substituting (3.16), (3.17) with (3.22), (3.23), respectively,(3.22)P(U,p)=∑k=1c∑n=1Nρ(‖μknm/2(Φ(xn)-Φ(pk))‖),(3.23)Q(i)(U,p)=∑k=1c∑n=1Nω((μkni)m/2(1-K(xn,pk(i))1/2))((μkn)m(1-K(xn,pk))).


## 3.2.3. Algorithms of Kernel-Based Fuzzy AttributeC-Means Clustering

Based on theorem proved in [45], the updating procedure of KFAMC can be summarized in the following iterative scheme.Step 1.
Setc, m, ε and tmax, and initialize U(0), W(0).Step 2.
Fori=1, calculate fuzzy cluster centers P(i), U(i). and W(i).Step 3.
If|Q(i)(U,P)-Q(i+1)(U,P)|<ε or i>tmax, stop, else go to Step 4.Step 4.
For stepi=i+1, update P(i+1), U(i+1), and W(i), turn to Step 3,
wherei denotes iterate step, tmax represents the maximum iteration times, and W(i) denotes the weighting matrix, respectively, which can be seen in [45] in detail.


## 3.3. The Proposed Algorithm

### 3.3.1. Classifier Establishment

In terms of SVDD, only support vectors are necessary to specify hyperspheres. But in the original algorithms [32, 33, 41], all the training samples are analyzed and thus computational cost is high consumption. Hence, if the data points, which are more likely to be candidates of support vectors, can be selected as training samples, the hypersphere will be specified with much less computational consumption.Just as illustrated in Figure4, only the data points, such as M, N positioned in fuzzy areas, which are more likely to be candidates of support vectors, are necessary to be classified with SVDD, while the ones in deterministic areas can be regarded as data points belonging to certain class.Figure 4
Thoughts of proposed methodology.So, the new methodology applied in SVDD is proposed as follows.(1)
Preprocess data points using FAMC to reduce amount of training samples. That is, if fuzzy membership of a data point to a class is great enough, the data point can be ranked to the class directly. Just as shown in Figure5, the data points positioned in deterministic area (shadow area A) are to be regarded as samples belonging to the class, while the other ones are selected as training samples.(2)
Accomplish SVDD specification with training samples positioned in fuzzy areas, which has been selected using KFAMC. That is, among the whole data points, only the ones in fuzzy area, rather than all the data points, are treated as candidates of support vectors. And the classifier applied in multiclassification can be developed based on Fuzzy SVDD by specifying hypersphere according to every class. Hence, the main thoughts of Fuzzy SVDD establishment combined with KFAMC can be illustrated in Figure 6.Figure 5
Data points selection using FAMC.Fuzzy SVDD establishment.
(a)
Training data points obtained by preprocessing
(b)
Hypersphere specification after data points preprocessingThe process of methods proposed in the paper can be depicted as follows.In high-dimensional space, the training samples are selected according to their fuzzy memberships to clustering centers. Based on preprocessing with KFAMC, a set of training samples is given, which is represented byX0m={(x1,μ1m),(x2,μ2m),…,(xl,μlm)}, where l∈N,xi∈Rn, and μlm∈[0,1] denote the number of training data, input pattern, and membership to class m, respectively.Hence, the process of Fuzzy SVDD specification can be summarized as follows.Step 1.
Set a thresholdθ>0, and apply KFAMC to calculate the membership of each xi,i=1,2,…,l, to each class. If  μim≥θ, μim  is to be set as 1 and  μit,t≠m, is to be set as 0.Step 2.
Survey the membership of eachxi,i=1,2,…,l. If μim=1, xi is to be ranked to class m directly and removed from the training set. And an updated training set can be obtained.Step 3.
With hypersphere specified for each class using the updated training set obtained in Step2, classifier for credit rating can be established using the algorithm of Fuzzy SVDD, just as illustrated in Figure 6.


### 3.3.2. Classification Rules for Testing Data Points

To accomplish multiclassification for testing data points using hyperspheres specified in Section3.3.1, the following two factors should be taken into consideration, just as illustrated in Figure 7:(1)
distances from the data point to centers of the hyperspheres;(2)
density of the data points belonging to the class implied with values of radius of each hypersphere.Figure 7
Classification of testing data point.Just as shown in Figure7, D(x,A), D(x,B) denote the distances from data point x to center of class A and class B, respectively. Even if D(x,A)=D(x,B), data point x is expected more likely to belong to class A rather than class B because of difference in distributions of data points. That is, data points circled by hypersphere of class A are sparser than the ones circled by hypersphere of class B since Ra is greater than Rb.So, classification rules can be concluded as follows.Letd denote the numbers of hyperspheres containing the data point.Case I (d=1).
Data point belongs to the class represented by the hypersphere.Case II (d=0 or d>1).
Calculate the index of membership of the data point to each hypersphere using (3.24), where Rc   denotes the radius of hypersphere c,D(xi,c) denotes the distance from data point xi to the center of hypersphere c(3.24)φ(xi,c)={λ(1-(D(xi,c)/Rc)1+(D(xi,c)/Rc))+γ,0≤D(xi,c)≤Rc,γ(RcD(xi,c)),D(xi,c)>Rc,λ,γ∈R+,λ+γ=1.And the testing data points can be classified according to the following rules represented with(3.25)F(xi)=argmaxcφ(xi,c).


## 3.3.1. Classifier Establishment

In terms of SVDD, only support vectors are necessary to specify hyperspheres. But in the original algorithms [32, 33, 41], all the training samples are analyzed and thus computational cost is high consumption. Hence, if the data points, which are more likely to be candidates of support vectors, can be selected as training samples, the hypersphere will be specified with much less computational consumption.Just as illustrated in Figure4, only the data points, such as M, N positioned in fuzzy areas, which are more likely to be candidates of support vectors, are necessary to be classified with SVDD, while the ones in deterministic areas can be regarded as data points belonging to certain class.Figure 4
Thoughts of proposed methodology.So, the new methodology applied in SVDD is proposed as follows.(1)
Preprocess data points using FAMC to reduce amount of training samples. That is, if fuzzy membership of a data point to a class is great enough, the data point can be ranked to the class directly. Just as shown in Figure5, the data points positioned in deterministic area (shadow area A) are to be regarded as samples belonging to the class, while the other ones are selected as training samples.(2)
Accomplish SVDD specification with training samples positioned in fuzzy areas, which has been selected using KFAMC. That is, among the whole data points, only the ones in fuzzy area, rather than all the data points, are treated as candidates of support vectors. And the classifier applied in multiclassification can be developed based on Fuzzy SVDD by specifying hypersphere according to every class. Hence, the main thoughts of Fuzzy SVDD establishment combined with KFAMC can be illustrated in Figure 6.Figure 5
Data points selection using FAMC.Fuzzy SVDD establishment.
(a)
Training data points obtained by preprocessing
(b)
Hypersphere specification after data points preprocessingThe process of methods proposed in the paper can be depicted as follows.In high-dimensional space, the training samples are selected according to their fuzzy memberships to clustering centers. Based on preprocessing with KFAMC, a set of training samples is given, which is represented byX0m={(x1,μ1m),(x2,μ2m),…,(xl,μlm)}, where l∈N,xi∈Rn, and μlm∈[0,1] denote the number of training data, input pattern, and membership to class m, respectively.Hence, the process of Fuzzy SVDD specification can be summarized as follows.Step 1.
Set a thresholdθ>0, and apply KFAMC to calculate the membership of each xi,i=1,2,…,l, to each class. If  μim≥θ, μim  is to be set as 1 and  μit,t≠m, is to be set as 0.Step 2.
Survey the membership of eachxi,i=1,2,…,l. If μim=1, xi is to be ranked to class m directly and removed from the training set. And an updated training set can be obtained.Step 3.
With hypersphere specified for each class using the updated training set obtained in Step2, classifier for credit rating can be established using the algorithm of Fuzzy SVDD, just as illustrated in Figure 6.


## 3.3.2. Classification Rules for Testing Data Points

To accomplish multiclassification for testing data points using hyperspheres specified in Section3.3.1, the following two factors should be taken into consideration, just as illustrated in Figure 7:(1)
distances from the data point to centers of the hyperspheres;(2)
density of the data points belonging to the class implied with values of radius of each hypersphere.Figure 7
Classification of testing data point.Just as shown in Figure7, D(x,A), D(x,B) denote the distances from data point x to center of class A and class B, respectively. Even if D(x,A)=D(x,B), data point x is expected more likely to belong to class A rather than class B because of difference in distributions of data points. That is, data points circled by hypersphere of class A are sparser than the ones circled by hypersphere of class B since Ra is greater than Rb.So, classification rules can be concluded as follows.Letd denote the numbers of hyperspheres containing the data point.Case I (d=1).
Data point belongs to the class represented by the hypersphere.Case II (d=0 or d>1).
Calculate the index of membership of the data point to each hypersphere using (3.24), where Rc   denotes the radius of hypersphere c,D(xi,c) denotes the distance from data point xi to the center of hypersphere c(3.24)φ(xi,c)={λ(1-(D(xi,c)/Rc)1+(D(xi,c)/Rc))+γ,0≤D(xi,c)≤Rc,γ(RcD(xi,c)),D(xi,c)>Rc,λ,γ∈R+,λ+γ=1.And the testing data points can be classified according to the following rules represented with(3.25)F(xi)=argmaxcφ(xi,c).


## 4. Experiments

### 4.1. Data Sets

For the purpose of this study, two bond-rating data sets from Korea and China market, which have been used in [46, 47], are applied, in order to validate the proposed methodology. The data are divided into the following four classes: A1, A2, A3, and A4.


### 4.2. Variables Selection

Methods including independent-samplest-test and F-value are applied in variable selection.In terms of Korea data set, 14 variables, which are listed in Table2, are selected from original ones, which were known to affect bond rating. For better comparison, similar methods were also used in China data set, with 12 variables among them being selected.Table 2
Table of selected variables.
No.
Description
Definition
X1
Shareholders’ equity
A firm’s total assets minus its total liabilities
X2
Sales
Sales
X3
Total debt
Total debt
X4
Sales per employee
Sales/the number of employees
X5
Net income per share
Net income/the number of issued shares
X6*
Years after foundation
Years after foundation
X7
Gross earning to total asset
Gross earning/total Asset
X8
Borrowings-dependency ratio
Interest cost/sales
X9
Financing cost to total cost
Financing cost/total cost
X10
Fixed ratio
Fixed assets/(total assets-debts)
X11*
Inventory assets to current assets
Inventory assets/current assets
X12
Short-term borrowings to total borrowings
Short-term borrowings/total borrowings
X13
Cash flow to total assets
Cash flow/total assets
X14
Cash flow from operating activity
Cash flow from operating activity
*Indicates variables excluded in China data set.


### 4.3. Experiment Results and Discussions

Based on the two data sets, some models based on AI are introduced for experiments. To evaluate the prediction performance, 10-fold cross validation, which has shown good performance in model selection [48], is followed. In the research, all features, which are represented with variables listed in Table 2, of data points range from 0 to 1 after Min-max transformation. To validate the methodology oriented multiclassification problem in credit rating, ten percent of the data points for each class are selected as testing samples. And the results of experiments on proposed method, with 0.9 being chosen as the value of threshold intuitively, are shown in Table 3.Table 3
Experimental results of the proposed method.
Data set
Korea data set
China data set
No.
Train (%)
Valid (%)
Train (%)
Valid (%)
1
68.26
67.14
67.29
66.17
2
80.01*
71.23
68.35
67.13
3
73.21
70.62
71.56
71.01
4
75.89
72.37
75.24
72.36
5
76.17
74.23
84.17*
83.91*
6
75.28
75.01
80.02
79.86
7
78.29
76.23*
76.64
74.39
8
77.29
74.17
72.17
71.89
9
75.23
71.88
83.27
80.09
10
70.16
68.34
72.16
70.16
Avg.
74.98
72.12
75.09
73.70
*The best performance for each data set.To compare with other methods, the proposed model is compared with some other MSVM techniques, namely, ANN, One-Against-All, One-Against-One, DAGSVM, Grammer & Singer, OMSVM [46], and standard SVDD. The results concluded in the paper are all shown as average values obtained following 10-fold cross validation based on platform of Matlab 7.0.To compare the performance of each algorithm, hit-ratio, which is defined according to the samples classified correctly, is applied. And the experiment results are listed in Table4.Table 4
Table of experiment results.
Type
Technique
Korea data set
China data set
Valid (%)
Time (Second)
Valid (%)
Time (Second)
Prior AI approach
ANN
62.78
1.67
67.19
1.52
Conventional MSVM
One-against-all
70.23
2.68
71.26
2.60
One-against-one
71.76
2.70
72.13
2.37
DAGSVM [28]
69.21
2.69
71.13
2.61
Grammer & Singer [29]
70.07
2.62
70.91
2.50
OMSVM [46]
71.61
2.67
72.08
2.59
The sphere-based classifier
Standard SVDD
72.09
1.70
72.98
1.04
proposed method (θ=0.9)
72.12
1.20
73.70
0.86As shown in Table4, the proposed method based on thoughts of hypersphere achieves better performance than conventional SVM models based on thoughts of hyperplane. Moreover, as one of modified models, some results obtained imply that the proposed method has better generalization ability and less computational complexity, which can be partially measured with training time labeled with “Time,” than standard SVDD.Furthermore, as one of modified models based on standard SVDD, the proposed method accomplishes data preprocessing using KFAMC. Since the fuzzy area is determined by thresholdθ, greater value of θ will lead to bigger fuzzy area. Especially, when θ=1, the algorithm proposed will be transformed to standard SVDD because almost all data points are positioned in fuzzy area. Hence, a model with too large threshold may be little different from standard SVDD, while a too small value will have poor ability of sphere-based classifier establishment due to lack of essential training samples. Thus, issues on choosing the appropriate threshold are discussed by empirical trials in the paper.In the following experiment, the proposed method with various threshold values is tested based on different data sets, just as shown in Figure8.Experiment results of generalization ability on data sets. (AUC represents hit-ratio of testing samples).
(a)
Experiments based on korea data set
(b)
Experiments based on china data setThe results illustrated in Figure8 showed that the proposed method achieved best performance with threshold of 0.9 based on Korea data set. But in terms of China market, it achieved best performance with the threshold of 0.8 rather than a larger one due to effects of more outliers existing in data set.Moreover, training time of proposed method can be also compared with standard SVDD, just as illustrated in Figure9.Experiment results of training time on data sets.
(a)
Experiments based on korea data set
(b)
Experiments based on china data setJust as shown in Figure9, results of experiments based on different data sets are similar. That is, with decline of threshold, more samples were eliminated from training set through preprocessing based on KFAMC to reduce training time. Hence, smaller values of threshold will lead to less computational consumption partly indicated as training time, while classification accuracy may be decreased due to lack of necessary training samples. Overall, threshold selection, which involves complex tradeoffs between computational consumption and classification accuracy, is essential to the proposed method.


## 4.1. Data Sets

For the purpose of this study, two bond-rating data sets from Korea and China market, which have been used in [46, 47], are applied, in order to validate the proposed methodology. The data are divided into the following four classes: A1, A2, A3, and A4.


## 4.2. Variables Selection

Methods including independent-samplest-test and F-value are applied in variable selection.In terms of Korea data set, 14 variables, which are listed in Table2, are selected from original ones, which were known to affect bond rating. For better comparison, similar methods were also used in China data set, with 12 variables among them being selected.Table 2
Table of selected variables.
No.
Description
Definition
X1
Shareholders’ equity
A firm’s total assets minus its total liabilities
X2
Sales
Sales
X3
Total debt
Total debt
X4
Sales per employee
Sales/the number of employees
X5
Net income per share
Net income/the number of issued shares
X6*
Years after foundation
Years after foundation
X7
Gross earning to total asset
Gross earning/total Asset
X8
Borrowings-dependency ratio
Interest cost/sales
X9
Financing cost to total cost
Financing cost/total cost
X10
Fixed ratio
Fixed assets/(total assets-debts)
X11*
Inventory assets to current assets
Inventory assets/current assets
X12
Short-term borrowings to total borrowings
Short-term borrowings/total borrowings
X13
Cash flow to total assets
Cash flow/total assets
X14
Cash flow from operating activity
Cash flow from operating activity
*Indicates variables excluded in China data set.


## 4.3. Experiment Results and Discussions

Based on the two data sets, some models based on AI are introduced for experiments. To evaluate the prediction performance, 10-fold cross validation, which has shown good performance in model selection [48], is followed. In the research, all features, which are represented with variables listed in Table 2, of data points range from 0 to 1 after Min-max transformation. To validate the methodology oriented multiclassification problem in credit rating, ten percent of the data points for each class are selected as testing samples. And the results of experiments on proposed method, with 0.9 being chosen as the value of threshold intuitively, are shown in Table 3.Table 3
Experimental results of the proposed method.
Data set
Korea data set
China data set
No.
Train (%)
Valid (%)
Train (%)
Valid (%)
1
68.26
67.14
67.29
66.17
2
80.01*
71.23
68.35
67.13
3
73.21
70.62
71.56
71.01
4
75.89
72.37
75.24
72.36
5
76.17
74.23
84.17*
83.91*
6
75.28
75.01
80.02
79.86
7
78.29
76.23*
76.64
74.39
8
77.29
74.17
72.17
71.89
9
75.23
71.88
83.27
80.09
10
70.16
68.34
72.16
70.16
Avg.
74.98
72.12
75.09
73.70
*The best performance for each data set.To compare with other methods, the proposed model is compared with some other MSVM techniques, namely, ANN, One-Against-All, One-Against-One, DAGSVM, Grammer & Singer, OMSVM [46], and standard SVDD. The results concluded in the paper are all shown as average values obtained following 10-fold cross validation based on platform of Matlab 7.0.To compare the performance of each algorithm, hit-ratio, which is defined according to the samples classified correctly, is applied. And the experiment results are listed in Table4.Table 4
Table of experiment results.
Type
Technique
Korea data set
China data set
Valid (%)
Time (Second)
Valid (%)
Time (Second)
Prior AI approach
ANN
62.78
1.67
67.19
1.52
Conventional MSVM
One-against-all
70.23
2.68
71.26
2.60
One-against-one
71.76
2.70
72.13
2.37
DAGSVM [28]
69.21
2.69
71.13
2.61
Grammer & Singer [29]
70.07
2.62
70.91
2.50
OMSVM [46]
71.61
2.67
72.08
2.59
The sphere-based classifier
Standard SVDD
72.09
1.70
72.98
1.04
proposed method (θ=0.9)
72.12
1.20
73.70
0.86As shown in Table4, the proposed method based on thoughts of hypersphere achieves better performance than conventional SVM models based on thoughts of hyperplane. Moreover, as one of modified models, some results obtained imply that the proposed method has better generalization ability and less computational complexity, which can be partially measured with training time labeled with “Time,” than standard SVDD.Furthermore, as one of modified models based on standard SVDD, the proposed method accomplishes data preprocessing using KFAMC. Since the fuzzy area is determined by thresholdθ, greater value of θ will lead to bigger fuzzy area. Especially, when θ=1, the algorithm proposed will be transformed to standard SVDD because almost all data points are positioned in fuzzy area. Hence, a model with too large threshold may be little different from standard SVDD, while a too small value will have poor ability of sphere-based classifier establishment due to lack of essential training samples. Thus, issues on choosing the appropriate threshold are discussed by empirical trials in the paper.In the following experiment, the proposed method with various threshold values is tested based on different data sets, just as shown in Figure8.Experiment results of generalization ability on data sets. (AUC represents hit-ratio of testing samples).
(a)
Experiments based on korea data set
(b)
Experiments based on china data setThe results illustrated in Figure8 showed that the proposed method achieved best performance with threshold of 0.9 based on Korea data set. But in terms of China market, it achieved best performance with the threshold of 0.8 rather than a larger one due to effects of more outliers existing in data set.Moreover, training time of proposed method can be also compared with standard SVDD, just as illustrated in Figure9.Experiment results of training time on data sets.
(a)
Experiments based on korea data set
(b)
Experiments based on china data setJust as shown in Figure9, results of experiments based on different data sets are similar. That is, with decline of threshold, more samples were eliminated from training set through preprocessing based on KFAMC to reduce training time. Hence, smaller values of threshold will lead to less computational consumption partly indicated as training time, while classification accuracy may be decreased due to lack of necessary training samples. Overall, threshold selection, which involves complex tradeoffs between computational consumption and classification accuracy, is essential to the proposed method.


## 5. Conclusions and Directions for Future Research

In the study, a novel algorithm based on Fuzzy SVDD combined with Fuzzy Clustering for credit rating is proposed. The underlying assumption of the proposed method is that sufficient boundary points could support a close boundary around the target data but too many ones might cause overfitting and poor generalization ability. In contrast to prior researches, which just applied conventional MSVM algorithms in credit ratings, the algorithm based on sphere-based classifier is introduced with samples preprocessed using fuzzy clustering algorithm.As a result, through appropriate threshold setting, generalization performance measured by hit-ratio of the proposed method is better than that of standard SVDD, which outperformed many kinds of conventional MSVM algorithms argued in prior literatures. Moreover, as a modified sphere-based classifier, proposed method has much less computational consumption than standard SVDD.One of the future directions is to accomplish survey studies comparing different bond-rating processes, with deeper market structure analysis also achieved. Moreover, as one of the MSVM algorithms, the proposed method can be applied in other areas besides credit ratings. And some more experiments on data sets such asUCI repository [49] are to be accomplished in the future.



---

*Source: 302624-2012-07-25.xml*