# Construction and Optimization of Fuzzy Rule-Based Classifier with a Swarm Intelligent Algorithm

Authors: Li Mao; Qidong Chen; Jun Sun
Journal: Mathematical Problems in Engineering
 (2020)

---

## Abstract

In this paper, we propose a particle swarm optimization method incorporating quantum qubit operation to construct and optimize fuzzy rule-based classifiers. The proposed algorithm, denoted as QiQPSO, is inspired by the quantum computing principles. It employs quantum rotation gates to update the probability of each qubit with the corresponding quantum angle updating according to the update equation of the quantum-behaved particle swarm optimization (QPSO). After description of the principle of QiQPSO, we show how to apply QiQPSO to establish a fuzzy classifier through two procedures. The QiQPSO algorithm is first used to construct the initial fuzzy classification system based on the sample data and the grid method of partitioning the feature space, and then the fuzzy rule base of the initial fuzzy classifier is optimized further by QiQPSO in order to reduce the number of the fuzzy rules and thus improve its interpretability. In order to verify the effectiveness of the proposed method, QiQPSO is tested on various real-world classification problems. The experimental results show that the QiQPSO is able to effectively select feature variables and fuzzy rules of the fuzzy classifiers with high classification accuracies. The performance comparison with other methods also shows that the fuzzy classifier optimized by QiQPSO has higher interpretability as well as comparable or even better classification accuracies.


---

## Body

## 1. Introduction

Pattern classification, as one of the most important problems in the field of pattern recognition and machine learning, attempts to assign each input value to one of a given set of classes [1]. It involves design of a classifier and the use of the classifier to classify the given data. Generally, in practical classification problems, uncertainty or noise interference is always contained in the data, making it difficult to classify the data accurately with classical crisp classification methods [2]. In contrast, fuzzy sets and fuzzy logic provide a means for soft classification of data that allows us to analyze the data samples in a more sensitive way. For example, the use of fuzzy rules for classification is known to be a good method for classification of knowledge representation, which is similar to human knowledge expression, so that it leads to efficient, transparent, and easily interpretable fuzzy classifiers [3]. During the past few decades, fuzzy classification has been widely used in many fields, including speech recognition, character recognition, text classification, image processing, and weather forecast, to name only a few [4–9].Fuzzy rule-based classification for a practical problem employs two kinds of methods of extracting fuzzy rules [10]. The first one is to generate fuzzy inference rules based on the professional knowledge of experts in a particular field, while the second one is to reason out corresponding fuzzy IF-THEN rules from a set of given data [9]. In this paper and other related literature, the so-called fuzzy classification employs the second type of method to generate a set of fuzzy classification rules. For the second type of methods, researchers also have proposed different approaches for the purpose of effectively extracting the fuzzy classification rules. For example, Abe and Lan [11] proposed to carry out clustering on the sample data first and then define a fuzzy rule for each cluster. Castellano and Fanelli [12] designed a grid partition method to generate the initial fuzzy classifier and then used the artificial neural network to reduce the number of the fuzzy rules. An important class of approaches in this type is to employ a training algorithm to perform the training task or to optimize fuzzy classification rules [13]. Many heuristic methods, such as genetic algorithm (GA), simulated annealing, genetic programming, and particle swarm optimization (PSO), have been used to optimize the fuzzy rules of classifiers [14–21]. These methods act as the training algorithms to construct or optimize the fuzzy rule base to maximize the classification accuracy and minimize the number of the fuzzy rules on the given training samples.The goal of this paper is to develop a novel effective method to optimize the fuzzy classifier based on a modified version of quantum-behaved particle swarm optimization (QPSO), which incorporates quantum computing principles into the original QPSO algorithm. As a variant of PSO, the QPSO algorithm has been shown to successfully solve a wide range of optimization problems due to its stronger search ability [22–27]. However, since the QPSO was designed aiming for continuous optimization problems, it cannot be directly applied to optimization of fuzzy classification rules, which is known as a discrete optimization problem. Hence, in this paper, we incorporate a quantum bit operation into the QPSO algorithm and discretize the algorithm to be suitable for optimizing fuzzy rule-based classifiers. Quantum computing principles have been employed to design evolutionary algorithms by many researchers [28–33]. For example, Han and Kim [30] proposed the quantum-inspired evolutionary algorithm (QEA), which is characterized by the concepts of qubits (Q-bit) and superposition of states. Hammed et al. [33] designed a quantum-inspired particle swarm optimization in which the rotation angle in the rotation gate is represented by the velocity of the particle. The proposed algorithm is motivated by these works and is denoted as quantum-inspired QPSO (QiQPSO), in which each individual particle is encoded as a string of qubits and is updated by a rotation gate with the rotation angle evolving according to the iterative equation of the original QPSO.The paper is structured as follows. In Section2, we describe how to construct a fuzzy rule-based classification system. Section 3 presents the proposed QiQPSO, and Section 4 describes the procedure of how to construct an initial fuzzy rule-based classifier and optimize its fuzzy rule base further by using the QiQPSO algorithm. Experiment results are presented and analyzed in Section 5. Finally, the paper is concluded in Section 6.


## 2. Fuzzy Rule-Based Classifiers

Fuzzy rule-based classifier is an important application of fuzzy sets and is essentially a set of IF-THEN classification rules [2]. The real-world classification problems generally contain uncertainty noise interference, which can be tackled effectively by fuzzy rule-based classification methods. The fuzzy rule-based classification system has many advantages such as its consistence with representation of human knowledge, better classification performance, understandability, and strong readability and interpretability.There are three types of fuzzy classifiers corresponding to three different types of consequents of the fuzzy rules [2]. The first type of consequents only indicates the class that the input data belongs to. That is, it has the following form:(1)Rk:…,THEN class isyok,where y_(o(k)) denotes the class represented by the o(k)-th input. The second type of consequents of fuzzy rules is expressed by(2)Rk:…,THEN  class isyokwithCFk,ok,where yok denotes the class that the ok-th input data belongs to and CFk,ok∈0,1 refers to the degree of confidence with which class the input data belongs to. The third type of the fuzzy classifier has the following form of consequents:(3)Rk:…,THEN class isyokiwithCFk,oki,where yoki indicates the class that the oki-th input data belongs to, ki denotes the all possible values with the scope of the output variable, that is, yoki represents all possible classes, and CFk,oki∈0,1 refers to the degree of confidence.In this paper, we adopt the form in (3) for the consequents of fuzzy rules. As such, the fuzzy classification system can be formulated in detail as follows. Suppose that there are n samples belonging to M classes. For the fuzzy classifier, there are m training samples with xp=xp1,xp2,…,xpm being the p-th 1≤p≤m training data, and yp∈1,2,…,M denotes the class that xp belongs to. The problem of classifying the n samples into M classes can be defined as follows. For the input variable x¯=x1,x2,…,xm, the IF-THEN rules for the fuzzy classifier are given by(4)Rj1,j2,...,jn:IFx1isA1,j1andx2isA2,j2and,…,xnisAn,jn,THENx¯belong  to  Classyj1,j2,...,jnwithCF=CFj1,j2,jn,where A1,j1 denotes theji-th fuzzy set for the input variable (or feature variable) xi in the antecedent of the fuzzy rule, yj1,j2,…,jn∈1,2,…M denotes the output variable of the consequent of the fuzzy rule (i.e., the class the sample belongs to), and CFj1,j2,…,jn represents the degree of confidence for the IF-THEN fuzzy rule, Rj1,j2,…,jn.For the fuzzy classification system represented by (4), we define the membership function of the input variablexi in the antecedent of the IF-THEN rule on fuzzy set ji:(5)Ai,jici,ji,wi,jil,wi,jir;xi=exp−xi−ci,jiwi,jil2;ifxi≤ci,ji,exp−xi−ci,jiwi,jir2;ifxi≤ci,ji.This membership is essentially Gaussian one including three parameters,ci,ji denoting the center of the membership function and wi,jil and wi,jir representing the left and right widths of the membership function, respectively, which are not necessarily equal. We can select the value of the three parameters to determine the membership function for the antecedent of the fuzzy rules.The center of the membership functionci,ji is the center of the fuzzy partition. When selecting wi,jil and wi,jir, we introduce a constantμ, which is known as an overlap factor and reflects the overlap degree of the membership:(6)Ai,jici,ji−1=exp−xi−ci,jiwi,jil2=μ,Ai,jici,ji+1=exp−xi−ci,jiwi,jir2=μ,in order to improve the interpretability of membership functions. Thus wi,jil and wi,jir can be determined by(7)wi,jil=−ci,ji−1−ci,ji2lnμ1/2,wi,jir=−ci,ji+1−ci,ji2lnμ1/2.


## 3. The Proposed QIQPSO

In this paper, we propose a quantum-inspired quantum-behaved particle swarm optimization (QiQPSO), of which the position of particlei at the t-th iteration is represented as a string of qubits. Unlike the bits representing the information in either 0’s or 1’s in the classical computing, in quantum computing, a qubit could take the value of 0, 1, or a superposition of both [33]. Superposition allows the possible states to represent both 0 and 1 simultaneously based on its probability. In this section, we first give a brief introduction of QPSO and then propose the QiQPSO algorithm.


### 3.1. A Brief Introduction to QPSO

Quantum-behaved particle swarm optimization (QPSO) is a variant of PSO motivated by quantum mechanics and the trajectory analysis of the canonical PSO [22, 23, 34, 35, 36]. In the QPSO with L individuals, each individual is treated as a volume-less and spin-less particle in the N-dimensional space, with the current position vector of particle i at the t-th iteration represented by Xi,t=Xi,t1,Xi,t2,…,Xi,tN. Besides, each particle i also has a vector Pi,t=Pi,t1,Pi,t2,…,Pi,tN, called personal best (pbest) position, which is the best previous position (the position giving the best objective function value or fitness value) of particle i. The whole swarm has vector Gt=Gt1,Gt2,…,GtN, known as the global best (gbest) position, which is the position of the best particle among all the particles in the population. Consider the following minimization problem:(8)minimizefX,s.tX∈S⊆RN,where fX is an objective function continuous almost everywhere and S is the feasible space. Note that fX always has many local minima, which leads to its difficulty in finding a global minima when minimizing it. Then, Pi,tcan be updated by(9)Pi,t=Xi,tiffXi,t<fPi,t−1,Pi,t−1ifXi,t≥fPi,t−1,and accordingly Gt can be found by Gt=Pg,t, where g=argmin1≤i≤LfPi,t. The particle in the QPSO updates its position in the following equation [22-27]:(10)Xi,t+1j=pi,tj±γXi,tj−Ctjln1ui,t+1j,for 1≤i≤L and 1≤j≤N, where(11)pi,tj=φi,tjPi,tj+1−φi,tjGtj,φi,tj∼U0,1.In (10), Ct=Ct1,Ct2,…,CtN is called the mean best (mbest) position defined by the average of the pbest positions of all particles, i.e., Ctj=1/L∑i=1LPi,tj,1≤j≤N, ui,t+1jIs a sequence of random numbers uniformly distributed on (0, 1), and parameter γ is known as the contraction-expansion (CE) coefficient, which can be adjusted to balance the local and global search of the algorithm during the optimization process.


### 3.2. Quantum-Inspired QPSO

In the quantum computing and quantum information, the quantum state is modeled by the Hilbert space of wave functions and is defined asα2(12)ψ=α1+β0,where α and β are the complex numbers defining probabilities at which the corresponding state is likely to appear when a qubit collapses, for instance, when reading or measuring. Probability fundamentals stated that(13)α2+β2=1,where α2 gives the probability that a qubit is in the OFF (0) state and β2 gives the probability that a qubit is in the ON (1) state. The probability of αβ can be represented as a quantum angle θ, where cosθsinθ satisfies the probability fundamental of(14)sinθ2+cosθ2=1,where the parameter θ has been used to calculate and update the probability in quantum-inspired EA [29, 30] and quantum-inspired PSO [33].In this paper, we proposed a quantum-inspired quantum-behaved particle swarm optimization. In this method, the position of particlei at the t-th iteration is represented as a string of qubits, which corresponds to a vector of quantum angles θi,t=θi,t1,θi,t2…,θi,tN, where θi,tj denotes the quantum angle of the j-th qubit 1≤j≤N and the vector of the quantum angles corresponding to the personal best position of particle i is θpbesti,n=θpbesti,t1,θpbesti,t2…,θpbesti,tN.The vector of quantum angles corresponds to the global best position of the swarm and is denoted asθgbestt=θgbestt1,θgbesti,t2,…,θgbesti,tN. Thus, according to equation (10), the quantum angle for each qubit is updated as(15)θi,t+1j=θpi,tj±γθmbesttj−θi,tjln1μi,t+1j,where θmbesttj is the quantum angle corresponding to the j-th qubit of the mbest position of the swarm at the t-th iteration. However, we have to obtain the increment of the quantum angle to update the probability by using a rotation gate. Thus, we further get the increment of the quantum angle for each qubit by(16)Δθi,t+1j=θi,t+1j−θi,tj=θpi,tj±γθmbesttj−θi,tjln1μi,t+1j−θi,tj,with which we can update the probability corresponding to each qubit by the following qubit rotation gate:(17)αi,t+1jβi,t+1j=cosΔθi,t+1j−sinΔθi,t+1jsinΔθi,t+1jcosΔθi,t+1jαi,tjβi,tj,where αi,tjβi,tj represents the probability of the j-th qubit of particle i at the t-th iteration, with which we can collapse the corresponding qubit to be “0” or “1”.


## 3.1. A Brief Introduction to QPSO

Quantum-behaved particle swarm optimization (QPSO) is a variant of PSO motivated by quantum mechanics and the trajectory analysis of the canonical PSO [22, 23, 34, 35, 36]. In the QPSO with L individuals, each individual is treated as a volume-less and spin-less particle in the N-dimensional space, with the current position vector of particle i at the t-th iteration represented by Xi,t=Xi,t1,Xi,t2,…,Xi,tN. Besides, each particle i also has a vector Pi,t=Pi,t1,Pi,t2,…,Pi,tN, called personal best (pbest) position, which is the best previous position (the position giving the best objective function value or fitness value) of particle i. The whole swarm has vector Gt=Gt1,Gt2,…,GtN, known as the global best (gbest) position, which is the position of the best particle among all the particles in the population. Consider the following minimization problem:(8)minimizefX,s.tX∈S⊆RN,where fX is an objective function continuous almost everywhere and S is the feasible space. Note that fX always has many local minima, which leads to its difficulty in finding a global minima when minimizing it. Then, Pi,tcan be updated by(9)Pi,t=Xi,tiffXi,t<fPi,t−1,Pi,t−1ifXi,t≥fPi,t−1,and accordingly Gt can be found by Gt=Pg,t, where g=argmin1≤i≤LfPi,t. The particle in the QPSO updates its position in the following equation [22-27]:(10)Xi,t+1j=pi,tj±γXi,tj−Ctjln1ui,t+1j,for 1≤i≤L and 1≤j≤N, where(11)pi,tj=φi,tjPi,tj+1−φi,tjGtj,φi,tj∼U0,1.In (10), Ct=Ct1,Ct2,…,CtN is called the mean best (mbest) position defined by the average of the pbest positions of all particles, i.e., Ctj=1/L∑i=1LPi,tj,1≤j≤N, ui,t+1jIs a sequence of random numbers uniformly distributed on (0, 1), and parameter γ is known as the contraction-expansion (CE) coefficient, which can be adjusted to balance the local and global search of the algorithm during the optimization process.


## 3.2. Quantum-Inspired QPSO

In the quantum computing and quantum information, the quantum state is modeled by the Hilbert space of wave functions and is defined asα2(12)ψ=α1+β0,where α and β are the complex numbers defining probabilities at which the corresponding state is likely to appear when a qubit collapses, for instance, when reading or measuring. Probability fundamentals stated that(13)α2+β2=1,where α2 gives the probability that a qubit is in the OFF (0) state and β2 gives the probability that a qubit is in the ON (1) state. The probability of αβ can be represented as a quantum angle θ, where cosθsinθ satisfies the probability fundamental of(14)sinθ2+cosθ2=1,where the parameter θ has been used to calculate and update the probability in quantum-inspired EA [29, 30] and quantum-inspired PSO [33].In this paper, we proposed a quantum-inspired quantum-behaved particle swarm optimization. In this method, the position of particlei at the t-th iteration is represented as a string of qubits, which corresponds to a vector of quantum angles θi,t=θi,t1,θi,t2…,θi,tN, where θi,tj denotes the quantum angle of the j-th qubit 1≤j≤N and the vector of the quantum angles corresponding to the personal best position of particle i is θpbesti,n=θpbesti,t1,θpbesti,t2…,θpbesti,tN.The vector of quantum angles corresponds to the global best position of the swarm and is denoted asθgbestt=θgbestt1,θgbesti,t2,…,θgbesti,tN. Thus, according to equation (10), the quantum angle for each qubit is updated as(15)θi,t+1j=θpi,tj±γθmbesttj−θi,tjln1μi,t+1j,where θmbesttj is the quantum angle corresponding to the j-th qubit of the mbest position of the swarm at the t-th iteration. However, we have to obtain the increment of the quantum angle to update the probability by using a rotation gate. Thus, we further get the increment of the quantum angle for each qubit by(16)Δθi,t+1j=θi,t+1j−θi,tj=θpi,tj±γθmbesttj−θi,tjln1μi,t+1j−θi,tj,with which we can update the probability corresponding to each qubit by the following qubit rotation gate:(17)αi,t+1jβi,t+1j=cosΔθi,t+1j−sinΔθi,t+1jsinΔθi,t+1jcosΔθi,t+1jαi,tjβi,tj,where αi,tjβi,tj represents the probability of the j-th qubit of particle i at the t-th iteration, with which we can collapse the corresponding qubit to be “0” or “1”.


## 4. Optimizing Fuzzy Classifier with QiQPSO

When using QiQPSO to optimize a fuzzy rule-based classifier, we first construct the initial fuzzy classifier with QiQPSO based on the fuzzy rule base obtained directly from the samples by using the grid partition method, and then we further optimize the fuzzy rule base to reduce the number of the rules with the classification accuracy unchanged. In the section, we describe these two procedures in detail.


### 4.1. Construction of the Initial Fuzzy Classification System with QiQPSO

When using QiQPSO to construct a fuzzy rule-based classifier, we should determine the antecedents of the fuzzy IF-THEN rules firstly. Then, the determined antecedent trains the known data to achieve the consequent of the fuzzy rules. The antecedent of the fuzzy rules contains information about the number of input variables, the partition of input space, and the number of input space partitions. Here, we present how to use QiQPSO to establish the initial fuzzy rule-based classifier, by adopting Gaussian membership functions and using the grid method for partitioning the input space (i.e., the feature space).


#### 4.1.1. Determining Antecedent of Fuzzy Rules

(1)
Selection of Feature Variables.For the purpose of improving the interpretability of a fuzzy system model, the number of the selected input variables (i.e., feature variables) should be as few as possible. The method employed in this paper is to implement feature variable selection by using QiQPSO during the construction of the initial fuzzy classification system. To this end, we represent each particle in QiQPSO as a string of qubits, with the firstn (n is the number of the feature variables) qubits, each reflecting whether the corresponding feature variable is selected or not. If the j-th 1≤j≤n bit is “1”, then the j-th feature variable (or input variable) is selected; otherwise, the feature variable is not selected.(2)
Fuzzy Partition of the Feature Space.We adopt the following procedure, i.e., the grid method, to partition the feature space. First, the feature space is divided evenly, with the number of partitions and the center of each partition determined. Then, QiQPSO is employed to select the midpoints to implement fuzzy partition of the feature variables. Specifically, suppose that the domain of thej-th feature variable xj has Kj even partitions, I1,I2,…,IKj, which is a qubit string composed of 0’s and 1’s. If a bit is “1,” the center of this even partition is selected as the valid center of the fuzzy partition; otherwise, the center is not selected. The number of all the valid centers in this string, namely, the sum of all 1’s in the string, is known as the number of fuzzy partitions kj.Letxjmin and xjmax be the maximum and the minimum of the feature variable xj. The k-th center of the fuzzy partition of xj is then given by(18)cj,k=xjmin+Ijk-1xjmax-xjminkj-1,k∈1,2,…,kj,where Ijk is the index of the k-th bit whose value is “1” in the string. The center of the fuzzy partition is regarded as the center of the membership function. In order to guarantee the interpretability of the membership function, according to (6), we have(19)Aj,kcj,k−1=exp−cj,k−1−cj,kwj,kl2=μ,(20)Aj,kcj,k+1=exp−cj,k+1−cj,kwj,kr2=μ,which means the membership function overlaps at constant μ as shown in Figure 1.Figure 1
Overlap degree of the membership function.Thus, according to (7), the left widthwj,kl and the right width wj,krare given by(21)wj,kl=cj,k−cj,k−12−lnμ,(22)wj,kr=cj,k+1−cj,k2−lnμ,where μ is the overlap factor and it guarantees that when the k-th membership function value is μ, its corresponding variable value coincides with the centers of the (k-1)th and the (k+1)th membership functions. Such membership functions generally have high interpretability.


#### 4.1.2. Encoding of the Particle Position in QPSO

The position of each particle is encoded as a qubit string with two sections, one of which represents the feature variable selection and the other of which reflects the selection of the centers of feature space partitions. As mentioned above, each qubit in a particle position string has a corresponding quantum angle which is updated and then determines the probability of the qubit value according to equations (16) and (17), respectively.


#### 4.1.3. Encoding of the Particle Position in QiQPSO

An important objective in constructing the initial fuzzy classification system is to maximize the number of the correctly classified samples. Thus, the classification accuracy should be one of the most important components in the fitness function. Another objective is to minimize the number of the selected features as the number of the fuzzy rules increases exponentially with the feature number. Thus, the number of the selected feature variables should be another component in the fitness function. The other component in the fitness function is the number of the fuzzy rules since too many rules reduce the interpretability of the fuzzy classifier.As such, the fitness function for the construction of the initial fuzzy classification system with the QiQPSO is given by(23)Nr=∏j=1Nfkj.


#### 4.1.4. Determining the Consequents of Fuzzy Rules

As mentioned in Section2, the consequents of the fuzzy rules include the index y of the class that the sample belongs to and the corresponding degree of confidence. When using the QiQPSO algorithm to construct the initial fuzzy classifier, we can determine the antecedents of the fuzzy rules for a given collapsed qubit string of the particle and then obtain the consequents of the fuzzy rules by the following procedure.First, according to the antecedents of fuzzy rules and the sample data, we determineβCT that reflects the extent to which the samples belong to class T:(24)βCT=∑Xp∈CT∏i=1nAi,jixi,T∈1,2,…,M.Then, we can obtain the class indexy in the consequent of the fuzzy rules by using the following formula:(25)Y=1≤T≤MargmaxβCT,(26)yj1,j2,…,jn=Y.Finally, we can compute the corresponding degree of confidence as(27)CFj1,j2,…,jn=βCY−β∑T=1MβCT,where(28)β=∑T=1MβCTM−1.


#### 4.1.5. Procedure of Constructing the Initial Fuzzy Classifier with QiQPSO

The procedure of constructing the initial fuzzy classifier with QiQPSO is described as follows:Step 1: set the parameters of the QiQPSO algorithm: the number of maximum iterations of the algorithmTmax, the swarm size L, the overlapping factor, and the weights of the components in the fitness function, etc.Step 2: set the number of the iterationt = 1 and randomly initialize the particle swarm, namely, randomly initialize the quantum angle of each qubit of each particle and collapse each qubit to be “0” or “1”.Step 3: do the following procedures.Step 4: whilei≤L, where i is the particle index, do the following procedures.Step 5: according to the qubit string of thek-th particle, determine the selected feature variables, the fuzzy partition of the feature space, and thus the antecedents of the fuzzy rules.Step 6: calculate the parameters of the consequents of the fuzzy rules according to equations (24) to (28).Step 7: calculate the fitness function values of the string of each particle according to (22).Step 8: updateθpbesti,t and θgbestt accordingly.Step 9: calculate the increment of each quantum angle according to (16), update the probability of each qubit according to (17), and collapse each qubit to “0” or “1” accordingly.Step 10: seti = i +1. If i≤L, return to Step 4; otherwise, continue to the next step.Step 11: sett = t + 1. If t≤Tmax, go to Step 3; otherwise, continue to the next step.Step 12: output the collapsed qubit string of the particle with the highest fitness value and determine the fuzzy classification model based on this binary string.


### 4.2. Optimizing the Fuzzy Classifier with QiQPSO

The initial fuzzy classification system constructed by QiQPSO may have many redundant and invalid fuzzy rules since the feature space for the membership functions is partitioned by the grid method. Thus, we need to optimize the fuzzy classifier by removing the redundant and invalid rules without changing the interpretability of the classifier. In this paper, QiQPSO is also employed to achieve this purpose.


#### 4.2.1. Encoding of the Particle Position

Each particle position is encoded as a qubit string with each qubit having a corresponding quantum angle. The value of each collapsed qubit represents whether a fuzzy rule should be included in the fuzzy classifier or not. If the qubit is “1,” the corresponding fuzzy rule is selected; otherwise, it is removed from the fuzzy rule base of the classifier.


#### 4.2.2. Fitness Function

The purpose of optimizing the fuzzy rule base of the fuzzy classifier is to delete the redundant and invalid rules in order to minimize the number of the fuzzy rules without reducing the classification accuracy of the classifier. Thus, there should be two components in the fitness function, one of which is the number of the correctly classified samples and the other of which is the number of fuzzy rules. As a result, the following fitness function is employed for fuzzy rule base optimization:(29)F2I=ω4Nc−ω5Nr,where ω4 and ω5 are the weights of the two components.


#### 4.2.3. The Procedure of the Fuzzy Rule Base Optimization with QiQPSO

With the above specification, the procedure of optimizing the fuzzy rule base of the classifier is presented as follows:Step 1: set the parameters of the QiQPSO algorithm: the number of maximum iterations of the algorithmTmax, the swarm size L, the weights of the components in the fitness function, etc.Step 2: set the number of the iterationt = 1 and randomly initialize the particle swarm, namely, randomly initialize the quantum angle of each qubit of each particle and collapse each qubit to be “0” or “1”.Step 3: do the following procedure.Step 4: for each particlei, do the following procedure.Step 5: according to the collapsed qubit string of thei-th particle, determine the fuzzy rules selected into the fuzzy classifier and find the number of selected fuzzy rules Nr.Step 6: classify all the samples by the given fuzzy classifier and determine the number of the correctly classified samplesNc.Step 7: calculate the fitness function values of the collapsed qubit string of each particle according to (29).Step 8: updateθpbesti,t and θgbestt accordingly.Step 9: calculate the increment of each quantum angle according to (16), update the probability of each qubit according to (17), and collapse each qubit to “0” or “1” accordingly.Step 10: seti = i + 1. If i≤L, return to Step 4; otherwise, continue to the next step.Step 11: sett = t + 1. If t≤Tmax, go to Step 3; otherwise, continue to the next step.Step 12: output the binary string of the particle with the highest fitness value and determine the fuzzy classification model based on this binary string.


## 4.1. Construction of the Initial Fuzzy Classification System with QiQPSO

When using QiQPSO to construct a fuzzy rule-based classifier, we should determine the antecedents of the fuzzy IF-THEN rules firstly. Then, the determined antecedent trains the known data to achieve the consequent of the fuzzy rules. The antecedent of the fuzzy rules contains information about the number of input variables, the partition of input space, and the number of input space partitions. Here, we present how to use QiQPSO to establish the initial fuzzy rule-based classifier, by adopting Gaussian membership functions and using the grid method for partitioning the input space (i.e., the feature space).


### 4.1.1. Determining Antecedent of Fuzzy Rules

(1)
Selection of Feature Variables.For the purpose of improving the interpretability of a fuzzy system model, the number of the selected input variables (i.e., feature variables) should be as few as possible. The method employed in this paper is to implement feature variable selection by using QiQPSO during the construction of the initial fuzzy classification system. To this end, we represent each particle in QiQPSO as a string of qubits, with the firstn (n is the number of the feature variables) qubits, each reflecting whether the corresponding feature variable is selected or not. If the j-th 1≤j≤n bit is “1”, then the j-th feature variable (or input variable) is selected; otherwise, the feature variable is not selected.(2)
Fuzzy Partition of the Feature Space.We adopt the following procedure, i.e., the grid method, to partition the feature space. First, the feature space is divided evenly, with the number of partitions and the center of each partition determined. Then, QiQPSO is employed to select the midpoints to implement fuzzy partition of the feature variables. Specifically, suppose that the domain of thej-th feature variable xj has Kj even partitions, I1,I2,…,IKj, which is a qubit string composed of 0’s and 1’s. If a bit is “1,” the center of this even partition is selected as the valid center of the fuzzy partition; otherwise, the center is not selected. The number of all the valid centers in this string, namely, the sum of all 1’s in the string, is known as the number of fuzzy partitions kj.Letxjmin and xjmax be the maximum and the minimum of the feature variable xj. The k-th center of the fuzzy partition of xj is then given by(18)cj,k=xjmin+Ijk-1xjmax-xjminkj-1,k∈1,2,…,kj,where Ijk is the index of the k-th bit whose value is “1” in the string. The center of the fuzzy partition is regarded as the center of the membership function. In order to guarantee the interpretability of the membership function, according to (6), we have(19)Aj,kcj,k−1=exp−cj,k−1−cj,kwj,kl2=μ,(20)Aj,kcj,k+1=exp−cj,k+1−cj,kwj,kr2=μ,which means the membership function overlaps at constant μ as shown in Figure 1.Figure 1
Overlap degree of the membership function.Thus, according to (7), the left widthwj,kl and the right width wj,krare given by(21)wj,kl=cj,k−cj,k−12−lnμ,(22)wj,kr=cj,k+1−cj,k2−lnμ,where μ is the overlap factor and it guarantees that when the k-th membership function value is μ, its corresponding variable value coincides with the centers of the (k-1)th and the (k+1)th membership functions. Such membership functions generally have high interpretability.


### 4.1.2. Encoding of the Particle Position in QPSO

The position of each particle is encoded as a qubit string with two sections, one of which represents the feature variable selection and the other of which reflects the selection of the centers of feature space partitions. As mentioned above, each qubit in a particle position string has a corresponding quantum angle which is updated and then determines the probability of the qubit value according to equations (16) and (17), respectively.


### 4.1.3. Encoding of the Particle Position in QiQPSO

An important objective in constructing the initial fuzzy classification system is to maximize the number of the correctly classified samples. Thus, the classification accuracy should be one of the most important components in the fitness function. Another objective is to minimize the number of the selected features as the number of the fuzzy rules increases exponentially with the feature number. Thus, the number of the selected feature variables should be another component in the fitness function. The other component in the fitness function is the number of the fuzzy rules since too many rules reduce the interpretability of the fuzzy classifier.As such, the fitness function for the construction of the initial fuzzy classification system with the QiQPSO is given by(23)Nr=∏j=1Nfkj.


### 4.1.4. Determining the Consequents of Fuzzy Rules

As mentioned in Section2, the consequents of the fuzzy rules include the index y of the class that the sample belongs to and the corresponding degree of confidence. When using the QiQPSO algorithm to construct the initial fuzzy classifier, we can determine the antecedents of the fuzzy rules for a given collapsed qubit string of the particle and then obtain the consequents of the fuzzy rules by the following procedure.First, according to the antecedents of fuzzy rules and the sample data, we determineβCT that reflects the extent to which the samples belong to class T:(24)βCT=∑Xp∈CT∏i=1nAi,jixi,T∈1,2,…,M.Then, we can obtain the class indexy in the consequent of the fuzzy rules by using the following formula:(25)Y=1≤T≤MargmaxβCT,(26)yj1,j2,…,jn=Y.Finally, we can compute the corresponding degree of confidence as(27)CFj1,j2,…,jn=βCY−β∑T=1MβCT,where(28)β=∑T=1MβCTM−1.


### 4.1.5. Procedure of Constructing the Initial Fuzzy Classifier with QiQPSO

The procedure of constructing the initial fuzzy classifier with QiQPSO is described as follows:Step 1: set the parameters of the QiQPSO algorithm: the number of maximum iterations of the algorithmTmax, the swarm size L, the overlapping factor, and the weights of the components in the fitness function, etc.Step 2: set the number of the iterationt = 1 and randomly initialize the particle swarm, namely, randomly initialize the quantum angle of each qubit of each particle and collapse each qubit to be “0” or “1”.Step 3: do the following procedures.Step 4: whilei≤L, where i is the particle index, do the following procedures.Step 5: according to the qubit string of thek-th particle, determine the selected feature variables, the fuzzy partition of the feature space, and thus the antecedents of the fuzzy rules.Step 6: calculate the parameters of the consequents of the fuzzy rules according to equations (24) to (28).Step 7: calculate the fitness function values of the string of each particle according to (22).Step 8: updateθpbesti,t and θgbestt accordingly.Step 9: calculate the increment of each quantum angle according to (16), update the probability of each qubit according to (17), and collapse each qubit to “0” or “1” accordingly.Step 10: seti = i +1. If i≤L, return to Step 4; otherwise, continue to the next step.Step 11: sett = t + 1. If t≤Tmax, go to Step 3; otherwise, continue to the next step.Step 12: output the collapsed qubit string of the particle with the highest fitness value and determine the fuzzy classification model based on this binary string.


## 4.1.1. Determining Antecedent of Fuzzy Rules

(1)
Selection of Feature Variables.For the purpose of improving the interpretability of a fuzzy system model, the number of the selected input variables (i.e., feature variables) should be as few as possible. The method employed in this paper is to implement feature variable selection by using QiQPSO during the construction of the initial fuzzy classification system. To this end, we represent each particle in QiQPSO as a string of qubits, with the firstn (n is the number of the feature variables) qubits, each reflecting whether the corresponding feature variable is selected or not. If the j-th 1≤j≤n bit is “1”, then the j-th feature variable (or input variable) is selected; otherwise, the feature variable is not selected.(2)
Fuzzy Partition of the Feature Space.We adopt the following procedure, i.e., the grid method, to partition the feature space. First, the feature space is divided evenly, with the number of partitions and the center of each partition determined. Then, QiQPSO is employed to select the midpoints to implement fuzzy partition of the feature variables. Specifically, suppose that the domain of thej-th feature variable xj has Kj even partitions, I1,I2,…,IKj, which is a qubit string composed of 0’s and 1’s. If a bit is “1,” the center of this even partition is selected as the valid center of the fuzzy partition; otherwise, the center is not selected. The number of all the valid centers in this string, namely, the sum of all 1’s in the string, is known as the number of fuzzy partitions kj.Letxjmin and xjmax be the maximum and the minimum of the feature variable xj. The k-th center of the fuzzy partition of xj is then given by(18)cj,k=xjmin+Ijk-1xjmax-xjminkj-1,k∈1,2,…,kj,where Ijk is the index of the k-th bit whose value is “1” in the string. The center of the fuzzy partition is regarded as the center of the membership function. In order to guarantee the interpretability of the membership function, according to (6), we have(19)Aj,kcj,k−1=exp−cj,k−1−cj,kwj,kl2=μ,(20)Aj,kcj,k+1=exp−cj,k+1−cj,kwj,kr2=μ,which means the membership function overlaps at constant μ as shown in Figure 1.Figure 1
Overlap degree of the membership function.Thus, according to (7), the left widthwj,kl and the right width wj,krare given by(21)wj,kl=cj,k−cj,k−12−lnμ,(22)wj,kr=cj,k+1−cj,k2−lnμ,where μ is the overlap factor and it guarantees that when the k-th membership function value is μ, its corresponding variable value coincides with the centers of the (k-1)th and the (k+1)th membership functions. Such membership functions generally have high interpretability.


## 4.1.2. Encoding of the Particle Position in QPSO

The position of each particle is encoded as a qubit string with two sections, one of which represents the feature variable selection and the other of which reflects the selection of the centers of feature space partitions. As mentioned above, each qubit in a particle position string has a corresponding quantum angle which is updated and then determines the probability of the qubit value according to equations (16) and (17), respectively.


## 4.1.3. Encoding of the Particle Position in QiQPSO

An important objective in constructing the initial fuzzy classification system is to maximize the number of the correctly classified samples. Thus, the classification accuracy should be one of the most important components in the fitness function. Another objective is to minimize the number of the selected features as the number of the fuzzy rules increases exponentially with the feature number. Thus, the number of the selected feature variables should be another component in the fitness function. The other component in the fitness function is the number of the fuzzy rules since too many rules reduce the interpretability of the fuzzy classifier.As such, the fitness function for the construction of the initial fuzzy classification system with the QiQPSO is given by(23)Nr=∏j=1Nfkj.


## 4.1.4. Determining the Consequents of Fuzzy Rules

As mentioned in Section2, the consequents of the fuzzy rules include the index y of the class that the sample belongs to and the corresponding degree of confidence. When using the QiQPSO algorithm to construct the initial fuzzy classifier, we can determine the antecedents of the fuzzy rules for a given collapsed qubit string of the particle and then obtain the consequents of the fuzzy rules by the following procedure.First, according to the antecedents of fuzzy rules and the sample data, we determineβCT that reflects the extent to which the samples belong to class T:(24)βCT=∑Xp∈CT∏i=1nAi,jixi,T∈1,2,…,M.Then, we can obtain the class indexy in the consequent of the fuzzy rules by using the following formula:(25)Y=1≤T≤MargmaxβCT,(26)yj1,j2,…,jn=Y.Finally, we can compute the corresponding degree of confidence as(27)CFj1,j2,…,jn=βCY−β∑T=1MβCT,where(28)β=∑T=1MβCTM−1.


## 4.1.5. Procedure of Constructing the Initial Fuzzy Classifier with QiQPSO

The procedure of constructing the initial fuzzy classifier with QiQPSO is described as follows:Step 1: set the parameters of the QiQPSO algorithm: the number of maximum iterations of the algorithmTmax, the swarm size L, the overlapping factor, and the weights of the components in the fitness function, etc.Step 2: set the number of the iterationt = 1 and randomly initialize the particle swarm, namely, randomly initialize the quantum angle of each qubit of each particle and collapse each qubit to be “0” or “1”.Step 3: do the following procedures.Step 4: whilei≤L, where i is the particle index, do the following procedures.Step 5: according to the qubit string of thek-th particle, determine the selected feature variables, the fuzzy partition of the feature space, and thus the antecedents of the fuzzy rules.Step 6: calculate the parameters of the consequents of the fuzzy rules according to equations (24) to (28).Step 7: calculate the fitness function values of the string of each particle according to (22).Step 8: updateθpbesti,t and θgbestt accordingly.Step 9: calculate the increment of each quantum angle according to (16), update the probability of each qubit according to (17), and collapse each qubit to “0” or “1” accordingly.Step 10: seti = i +1. If i≤L, return to Step 4; otherwise, continue to the next step.Step 11: sett = t + 1. If t≤Tmax, go to Step 3; otherwise, continue to the next step.Step 12: output the collapsed qubit string of the particle with the highest fitness value and determine the fuzzy classification model based on this binary string.


## 4.2. Optimizing the Fuzzy Classifier with QiQPSO

The initial fuzzy classification system constructed by QiQPSO may have many redundant and invalid fuzzy rules since the feature space for the membership functions is partitioned by the grid method. Thus, we need to optimize the fuzzy classifier by removing the redundant and invalid rules without changing the interpretability of the classifier. In this paper, QiQPSO is also employed to achieve this purpose.


### 4.2.1. Encoding of the Particle Position

Each particle position is encoded as a qubit string with each qubit having a corresponding quantum angle. The value of each collapsed qubit represents whether a fuzzy rule should be included in the fuzzy classifier or not. If the qubit is “1,” the corresponding fuzzy rule is selected; otherwise, it is removed from the fuzzy rule base of the classifier.


### 4.2.2. Fitness Function

The purpose of optimizing the fuzzy rule base of the fuzzy classifier is to delete the redundant and invalid rules in order to minimize the number of the fuzzy rules without reducing the classification accuracy of the classifier. Thus, there should be two components in the fitness function, one of which is the number of the correctly classified samples and the other of which is the number of fuzzy rules. As a result, the following fitness function is employed for fuzzy rule base optimization:(29)F2I=ω4Nc−ω5Nr,where ω4 and ω5 are the weights of the two components.


### 4.2.3. The Procedure of the Fuzzy Rule Base Optimization with QiQPSO

With the above specification, the procedure of optimizing the fuzzy rule base of the classifier is presented as follows:Step 1: set the parameters of the QiQPSO algorithm: the number of maximum iterations of the algorithmTmax, the swarm size L, the weights of the components in the fitness function, etc.Step 2: set the number of the iterationt = 1 and randomly initialize the particle swarm, namely, randomly initialize the quantum angle of each qubit of each particle and collapse each qubit to be “0” or “1”.Step 3: do the following procedure.Step 4: for each particlei, do the following procedure.Step 5: according to the collapsed qubit string of thei-th particle, determine the fuzzy rules selected into the fuzzy classifier and find the number of selected fuzzy rules Nr.Step 6: classify all the samples by the given fuzzy classifier and determine the number of the correctly classified samplesNc.Step 7: calculate the fitness function values of the collapsed qubit string of each particle according to (29).Step 8: updateθpbesti,t and θgbestt accordingly.Step 9: calculate the increment of each quantum angle according to (16), update the probability of each qubit according to (17), and collapse each qubit to “0” or “1” accordingly.Step 10: seti = i + 1. If i≤L, return to Step 4; otherwise, continue to the next step.Step 11: sett = t + 1. If t≤Tmax, go to Step 3; otherwise, continue to the next step.Step 12: output the binary string of the particle with the highest fitness value and determine the fuzzy classification model based on this binary string.


## 4.2.1. Encoding of the Particle Position

Each particle position is encoded as a qubit string with each qubit having a corresponding quantum angle. The value of each collapsed qubit represents whether a fuzzy rule should be included in the fuzzy classifier or not. If the qubit is “1,” the corresponding fuzzy rule is selected; otherwise, it is removed from the fuzzy rule base of the classifier.


## 4.2.2. Fitness Function

The purpose of optimizing the fuzzy rule base of the fuzzy classifier is to delete the redundant and invalid rules in order to minimize the number of the fuzzy rules without reducing the classification accuracy of the classifier. Thus, there should be two components in the fitness function, one of which is the number of the correctly classified samples and the other of which is the number of fuzzy rules. As a result, the following fitness function is employed for fuzzy rule base optimization:(29)F2I=ω4Nc−ω5Nr,where ω4 and ω5 are the weights of the two components.


## 4.2.3. The Procedure of the Fuzzy Rule Base Optimization with QiQPSO

With the above specification, the procedure of optimizing the fuzzy rule base of the classifier is presented as follows:Step 1: set the parameters of the QiQPSO algorithm: the number of maximum iterations of the algorithmTmax, the swarm size L, the weights of the components in the fitness function, etc.Step 2: set the number of the iterationt = 1 and randomly initialize the particle swarm, namely, randomly initialize the quantum angle of each qubit of each particle and collapse each qubit to be “0” or “1”.Step 3: do the following procedure.Step 4: for each particlei, do the following procedure.Step 5: according to the collapsed qubit string of thei-th particle, determine the fuzzy rules selected into the fuzzy classifier and find the number of selected fuzzy rules Nr.Step 6: classify all the samples by the given fuzzy classifier and determine the number of the correctly classified samplesNc.Step 7: calculate the fitness function values of the collapsed qubit string of each particle according to (29).Step 8: updateθpbesti,t and θgbestt accordingly.Step 9: calculate the increment of each quantum angle according to (16), update the probability of each qubit according to (17), and collapse each qubit to “0” or “1” accordingly.Step 10: seti = i + 1. If i≤L, return to Step 4; otherwise, continue to the next step.Step 11: sett = t + 1. If t≤Tmax, go to Step 3; otherwise, continue to the next step.Step 12: output the binary string of the particle with the highest fitness value and determine the fuzzy classification model based on this binary string.


## 5. Experimental Results

In this section, we first introduce the data sets used and parameter setting in experiments in detail in Section5.1. In Section 5.2, experimental results for each data set are discussed separately, and we also had done experiments on those data sets with other state-of-the-art algorithms and compared them with the proposed algorithm QiQPSO.


### 5.1. Data Sets and Experiment Settings

The experiments were carried out on various high-dimensional classification problems to establish and optimize fuzzy classifiers for these data sets by using the proposed QiQPSO algorithm, and Table1 summarizes the most important characteristics of these data sets. Iris data consist of 150 samples with four dimensions (namely, calyx length, calyx width, petal length, and petal width) and is divided into three classes, each of which is composed of 50 samples. The samples of the first and the second classes are separated completely, while the second and the third classes intersect. The Wine data set has 178 samples of chemical compositions of three different wines, each sample having 13 attributes. The Glass data set is composed of 9-dimensional 210 samples that can be classified into 6 categories. The Seeds data set consists of 210 data that can be divided into three categories and each sample has seven attributes. In order to verify the performance of the proposed algorithm on more complex real-world problems, Wdbc and Satimage are selected as testing data sets. These two data sets are both high-dimension real-world problems from the UCI repository [11] containing 569 and 6435 samples, respectively.Table 1
Summary of experiment data sets.
Data set name
Patterns
Features
Classes
Iris
150
4
3
Wine
178
13
3
Glass
210
9
6
Seeds
210
7
3
Wdbc
569
30
2
Satimage
6435
36
6In all the experiments, the maximum number of iterations that the QiQPSO algorithm executed for was set to be 100 and the swarm size wasL = 20. Note that these two parameters are set empirically according to the dimension of the optimization problems. That is, higher-dimensional problems generally needs larger maximum number of iterations and larger swarm size. The CE coefficient of the algorithm decreased linearly from 1.0 to 0.5. When constructing the initial fuzzy classifiers, the weights in the fitness function was set to be ω1=1, ω2=0.5, and ω3=2. The overlap factor for the membership functions was set as μ=0.2. When the fuzzy rule bases were optimized with QiQPSO, the weights in the fitness function were set as ω4=1 and ω5=2.


### 5.2. Experimental Results and Analysis

#### 5.2.1. Results for Iris Data Set

For the Iris data set, we denote the four features, i.e., calyx length, calyx width, petal length, and petal width, asx1, x2, x3, and x4, respectively. The initial fuzzy classifier constructed by QiQPSO for this data is shown in Table 2. It can be seen that the initial classification system only uses two feature variables, namely, x3 and x4, both of which employ four Gaussian membership functions to divide the feature space as shown in Figure 2. Thus, the initial classifier has 16 fuzzy rules in total.Table 2
The initial fuzzy classification system constructed by QiQPSO for Iris.
RU
Inputx3
Inputx4
C
CF
Cen
L
R
Cen
L
R
1
1
0
0.9654
0.1
0
0.6548
1
1.0
2
1
0
0.9654
0.7
0.6548
0.6548
1
1.0
3
1
0
0.9654
1.3
0.6548
0.6548
2
0.6842
4
1
0
0.9654
2.4
0.6548
0
1
0.9385
5
2.5
0.9654
0.9654
0.1
0
0.6548
1
1.0
6
2.5
0.9654
0.9654
0.7
0.6548
0.6548
1
1.0
7
2.5
0.9654
0.9654
1.3
0.6548
0.6548
2
0.3335
8
2.5
0.9654
0.9654
2.4
0.6548
0
2
0.3424
9
4.0
0.9654
0.9654
0.1
0
0.6548
2
0.7506
10
4.0
0.9654
0.9654
0.7
0.6548
0.6548
2
1.0
11
4.0
0.9654
0.9654
1.3
0.6548
0.6548
2
0.8935
12
4.0
0.9654
0.9654
2.4
0.6548
0
3
1.0
13
5.9
0.9654
0
0.1
0
0.6548
0
NULL
14
5.9
0.9654
0
0.7
0.6548
0.6548
0
NULL
15
5.9
0.9654
0
1.3
0.6548
0.6548
3
0.7510
16
5.9
0.9654
0
2.4
0.6548
0
3
1.0
RU = rules, Cen = center,L = left width, R = right width, C = classification, and CF = confidence.Figure 2
Membership function of the Iris initial fuzzy classification system.From Table2, it is shown that the initial fuzzy classifier for the Iris data set contains two invalid fuzzy rules (i.e., R13 and R14), suggesting that there is no data under these two fuzzy rules. Using this fuzzy classifier, we can find that the number of the correctly classifier samples is 143 although there are intersections between the second and the third classes.The final fuzzy classifier further optimized by QiQPSO is presented in Table3, in which only 9 fuzzy rules are included. More specifically, the rules in the final fuzzy classifier are(30)R1:IFx3isA4,1andx4isB4,1THENxbelongs to Class1withCF=1.0R2:IFx3isA4,1andx4isB4,2⋯⋯R15:IFx3isA4,4andx4isB4,3THENxbelongs to Class3withCF=0.7510R16:IFx3isA4,4andx4isB4,4THENxbelongs to Class3withCF=1.0.Table 3
The final fuzzy classifier with the fuzzy rule base further optimized by QiQPSO for the Iris data set.
RU
Inputx3
Inputx4
C
CF
Cen
L
R
Cen
L
R
1
1
0
0.9654
0.1
0
0.6548
1
1.0
2
1
0
0.9654
0.7
0.6548
0.6548
1
1.0
5
2.5
0.9654
0.9654
0.1
0
0.6548
1
1.0
6
2.5
0.9654
0.9654
0.7
0.6548
0.6548
1
1.0
10
4.0
0.9654
0.9654
0.7
0.6548
0.6548
2
1.0
11
4.0
0.9654
0.9654
1.3
0.6548
0.6548
2
0.8935
12
4.0
0.9654
0.9654
2.4
0.6548
0
3
1.0
15
5.9
0.9654
0
1.3
0.6548
0.6548
3
0.7510
16
5.9
0.9654
0
2.4
0.6548
0
3
1.0
RU = rules, Cen = center,L = left width, R = right width, C = classification, and CF = confidence.The final fuzzy classifier further optimized by QiQPSO was then tested on the Iris data set. Although the classification accuracy remains the same, the number of the fuzzy rules reduced from 16 to 9 after the fuzzy rule base of the initial classifier was optimized. This means that after the rule base was optimized by QiQPSO, the redundant and invalid fuzzy rules were removed so that the interpretability of the classifier is improved.


#### 5.2.2. Results for Wine Data Set

For the Wine data set, the 13 attributes (i.e., compositions) are alcoholx1, malic acid x2, ash contentx3, ash content alkalinityx4, magnesiumx5, total phenolx6, falconoidx7, phenolic x8, anthocyaninx9, color intensityx10, chromax11, OD280/OD315 dilute winex12, and amino acidx13. Among them, alcohol x1, flavonoidx7, chromax11, OD280/OD315 dilute winex12, and amino acidx13 are the input variables of the initial fuzzy classification system established by QiQPSO, and the feature space of each variable is partitioned by using Gaussian membership functions as shown in Figure 3.Figure 3
Membership functions of the initial fuzzy classification system for the Wine data set.For the wine data set, the initial fuzzy classifier consists of 32 fuzzy rules with five input variables. It can correctly classify 161 samples. After the fuzzy rule base was optimized by QiQPSO, the improved classifier contains only 10 fuzzy rules. More specifically, these rules are expressed as(31)R1:IFx1isA1,1andx7isA7,1andx11isA11,1andx12isA12,1andx13isA13,1THENxbelongs to Class3withCF=1.0R2:IFx1isA1,1andx7isA7,1andx11isA11,1andx12isA12,1andx13isA13,1THENxbelongs to Class3withCF=1.0⋯⋯R15:IFx1isA1,1andx7isA7,1andx11isA11,2andx12isA12,2andx13isA13,1THENxbelongs to Class1withCF=0.9900R16:IFx1isA1,2andx7isA7,1andx11isA11,1andx12isA12,1andx13isA13,1THENxbelongs to Class1withCF=0.9977.The comparison is between the initial fuzzy classification system and the one improved by QiQPSO. It is shown that the initial classification system established by QiQPSO has 5 input variables, 10 fuzzy sets for the antecedents, and 32 fuzzy rules. With this initial classifier, the classification accuracy is 90.45% for the data set. After the fuzzy rule base was optimized by QiQPSO, the number of fuzzy rules reduced to 10 with the classification accuracy unchanged.


#### 5.2.3. Results for Glass Data Set

Each sample in the Glass data set has nine features. However, after optimized by QiQPSO, the initial fuzzy classifier only needs five features, i.e.,x3, x4, x6, x8, and x9, and the feature space of each variable is partitioned by 2 Gaussian membership functions. Thus, the initial fuzzy classifier established by the algorithm has 25=32 fuzzy rules. Due to the space limitation, these fuzzy rules are not listed here. After the fuzzy rule base was optimized by QiQPSO, the improved classifier only needs 17 fuzzy rules. Comparing to the initial fuzzy classification system, the one with the fuzzy rule base optimized by QiQPSO can classify the sample data with an accuracy of 84.11% the same as the initial fuzzy classification system, but the optimized classifier only has 17 fuzzy rules, which means that the interpretability of the classifier is enhanced by QiQPSO.


#### 5.2.4. Results for Seeds Data Set

Each sample in the Seeds data set has 7 features. After feature selection by the QiQPSO algorithm, the dimensionality of the data is reduced to 5, onlyx1, x2, x4, x5, and x7 being selected as the input variables of the initial fuzzy classification system. The feature space of each input variable is partitioned by 3 Gaussian membership functions so that the initial fuzzy classifier has 35=243 fuzzy rules, which are not given here due to the space limitation. The fuzzy rule base was further optimized by QiQPSO, and the resulting fuzzy classifier only needs 12 rules.The classifiers classify the sample data with an accuracy of 84.86%, but the one with the fuzzy rule base optimized only has 12 rules, compared to the 243 rules in the initial system.


#### 5.2.5. Results for Wdbc Data Set

Each sample in the Wdbc data set has 64 features. However, after being optimized by QiQPSO, the initial fuzzy classifier only needs 10 features, and the feature space of each variable is partitioned by 3 Gaussian membership functions. Thus, the initial fuzzy classifier established by the algorithm has310=59049 fuzzy rules. After the fuzzy rule base was optimized by QiQPSO, the improved classifier only needs 97 fuzzy rules. It is shown that both classifiers can classify the sample data with an accuracy of 79.31%, but the optimized classifier only has 200 fuzzy rules, which means that the interpretability of the classifier is enhanced after it is the QiQPSO.


#### 5.2.6. Results for Satimage Data Set

Each sample in the Satimage data set has 36 features. However, after optimized by QiQPSO, the initial fuzzy classifier only needs 10 features, and the feature space of each variable is partitioned by 3 Gaussian membership functions. Thus, the initial fuzzy classifier established by the algorithm has310 fuzzy rules. After the fuzzy rule base was optimized by QiQPSO, only 153 fuzzy rules were selected for the final classification. The accuracy of these two algorithms is both 81.53%, but the optimized classifier only has 153 fuzzy rules, implying that the optimized classifier has higher interpretability than the original one.


#### 5.2.7. Performance Comparison with Other Methods

In this section, we first compared the proposed algorithm to C4.5, and then we used other optimization methods, namely, genetic algorithm (GA) and the binary PSO (BPSO) [37], to establish the initial fuzzy classifiers and further optimize the fuzzy rule bases for the six data sets and to make a performance comparison between these methods and QiQPSO. Just as the construction of the initial fuzzy classifiers with QiQPSO, we also used GA or BPSO to build the initial fuzzy classification systems based on the system directly established from the sample data.We all know that using particle swarm optimization to optimize a problem enhances the computational complexity. However, as shown in Table4, although the time complexity of the proposed algorithm is not better than C4.5, the proposed algorithm has a much higher classification accuracy than C4.5. Therefore, it is still acceptable although the time complexity is higher.Table 4
Comparison between the initial fuzzy classifier and the optimized one for the Iris data set.
Data set name
Fuzzy classification system (FCS)
The number of feature variables
The number of fuzzy sets
The number of fuzzy rules
The number of correctly classified samples
Classification accuracy (%)
Iris
Initial FCS
2
8
16
143
95.33
Optimized FCS
2
8
9
143
95.33
C4.5
2
8
9
138
92.00
Wine
Initial FCS
5
10
32
161
90.45
Optimized FCS
5
10
10
161
90.45
C4.5
5
10
10
159
89.32
Glass
Initial FCS
5
10
32
180
84.11
Optimized FCS
5
10
17
180
84.11
C4.5
5
10
17
171
81.42
Seeds
Initial FCS
5
15
243
174
82.86
Optimized FCS
5
15
12
174
82.86
C4.5
5
15
12
167
79.52
Wdbc
Initial FCS
10
20
1000
451
79.31
Optimized FCS
10
20
97
451
79.31
C4.5
10
20
97
436
76.62
Satimage
Initial FCS
10
20
1000
5246
81.53
Optimized FCS
10
20
153
5246
81.53
C4.5
10
20
153
4654
72.32The parameter settings for GA and BPSO are as follows. For GA, the crossover probability was set to be 0.5 and the mutation probability was 0.05. The population size was 50, and the maximum number of generations was also 100. For BPSO, the swarm size and the maximum number of iterations were set to be 20 and 100, respectively. The fitness functions for constructing the initial classifiers and optimizing the fuzzy rule bases were the same as those used in the experiments for QiQPSO. The experimental results are shown in Table5, where the results of the fuzzy classifier established directly from the sample data and further optimized by GA are also listed for comparison.Table 5
The comparison among the fuzzy classifiers generated by different methods for the four data sets.
Modeling method
Data set
The number of fuzzy rules
The number of the fuzzy rules after optimized
The number of correctly classified samples
Sample size
The classification accuracy (%)
FCS directly established from the samples
Iris
4
4
=
256
26
143
150
95.33
Wine
2
13
=
8192
70
170
178
95.51
Glass
2
9
=
512
118
188
214
87.85
Seeds
3
7
=
2187
34
191
210
90.95
Wdbc
3
6
=
729
102
441
569
77.51
Satimage
3
6
=
729
157
5117
6435
78.31
FCS established and optimized with GA
Iris
4
2
=
16
12
142
150
94.67
Wine
2
5
=
32
12
160
178
89.89
Glass
2
6
=
64
17
170
214
79.43
Seeds
3
6
=
729
15
183
210
87.14
Wdbc
3
6
=
729
108
440
569
77.43
Satimage
3
6
=
729
133
5061
6435
78.65
FCS established and optimized with BPSO
Iris
4
2
=
16
12
142
150
94.67
Wine
2
5
=
32
12
160
178
89.89
Glass
2
6
=
64
17
173
214
80.84
Seeds
3
6
=
729
12
185
210
88.10
Wdbc
3
6
=
729
104
418
569
73.53
Satimage
3
6
=
729
160
4932
6435
76.65
FCS established and optimized with QiQPSO
Iris
4
2
=
16
9
143
150
95.33
Wine
2
5
=
32
10
161
178
90.45
Glass
2
5
=
32
17
180
214
84.11
Seeds
3
5
=
243
12
190
210
90.47
Wdbc
3
10
=
59049
200
449
569
78.91
Satimage
3
10
=
59049
153
5189
6435
80.65For the Iris data set, the fuzzy classifier constructed directly from the sample data has 256 rules. After optimized by GA, the number of the rules decreases to 26 with a classification accuracy of 95.33%. The fuzzy classification system constructed by GA consists of 16 rules, and the final classifier improved by the algorithm only needs 12 rules that correctly classify 94.67% of the samples. The classifier constructed and further optimized by BPSO has the same number of the final fuzzy rules and the same classification accuracy as those by GA. QiQPSO generated the best classifier with the least number of rules and the highest classification accuracy (95.33%).For the Wine data set, the fuzzy classifier constructed directly from the sample data and optimized by GA has the highest classification accuracy (95.51%) but needs 70 fuzzy rules. QiQPSO obtained the final classifier with the second highest classification accuracy,, but only 10 fuzzy rules are needed to achieve this classification accuracy. GA and BPSO also showed the same performance in constructing and optimizing the fuzzy classifier for the data set, but their performance is obviously poorer than that of QiQPSO.For the Glass data set, the fuzzy classifier constructed directly from the samples and optimized by GA has the highest classification accuracy (87.85%), but it needs 118 fuzzy rules. Although the final fuzzy classifier based on QiQPSO obtained the second highest classification accuracy (84.11%), the number of its fuzzy rules is far smaller, meaning that the classifier has the highest interpretability. Between the GA-based and the BPSO-based classifiers, they have the same number of fuzzy rules, but the latter has higher classification accuracy.For the Wdbc data set, the fuzzy classification system constructed directly from the samples and improved by GA employs 200 fuzzy rules to obtain a classification of 79.31%. QiQPSO obtained the final classifier with the second highest classification accuracy (77.51%), which only needs 97 fuzzy rules. It is shown that BPSO had better performance than GA in constructing and optimizing the fuzzy classifier for this data set.For the Satimage data set, the fuzzy classification system constructed directly from the samples and improved by GA employs 150 fuzzy rules to obtain a classification of 81.53%. QiQPSO yielded the final classifier with the second highest classification accuracy (80.65%), which only needs 12 fuzzy rules. It is shown that BPSO had better performance than GA in constructing and optimizing the fuzzy classifier for this data set.For the seeds data set, the fuzzy classification system constructed directly from the samples and improved by GA employs 34 fuzzy rules to obtain a classification of 90.95%. QiQPSO obtained the final classifier with the second highest classification accuracy (90.40%), which only needs 153 fuzzy rules. It is shown that BPSO had better performance than GA in constructing and optimizing the fuzzy classifier for this data set.Overall, from Table5, we can see that the fuzzy classifiers constructed and optimized by QiQPSO have satisfactory classification accuracies, though not the best ones in general, and has the least number of fuzzy rules. This implies that QiQPSO can generate more interpretable fuzzy classifiers than its competitors with comparable or even better classification accuracies.


## 5.1. Data Sets and Experiment Settings

The experiments were carried out on various high-dimensional classification problems to establish and optimize fuzzy classifiers for these data sets by using the proposed QiQPSO algorithm, and Table1 summarizes the most important characteristics of these data sets. Iris data consist of 150 samples with four dimensions (namely, calyx length, calyx width, petal length, and petal width) and is divided into three classes, each of which is composed of 50 samples. The samples of the first and the second classes are separated completely, while the second and the third classes intersect. The Wine data set has 178 samples of chemical compositions of three different wines, each sample having 13 attributes. The Glass data set is composed of 9-dimensional 210 samples that can be classified into 6 categories. The Seeds data set consists of 210 data that can be divided into three categories and each sample has seven attributes. In order to verify the performance of the proposed algorithm on more complex real-world problems, Wdbc and Satimage are selected as testing data sets. These two data sets are both high-dimension real-world problems from the UCI repository [11] containing 569 and 6435 samples, respectively.Table 1
Summary of experiment data sets.
Data set name
Patterns
Features
Classes
Iris
150
4
3
Wine
178
13
3
Glass
210
9
6
Seeds
210
7
3
Wdbc
569
30
2
Satimage
6435
36
6In all the experiments, the maximum number of iterations that the QiQPSO algorithm executed for was set to be 100 and the swarm size wasL = 20. Note that these two parameters are set empirically according to the dimension of the optimization problems. That is, higher-dimensional problems generally needs larger maximum number of iterations and larger swarm size. The CE coefficient of the algorithm decreased linearly from 1.0 to 0.5. When constructing the initial fuzzy classifiers, the weights in the fitness function was set to be ω1=1, ω2=0.5, and ω3=2. The overlap factor for the membership functions was set as μ=0.2. When the fuzzy rule bases were optimized with QiQPSO, the weights in the fitness function were set as ω4=1 and ω5=2.


## 5.2. Experimental Results and Analysis

### 5.2.1. Results for Iris Data Set

For the Iris data set, we denote the four features, i.e., calyx length, calyx width, petal length, and petal width, asx1, x2, x3, and x4, respectively. The initial fuzzy classifier constructed by QiQPSO for this data is shown in Table 2. It can be seen that the initial classification system only uses two feature variables, namely, x3 and x4, both of which employ four Gaussian membership functions to divide the feature space as shown in Figure 2. Thus, the initial classifier has 16 fuzzy rules in total.Table 2
The initial fuzzy classification system constructed by QiQPSO for Iris.
RU
Inputx3
Inputx4
C
CF
Cen
L
R
Cen
L
R
1
1
0
0.9654
0.1
0
0.6548
1
1.0
2
1
0
0.9654
0.7
0.6548
0.6548
1
1.0
3
1
0
0.9654
1.3
0.6548
0.6548
2
0.6842
4
1
0
0.9654
2.4
0.6548
0
1
0.9385
5
2.5
0.9654
0.9654
0.1
0
0.6548
1
1.0
6
2.5
0.9654
0.9654
0.7
0.6548
0.6548
1
1.0
7
2.5
0.9654
0.9654
1.3
0.6548
0.6548
2
0.3335
8
2.5
0.9654
0.9654
2.4
0.6548
0
2
0.3424
9
4.0
0.9654
0.9654
0.1
0
0.6548
2
0.7506
10
4.0
0.9654
0.9654
0.7
0.6548
0.6548
2
1.0
11
4.0
0.9654
0.9654
1.3
0.6548
0.6548
2
0.8935
12
4.0
0.9654
0.9654
2.4
0.6548
0
3
1.0
13
5.9
0.9654
0
0.1
0
0.6548
0
NULL
14
5.9
0.9654
0
0.7
0.6548
0.6548
0
NULL
15
5.9
0.9654
0
1.3
0.6548
0.6548
3
0.7510
16
5.9
0.9654
0
2.4
0.6548
0
3
1.0
RU = rules, Cen = center,L = left width, R = right width, C = classification, and CF = confidence.Figure 2
Membership function of the Iris initial fuzzy classification system.From Table2, it is shown that the initial fuzzy classifier for the Iris data set contains two invalid fuzzy rules (i.e., R13 and R14), suggesting that there is no data under these two fuzzy rules. Using this fuzzy classifier, we can find that the number of the correctly classifier samples is 143 although there are intersections between the second and the third classes.The final fuzzy classifier further optimized by QiQPSO is presented in Table3, in which only 9 fuzzy rules are included. More specifically, the rules in the final fuzzy classifier are(30)R1:IFx3isA4,1andx4isB4,1THENxbelongs to Class1withCF=1.0R2:IFx3isA4,1andx4isB4,2⋯⋯R15:IFx3isA4,4andx4isB4,3THENxbelongs to Class3withCF=0.7510R16:IFx3isA4,4andx4isB4,4THENxbelongs to Class3withCF=1.0.Table 3
The final fuzzy classifier with the fuzzy rule base further optimized by QiQPSO for the Iris data set.
RU
Inputx3
Inputx4
C
CF
Cen
L
R
Cen
L
R
1
1
0
0.9654
0.1
0
0.6548
1
1.0
2
1
0
0.9654
0.7
0.6548
0.6548
1
1.0
5
2.5
0.9654
0.9654
0.1
0
0.6548
1
1.0
6
2.5
0.9654
0.9654
0.7
0.6548
0.6548
1
1.0
10
4.0
0.9654
0.9654
0.7
0.6548
0.6548
2
1.0
11
4.0
0.9654
0.9654
1.3
0.6548
0.6548
2
0.8935
12
4.0
0.9654
0.9654
2.4
0.6548
0
3
1.0
15
5.9
0.9654
0
1.3
0.6548
0.6548
3
0.7510
16
5.9
0.9654
0
2.4
0.6548
0
3
1.0
RU = rules, Cen = center,L = left width, R = right width, C = classification, and CF = confidence.The final fuzzy classifier further optimized by QiQPSO was then tested on the Iris data set. Although the classification accuracy remains the same, the number of the fuzzy rules reduced from 16 to 9 after the fuzzy rule base of the initial classifier was optimized. This means that after the rule base was optimized by QiQPSO, the redundant and invalid fuzzy rules were removed so that the interpretability of the classifier is improved.


### 5.2.2. Results for Wine Data Set

For the Wine data set, the 13 attributes (i.e., compositions) are alcoholx1, malic acid x2, ash contentx3, ash content alkalinityx4, magnesiumx5, total phenolx6, falconoidx7, phenolic x8, anthocyaninx9, color intensityx10, chromax11, OD280/OD315 dilute winex12, and amino acidx13. Among them, alcohol x1, flavonoidx7, chromax11, OD280/OD315 dilute winex12, and amino acidx13 are the input variables of the initial fuzzy classification system established by QiQPSO, and the feature space of each variable is partitioned by using Gaussian membership functions as shown in Figure 3.Figure 3
Membership functions of the initial fuzzy classification system for the Wine data set.For the wine data set, the initial fuzzy classifier consists of 32 fuzzy rules with five input variables. It can correctly classify 161 samples. After the fuzzy rule base was optimized by QiQPSO, the improved classifier contains only 10 fuzzy rules. More specifically, these rules are expressed as(31)R1:IFx1isA1,1andx7isA7,1andx11isA11,1andx12isA12,1andx13isA13,1THENxbelongs to Class3withCF=1.0R2:IFx1isA1,1andx7isA7,1andx11isA11,1andx12isA12,1andx13isA13,1THENxbelongs to Class3withCF=1.0⋯⋯R15:IFx1isA1,1andx7isA7,1andx11isA11,2andx12isA12,2andx13isA13,1THENxbelongs to Class1withCF=0.9900R16:IFx1isA1,2andx7isA7,1andx11isA11,1andx12isA12,1andx13isA13,1THENxbelongs to Class1withCF=0.9977.The comparison is between the initial fuzzy classification system and the one improved by QiQPSO. It is shown that the initial classification system established by QiQPSO has 5 input variables, 10 fuzzy sets for the antecedents, and 32 fuzzy rules. With this initial classifier, the classification accuracy is 90.45% for the data set. After the fuzzy rule base was optimized by QiQPSO, the number of fuzzy rules reduced to 10 with the classification accuracy unchanged.


### 5.2.3. Results for Glass Data Set

Each sample in the Glass data set has nine features. However, after optimized by QiQPSO, the initial fuzzy classifier only needs five features, i.e.,x3, x4, x6, x8, and x9, and the feature space of each variable is partitioned by 2 Gaussian membership functions. Thus, the initial fuzzy classifier established by the algorithm has 25=32 fuzzy rules. Due to the space limitation, these fuzzy rules are not listed here. After the fuzzy rule base was optimized by QiQPSO, the improved classifier only needs 17 fuzzy rules. Comparing to the initial fuzzy classification system, the one with the fuzzy rule base optimized by QiQPSO can classify the sample data with an accuracy of 84.11% the same as the initial fuzzy classification system, but the optimized classifier only has 17 fuzzy rules, which means that the interpretability of the classifier is enhanced by QiQPSO.


### 5.2.4. Results for Seeds Data Set

Each sample in the Seeds data set has 7 features. After feature selection by the QiQPSO algorithm, the dimensionality of the data is reduced to 5, onlyx1, x2, x4, x5, and x7 being selected as the input variables of the initial fuzzy classification system. The feature space of each input variable is partitioned by 3 Gaussian membership functions so that the initial fuzzy classifier has 35=243 fuzzy rules, which are not given here due to the space limitation. The fuzzy rule base was further optimized by QiQPSO, and the resulting fuzzy classifier only needs 12 rules.The classifiers classify the sample data with an accuracy of 84.86%, but the one with the fuzzy rule base optimized only has 12 rules, compared to the 243 rules in the initial system.


### 5.2.5. Results for Wdbc Data Set

Each sample in the Wdbc data set has 64 features. However, after being optimized by QiQPSO, the initial fuzzy classifier only needs 10 features, and the feature space of each variable is partitioned by 3 Gaussian membership functions. Thus, the initial fuzzy classifier established by the algorithm has310=59049 fuzzy rules. After the fuzzy rule base was optimized by QiQPSO, the improved classifier only needs 97 fuzzy rules. It is shown that both classifiers can classify the sample data with an accuracy of 79.31%, but the optimized classifier only has 200 fuzzy rules, which means that the interpretability of the classifier is enhanced after it is the QiQPSO.


### 5.2.6. Results for Satimage Data Set

Each sample in the Satimage data set has 36 features. However, after optimized by QiQPSO, the initial fuzzy classifier only needs 10 features, and the feature space of each variable is partitioned by 3 Gaussian membership functions. Thus, the initial fuzzy classifier established by the algorithm has310 fuzzy rules. After the fuzzy rule base was optimized by QiQPSO, only 153 fuzzy rules were selected for the final classification. The accuracy of these two algorithms is both 81.53%, but the optimized classifier only has 153 fuzzy rules, implying that the optimized classifier has higher interpretability than the original one.


### 5.2.7. Performance Comparison with Other Methods

In this section, we first compared the proposed algorithm to C4.5, and then we used other optimization methods, namely, genetic algorithm (GA) and the binary PSO (BPSO) [37], to establish the initial fuzzy classifiers and further optimize the fuzzy rule bases for the six data sets and to make a performance comparison between these methods and QiQPSO. Just as the construction of the initial fuzzy classifiers with QiQPSO, we also used GA or BPSO to build the initial fuzzy classification systems based on the system directly established from the sample data.We all know that using particle swarm optimization to optimize a problem enhances the computational complexity. However, as shown in Table4, although the time complexity of the proposed algorithm is not better than C4.5, the proposed algorithm has a much higher classification accuracy than C4.5. Therefore, it is still acceptable although the time complexity is higher.Table 4
Comparison between the initial fuzzy classifier and the optimized one for the Iris data set.
Data set name
Fuzzy classification system (FCS)
The number of feature variables
The number of fuzzy sets
The number of fuzzy rules
The number of correctly classified samples
Classification accuracy (%)
Iris
Initial FCS
2
8
16
143
95.33
Optimized FCS
2
8
9
143
95.33
C4.5
2
8
9
138
92.00
Wine
Initial FCS
5
10
32
161
90.45
Optimized FCS
5
10
10
161
90.45
C4.5
5
10
10
159
89.32
Glass
Initial FCS
5
10
32
180
84.11
Optimized FCS
5
10
17
180
84.11
C4.5
5
10
17
171
81.42
Seeds
Initial FCS
5
15
243
174
82.86
Optimized FCS
5
15
12
174
82.86
C4.5
5
15
12
167
79.52
Wdbc
Initial FCS
10
20
1000
451
79.31
Optimized FCS
10
20
97
451
79.31
C4.5
10
20
97
436
76.62
Satimage
Initial FCS
10
20
1000
5246
81.53
Optimized FCS
10
20
153
5246
81.53
C4.5
10
20
153
4654
72.32The parameter settings for GA and BPSO are as follows. For GA, the crossover probability was set to be 0.5 and the mutation probability was 0.05. The population size was 50, and the maximum number of generations was also 100. For BPSO, the swarm size and the maximum number of iterations were set to be 20 and 100, respectively. The fitness functions for constructing the initial classifiers and optimizing the fuzzy rule bases were the same as those used in the experiments for QiQPSO. The experimental results are shown in Table5, where the results of the fuzzy classifier established directly from the sample data and further optimized by GA are also listed for comparison.Table 5
The comparison among the fuzzy classifiers generated by different methods for the four data sets.
Modeling method
Data set
The number of fuzzy rules
The number of the fuzzy rules after optimized
The number of correctly classified samples
Sample size
The classification accuracy (%)
FCS directly established from the samples
Iris
4
4
=
256
26
143
150
95.33
Wine
2
13
=
8192
70
170
178
95.51
Glass
2
9
=
512
118
188
214
87.85
Seeds
3
7
=
2187
34
191
210
90.95
Wdbc
3
6
=
729
102
441
569
77.51
Satimage
3
6
=
729
157
5117
6435
78.31
FCS established and optimized with GA
Iris
4
2
=
16
12
142
150
94.67
Wine
2
5
=
32
12
160
178
89.89
Glass
2
6
=
64
17
170
214
79.43
Seeds
3
6
=
729
15
183
210
87.14
Wdbc
3
6
=
729
108
440
569
77.43
Satimage
3
6
=
729
133
5061
6435
78.65
FCS established and optimized with BPSO
Iris
4
2
=
16
12
142
150
94.67
Wine
2
5
=
32
12
160
178
89.89
Glass
2
6
=
64
17
173
214
80.84
Seeds
3
6
=
729
12
185
210
88.10
Wdbc
3
6
=
729
104
418
569
73.53
Satimage
3
6
=
729
160
4932
6435
76.65
FCS established and optimized with QiQPSO
Iris
4
2
=
16
9
143
150
95.33
Wine
2
5
=
32
10
161
178
90.45
Glass
2
5
=
32
17
180
214
84.11
Seeds
3
5
=
243
12
190
210
90.47
Wdbc
3
10
=
59049
200
449
569
78.91
Satimage
3
10
=
59049
153
5189
6435
80.65For the Iris data set, the fuzzy classifier constructed directly from the sample data has 256 rules. After optimized by GA, the number of the rules decreases to 26 with a classification accuracy of 95.33%. The fuzzy classification system constructed by GA consists of 16 rules, and the final classifier improved by the algorithm only needs 12 rules that correctly classify 94.67% of the samples. The classifier constructed and further optimized by BPSO has the same number of the final fuzzy rules and the same classification accuracy as those by GA. QiQPSO generated the best classifier with the least number of rules and the highest classification accuracy (95.33%).For the Wine data set, the fuzzy classifier constructed directly from the sample data and optimized by GA has the highest classification accuracy (95.51%) but needs 70 fuzzy rules. QiQPSO obtained the final classifier with the second highest classification accuracy,, but only 10 fuzzy rules are needed to achieve this classification accuracy. GA and BPSO also showed the same performance in constructing and optimizing the fuzzy classifier for the data set, but their performance is obviously poorer than that of QiQPSO.For the Glass data set, the fuzzy classifier constructed directly from the samples and optimized by GA has the highest classification accuracy (87.85%), but it needs 118 fuzzy rules. Although the final fuzzy classifier based on QiQPSO obtained the second highest classification accuracy (84.11%), the number of its fuzzy rules is far smaller, meaning that the classifier has the highest interpretability. Between the GA-based and the BPSO-based classifiers, they have the same number of fuzzy rules, but the latter has higher classification accuracy.For the Wdbc data set, the fuzzy classification system constructed directly from the samples and improved by GA employs 200 fuzzy rules to obtain a classification of 79.31%. QiQPSO obtained the final classifier with the second highest classification accuracy (77.51%), which only needs 97 fuzzy rules. It is shown that BPSO had better performance than GA in constructing and optimizing the fuzzy classifier for this data set.For the Satimage data set, the fuzzy classification system constructed directly from the samples and improved by GA employs 150 fuzzy rules to obtain a classification of 81.53%. QiQPSO yielded the final classifier with the second highest classification accuracy (80.65%), which only needs 12 fuzzy rules. It is shown that BPSO had better performance than GA in constructing and optimizing the fuzzy classifier for this data set.For the seeds data set, the fuzzy classification system constructed directly from the samples and improved by GA employs 34 fuzzy rules to obtain a classification of 90.95%. QiQPSO obtained the final classifier with the second highest classification accuracy (90.40%), which only needs 153 fuzzy rules. It is shown that BPSO had better performance than GA in constructing and optimizing the fuzzy classifier for this data set.Overall, from Table5, we can see that the fuzzy classifiers constructed and optimized by QiQPSO have satisfactory classification accuracies, though not the best ones in general, and has the least number of fuzzy rules. This implies that QiQPSO can generate more interpretable fuzzy classifiers than its competitors with comparable or even better classification accuracies.


## 5.2.1. Results for Iris Data Set

For the Iris data set, we denote the four features, i.e., calyx length, calyx width, petal length, and petal width, asx1, x2, x3, and x4, respectively. The initial fuzzy classifier constructed by QiQPSO for this data is shown in Table 2. It can be seen that the initial classification system only uses two feature variables, namely, x3 and x4, both of which employ four Gaussian membership functions to divide the feature space as shown in Figure 2. Thus, the initial classifier has 16 fuzzy rules in total.Table 2
The initial fuzzy classification system constructed by QiQPSO for Iris.
RU
Inputx3
Inputx4
C
CF
Cen
L
R
Cen
L
R
1
1
0
0.9654
0.1
0
0.6548
1
1.0
2
1
0
0.9654
0.7
0.6548
0.6548
1
1.0
3
1
0
0.9654
1.3
0.6548
0.6548
2
0.6842
4
1
0
0.9654
2.4
0.6548
0
1
0.9385
5
2.5
0.9654
0.9654
0.1
0
0.6548
1
1.0
6
2.5
0.9654
0.9654
0.7
0.6548
0.6548
1
1.0
7
2.5
0.9654
0.9654
1.3
0.6548
0.6548
2
0.3335
8
2.5
0.9654
0.9654
2.4
0.6548
0
2
0.3424
9
4.0
0.9654
0.9654
0.1
0
0.6548
2
0.7506
10
4.0
0.9654
0.9654
0.7
0.6548
0.6548
2
1.0
11
4.0
0.9654
0.9654
1.3
0.6548
0.6548
2
0.8935
12
4.0
0.9654
0.9654
2.4
0.6548
0
3
1.0
13
5.9
0.9654
0
0.1
0
0.6548
0
NULL
14
5.9
0.9654
0
0.7
0.6548
0.6548
0
NULL
15
5.9
0.9654
0
1.3
0.6548
0.6548
3
0.7510
16
5.9
0.9654
0
2.4
0.6548
0
3
1.0
RU = rules, Cen = center,L = left width, R = right width, C = classification, and CF = confidence.Figure 2
Membership function of the Iris initial fuzzy classification system.From Table2, it is shown that the initial fuzzy classifier for the Iris data set contains two invalid fuzzy rules (i.e., R13 and R14), suggesting that there is no data under these two fuzzy rules. Using this fuzzy classifier, we can find that the number of the correctly classifier samples is 143 although there are intersections between the second and the third classes.The final fuzzy classifier further optimized by QiQPSO is presented in Table3, in which only 9 fuzzy rules are included. More specifically, the rules in the final fuzzy classifier are(30)R1:IFx3isA4,1andx4isB4,1THENxbelongs to Class1withCF=1.0R2:IFx3isA4,1andx4isB4,2⋯⋯R15:IFx3isA4,4andx4isB4,3THENxbelongs to Class3withCF=0.7510R16:IFx3isA4,4andx4isB4,4THENxbelongs to Class3withCF=1.0.Table 3
The final fuzzy classifier with the fuzzy rule base further optimized by QiQPSO for the Iris data set.
RU
Inputx3
Inputx4
C
CF
Cen
L
R
Cen
L
R
1
1
0
0.9654
0.1
0
0.6548
1
1.0
2
1
0
0.9654
0.7
0.6548
0.6548
1
1.0
5
2.5
0.9654
0.9654
0.1
0
0.6548
1
1.0
6
2.5
0.9654
0.9654
0.7
0.6548
0.6548
1
1.0
10
4.0
0.9654
0.9654
0.7
0.6548
0.6548
2
1.0
11
4.0
0.9654
0.9654
1.3
0.6548
0.6548
2
0.8935
12
4.0
0.9654
0.9654
2.4
0.6548
0
3
1.0
15
5.9
0.9654
0
1.3
0.6548
0.6548
3
0.7510
16
5.9
0.9654
0
2.4
0.6548
0
3
1.0
RU = rules, Cen = center,L = left width, R = right width, C = classification, and CF = confidence.The final fuzzy classifier further optimized by QiQPSO was then tested on the Iris data set. Although the classification accuracy remains the same, the number of the fuzzy rules reduced from 16 to 9 after the fuzzy rule base of the initial classifier was optimized. This means that after the rule base was optimized by QiQPSO, the redundant and invalid fuzzy rules were removed so that the interpretability of the classifier is improved.


## 5.2.2. Results for Wine Data Set

For the Wine data set, the 13 attributes (i.e., compositions) are alcoholx1, malic acid x2, ash contentx3, ash content alkalinityx4, magnesiumx5, total phenolx6, falconoidx7, phenolic x8, anthocyaninx9, color intensityx10, chromax11, OD280/OD315 dilute winex12, and amino acidx13. Among them, alcohol x1, flavonoidx7, chromax11, OD280/OD315 dilute winex12, and amino acidx13 are the input variables of the initial fuzzy classification system established by QiQPSO, and the feature space of each variable is partitioned by using Gaussian membership functions as shown in Figure 3.Figure 3
Membership functions of the initial fuzzy classification system for the Wine data set.For the wine data set, the initial fuzzy classifier consists of 32 fuzzy rules with five input variables. It can correctly classify 161 samples. After the fuzzy rule base was optimized by QiQPSO, the improved classifier contains only 10 fuzzy rules. More specifically, these rules are expressed as(31)R1:IFx1isA1,1andx7isA7,1andx11isA11,1andx12isA12,1andx13isA13,1THENxbelongs to Class3withCF=1.0R2:IFx1isA1,1andx7isA7,1andx11isA11,1andx12isA12,1andx13isA13,1THENxbelongs to Class3withCF=1.0⋯⋯R15:IFx1isA1,1andx7isA7,1andx11isA11,2andx12isA12,2andx13isA13,1THENxbelongs to Class1withCF=0.9900R16:IFx1isA1,2andx7isA7,1andx11isA11,1andx12isA12,1andx13isA13,1THENxbelongs to Class1withCF=0.9977.The comparison is between the initial fuzzy classification system and the one improved by QiQPSO. It is shown that the initial classification system established by QiQPSO has 5 input variables, 10 fuzzy sets for the antecedents, and 32 fuzzy rules. With this initial classifier, the classification accuracy is 90.45% for the data set. After the fuzzy rule base was optimized by QiQPSO, the number of fuzzy rules reduced to 10 with the classification accuracy unchanged.


## 5.2.3. Results for Glass Data Set

Each sample in the Glass data set has nine features. However, after optimized by QiQPSO, the initial fuzzy classifier only needs five features, i.e.,x3, x4, x6, x8, and x9, and the feature space of each variable is partitioned by 2 Gaussian membership functions. Thus, the initial fuzzy classifier established by the algorithm has 25=32 fuzzy rules. Due to the space limitation, these fuzzy rules are not listed here. After the fuzzy rule base was optimized by QiQPSO, the improved classifier only needs 17 fuzzy rules. Comparing to the initial fuzzy classification system, the one with the fuzzy rule base optimized by QiQPSO can classify the sample data with an accuracy of 84.11% the same as the initial fuzzy classification system, but the optimized classifier only has 17 fuzzy rules, which means that the interpretability of the classifier is enhanced by QiQPSO.


## 5.2.4. Results for Seeds Data Set

Each sample in the Seeds data set has 7 features. After feature selection by the QiQPSO algorithm, the dimensionality of the data is reduced to 5, onlyx1, x2, x4, x5, and x7 being selected as the input variables of the initial fuzzy classification system. The feature space of each input variable is partitioned by 3 Gaussian membership functions so that the initial fuzzy classifier has 35=243 fuzzy rules, which are not given here due to the space limitation. The fuzzy rule base was further optimized by QiQPSO, and the resulting fuzzy classifier only needs 12 rules.The classifiers classify the sample data with an accuracy of 84.86%, but the one with the fuzzy rule base optimized only has 12 rules, compared to the 243 rules in the initial system.


## 5.2.5. Results for Wdbc Data Set

Each sample in the Wdbc data set has 64 features. However, after being optimized by QiQPSO, the initial fuzzy classifier only needs 10 features, and the feature space of each variable is partitioned by 3 Gaussian membership functions. Thus, the initial fuzzy classifier established by the algorithm has310=59049 fuzzy rules. After the fuzzy rule base was optimized by QiQPSO, the improved classifier only needs 97 fuzzy rules. It is shown that both classifiers can classify the sample data with an accuracy of 79.31%, but the optimized classifier only has 200 fuzzy rules, which means that the interpretability of the classifier is enhanced after it is the QiQPSO.


## 5.2.6. Results for Satimage Data Set

Each sample in the Satimage data set has 36 features. However, after optimized by QiQPSO, the initial fuzzy classifier only needs 10 features, and the feature space of each variable is partitioned by 3 Gaussian membership functions. Thus, the initial fuzzy classifier established by the algorithm has310 fuzzy rules. After the fuzzy rule base was optimized by QiQPSO, only 153 fuzzy rules were selected for the final classification. The accuracy of these two algorithms is both 81.53%, but the optimized classifier only has 153 fuzzy rules, implying that the optimized classifier has higher interpretability than the original one.


## 5.2.7. Performance Comparison with Other Methods

In this section, we first compared the proposed algorithm to C4.5, and then we used other optimization methods, namely, genetic algorithm (GA) and the binary PSO (BPSO) [37], to establish the initial fuzzy classifiers and further optimize the fuzzy rule bases for the six data sets and to make a performance comparison between these methods and QiQPSO. Just as the construction of the initial fuzzy classifiers with QiQPSO, we also used GA or BPSO to build the initial fuzzy classification systems based on the system directly established from the sample data.We all know that using particle swarm optimization to optimize a problem enhances the computational complexity. However, as shown in Table4, although the time complexity of the proposed algorithm is not better than C4.5, the proposed algorithm has a much higher classification accuracy than C4.5. Therefore, it is still acceptable although the time complexity is higher.Table 4
Comparison between the initial fuzzy classifier and the optimized one for the Iris data set.
Data set name
Fuzzy classification system (FCS)
The number of feature variables
The number of fuzzy sets
The number of fuzzy rules
The number of correctly classified samples
Classification accuracy (%)
Iris
Initial FCS
2
8
16
143
95.33
Optimized FCS
2
8
9
143
95.33
C4.5
2
8
9
138
92.00
Wine
Initial FCS
5
10
32
161
90.45
Optimized FCS
5
10
10
161
90.45
C4.5
5
10
10
159
89.32
Glass
Initial FCS
5
10
32
180
84.11
Optimized FCS
5
10
17
180
84.11
C4.5
5
10
17
171
81.42
Seeds
Initial FCS
5
15
243
174
82.86
Optimized FCS
5
15
12
174
82.86
C4.5
5
15
12
167
79.52
Wdbc
Initial FCS
10
20
1000
451
79.31
Optimized FCS
10
20
97
451
79.31
C4.5
10
20
97
436
76.62
Satimage
Initial FCS
10
20
1000
5246
81.53
Optimized FCS
10
20
153
5246
81.53
C4.5
10
20
153
4654
72.32The parameter settings for GA and BPSO are as follows. For GA, the crossover probability was set to be 0.5 and the mutation probability was 0.05. The population size was 50, and the maximum number of generations was also 100. For BPSO, the swarm size and the maximum number of iterations were set to be 20 and 100, respectively. The fitness functions for constructing the initial classifiers and optimizing the fuzzy rule bases were the same as those used in the experiments for QiQPSO. The experimental results are shown in Table5, where the results of the fuzzy classifier established directly from the sample data and further optimized by GA are also listed for comparison.Table 5
The comparison among the fuzzy classifiers generated by different methods for the four data sets.
Modeling method
Data set
The number of fuzzy rules
The number of the fuzzy rules after optimized
The number of correctly classified samples
Sample size
The classification accuracy (%)
FCS directly established from the samples
Iris
4
4
=
256
26
143
150
95.33
Wine
2
13
=
8192
70
170
178
95.51
Glass
2
9
=
512
118
188
214
87.85
Seeds
3
7
=
2187
34
191
210
90.95
Wdbc
3
6
=
729
102
441
569
77.51
Satimage
3
6
=
729
157
5117
6435
78.31
FCS established and optimized with GA
Iris
4
2
=
16
12
142
150
94.67
Wine
2
5
=
32
12
160
178
89.89
Glass
2
6
=
64
17
170
214
79.43
Seeds
3
6
=
729
15
183
210
87.14
Wdbc
3
6
=
729
108
440
569
77.43
Satimage
3
6
=
729
133
5061
6435
78.65
FCS established and optimized with BPSO
Iris
4
2
=
16
12
142
150
94.67
Wine
2
5
=
32
12
160
178
89.89
Glass
2
6
=
64
17
173
214
80.84
Seeds
3
6
=
729
12
185
210
88.10
Wdbc
3
6
=
729
104
418
569
73.53
Satimage
3
6
=
729
160
4932
6435
76.65
FCS established and optimized with QiQPSO
Iris
4
2
=
16
9
143
150
95.33
Wine
2
5
=
32
10
161
178
90.45
Glass
2
5
=
32
17
180
214
84.11
Seeds
3
5
=
243
12
190
210
90.47
Wdbc
3
10
=
59049
200
449
569
78.91
Satimage
3
10
=
59049
153
5189
6435
80.65For the Iris data set, the fuzzy classifier constructed directly from the sample data has 256 rules. After optimized by GA, the number of the rules decreases to 26 with a classification accuracy of 95.33%. The fuzzy classification system constructed by GA consists of 16 rules, and the final classifier improved by the algorithm only needs 12 rules that correctly classify 94.67% of the samples. The classifier constructed and further optimized by BPSO has the same number of the final fuzzy rules and the same classification accuracy as those by GA. QiQPSO generated the best classifier with the least number of rules and the highest classification accuracy (95.33%).For the Wine data set, the fuzzy classifier constructed directly from the sample data and optimized by GA has the highest classification accuracy (95.51%) but needs 70 fuzzy rules. QiQPSO obtained the final classifier with the second highest classification accuracy,, but only 10 fuzzy rules are needed to achieve this classification accuracy. GA and BPSO also showed the same performance in constructing and optimizing the fuzzy classifier for the data set, but their performance is obviously poorer than that of QiQPSO.For the Glass data set, the fuzzy classifier constructed directly from the samples and optimized by GA has the highest classification accuracy (87.85%), but it needs 118 fuzzy rules. Although the final fuzzy classifier based on QiQPSO obtained the second highest classification accuracy (84.11%), the number of its fuzzy rules is far smaller, meaning that the classifier has the highest interpretability. Between the GA-based and the BPSO-based classifiers, they have the same number of fuzzy rules, but the latter has higher classification accuracy.For the Wdbc data set, the fuzzy classification system constructed directly from the samples and improved by GA employs 200 fuzzy rules to obtain a classification of 79.31%. QiQPSO obtained the final classifier with the second highest classification accuracy (77.51%), which only needs 97 fuzzy rules. It is shown that BPSO had better performance than GA in constructing and optimizing the fuzzy classifier for this data set.For the Satimage data set, the fuzzy classification system constructed directly from the samples and improved by GA employs 150 fuzzy rules to obtain a classification of 81.53%. QiQPSO yielded the final classifier with the second highest classification accuracy (80.65%), which only needs 12 fuzzy rules. It is shown that BPSO had better performance than GA in constructing and optimizing the fuzzy classifier for this data set.For the seeds data set, the fuzzy classification system constructed directly from the samples and improved by GA employs 34 fuzzy rules to obtain a classification of 90.95%. QiQPSO obtained the final classifier with the second highest classification accuracy (90.40%), which only needs 153 fuzzy rules. It is shown that BPSO had better performance than GA in constructing and optimizing the fuzzy classifier for this data set.Overall, from Table5, we can see that the fuzzy classifiers constructed and optimized by QiQPSO have satisfactory classification accuracies, though not the best ones in general, and has the least number of fuzzy rules. This implies that QiQPSO can generate more interpretable fuzzy classifiers than its competitors with comparable or even better classification accuracies.


## 6. Conclusion

In this paper, we first proposed the quantum-inspired QPSO (QiQPSO) and then applied the algorithm in optimizing fuzzy rule-based classifiers. Since the original QPSO was developed for continuous optimization problems, we incorporate the quantum computing principles into the QPSO to discretize the particle position and the corresponding operations. In QiQPSO, each particle position is encoded as a string of qubits, the quantum angle of each qubit is updated in line with the equation of QPSO, and a rotation gate is used to update the probability of the qubit according to which the qubit is collapsed to “0” or “1.” When QiQPSO is employed to optimize a fuzzy rule-based classifier, the algorithm is first used to construct the initial fuzzy classifier based on the sample data and the grid method of partitioning the feature space. The objectives in construction of the initial fuzzy classification system are to reduce the number of feature variables, minimize the number of fuzzy rules, and maximize the classification accuracy. After the initial classifier is established, its fuzzy rule base is further optimized by QiQPSO in order to minimize the number of fuzzy rules with the classification accuracy unchanged. The proposed method was finally tested on six well-known data sets to construct and optimize the corresponding fuzzy classifiers. The experimental results showed that the QiQPSO can effectively construct the initial fuzzy classifier and optimize its fuzzy rule base. It is also shown that the fuzzy rule-based classifiers optimized by QiQPSO have higher interpretabilities than those optimized by other methods, with comparable and even better classification accuracies.



---

*Source: 9319364-2020-04-27.xml*