Fuzzy Learning Machine
Anonymous Author(s)
AfﬁliationAddress email
Abstract
Classiﬁcation is one of the most important problems in machine learning and thenature of it is concept cognition.
So far, dozens of different classiﬁers have beendesigned.
Although their working mechanisms vary widely, few of them fullyconsider concept cognition.
In this paper, a new learning machine, fuzzy learningmachine (FLM), is proposed from the perspective of concept cognition.
Inspiredby cognitive science, its working mechanism is of strong interpretability.
At thesame time, FLM roots in set theory and fuzzy set theory, so FLM has a solidmathematical foundation.
The systematic experimental results on a large number ofdata sets show that FLM can achieve excellent performance, even with the simpleimplementation.
Introduction
Given a input space X , a output space (a ﬁnite set) Y , and an unknown function ϕ : X → Y , the goal 12 of classiﬁcation is ﬁnding the ϕ or an approximation to it.
Classiﬁcation is one of the most importantproblems in machine learning, dating back to the origins of machine learning.
In classiﬁcation, eachmember of the output space Y corresponds to a concept.
In essence, the process of classiﬁcationis the process of concept cognition.
Concept contains our knowledge about the world, and we useconcept to understand and organize the world.
Without it, there will be no human intelligence at all.
That is why classiﬁcation plays an important role in machine learning even in artiﬁcial intelligence.
Over the past decades, a large number of classiﬁers have been designed and achieved success ondifferent tasks.
The literature [1] systematically summarizes 179 classiﬁers from 17 different families,which contains almost all existing classiﬁers.
Usually, generalization performance is regarded asan important dimension for evaluating classiﬁer.
With the increasing depth and breadth of machinelearning applications, classiﬁer is expected to process more and more good properties, such asinterpretability [2, 3, 4], robustness [5, 6, 7], and so on.
The existing classiﬁers follow different design principles (see Appendix A.2.1 for detailed analysis),but they are designed without full consideration for the nature of classiﬁcation and how human do it.
For a speciﬁc scenario, the existing classiﬁers often perform well on one evaluation dimension butpoorly on others.
For example, in visual recognition tasks, the performance of deep neural networkshave reached even exceeded human level [8].
Unfortunately, its working mechanism is difﬁcult to beunderstood due to the complexity of the network structure [9, 10], and it is also very vulnerable to beattacked [11, 12].
Instead, human can perform well on a variety of tasks and on different evaluationdimensions.
To implement a human-like classiﬁer, a direct and efﬁcient approach is simulating the process ofhuman classiﬁcation.
As mentioned above, classiﬁcation process is essentially the process of conceptcognition.
In cognitive science, there is a lot of valuable theories about how human learn, representand use concept [13].
However, few of the existing classiﬁers are designed based on these theories,which leads to that the existing classiﬁers are unable to take into account the performance on multiple evaluation dimensions.
Inspires by this, we are encouraged to design a new classiﬁer based onconcept cognition, which can simultaneously perform well on different evaluation dimensions.
Inthis paper, a new learning machine is designed based on the following principles.
Similarity is the basis of concept representation.
Similarity is a fundamental concept withincognitive science and plays a crucial role in the process of human classiﬁcation [14].
Concept isrepresented based on similarity for children, which is also a basic choice for adults [15].
Based onthis principle, we prove the equivalence between classiﬁcation and equivalence relation (ER) viabinary relation in set theory.
Considering that ER is a special similarity, we actually demonstrate thatall classiﬁcation problems can be solved based on similarity (see Section 2.1).
Concept is fuzzy.
An important conclusion in cognitive science is that fuzziness is the intrinsicproperty of concept [13].
The fuzziness and the randomness are the two most important sources ofthe uncertainty.
In machine learning, probability theory is often used for dealing with the uncertainty,such as Bayesian methods [16, 17], label distribution learning [18], etc.
It is sufﬁcient for probabilitytheory to deal with the randomness rather than the fuzziness [19].
However, the intrinsic property ofconcept is just the fuzziness rather than the randomness.
An intuitive example is shown in Figure 1.
Concept is represented based on exemplar theory.
In cognitive science, there are several theoriesof concept representation [13].
Exemplar theory is chosen to solve concept representation problem,in this paper.
Because it not only has a good self-consistency in cognitive science, but also it can benaturally integrated with the data-driven machine learning paradigm (see Section 2.3).
To summarize, the main contributions of this paper are four-folds as below:
(1) This paper revisits classiﬁcation problem from the perspective of concept cognition, and provesthe equivalence between classiﬁcation problem and ER.
Furthermore, considering that concept isintrinsically fuzzy, the classiﬁcation problem is modeled as a FER problem.(2) A new and general learning machine FLM is proposed based on concept cognition for solvingFER problem.(3)
A new fuzziness permissible loss is designed.
Given the new loss, this paper prove that FER canbe approximated effectively by fuzzy similarity relation (FSR).
And then a neural network basedFLM (NN-FLM) is designed.(4) Extensive experimental results demonstrate the superiority of NN-FLM on different evaluationdimensions including interpretability, robustness, and generalization performance.
Fuzzy Learning Machine
Basic notations Let X and Y = { c 1 , c 2 , · · · , c k } be the input and output spaces, respectively.
Let 71 ϕ :
X → Y be the unknown target function and ∀ x ∈ X , ϕ
( x ) ∈ Y is the true class label of 72 the sample x .
The goal of classiﬁcation is to approximate ϕ as closely as possible.
Let D train =
( x i , y i )
| x i ∈ X , y i = ϕ
( x i ) ∈ Y , i = 1 , 2 , · · · , n } and D test = { x j | x j ∈ X , j = n + 1 , n + 74 2 , · · · , n + m } be the training and test data sets, respectively.
2.1 Classiﬁcation Problem Can Be
Solved Based on
Similarity
Researches [14, 15] on cognitive science prove that similarity is the basis of concept representation.
However, the relationship between similarity and classiﬁcation (e.g. whether all classiﬁcationproblems can be solved by using similarity?) still lacks a mathematical justiﬁcation.
This sectionwill accomplish it.
The mathematical tool used in this section is binary relation in set theory.
See
Appendix A.1.2 for a brief introduction.
First, the formal deﬁnition of classiﬁcation problem is given.
Deﬁnition
For a input space X , a output space Y = { c 1 , c 2 , · · · , c k } , and an unknown target 83 function ϕ .
The (
X , Y , ϕ ) -classiﬁcation problem is deﬁned as: ∀ x ∈ X , ﬁnding the ϕ ( x ) ∈ Y .
It should be pointed out that the Deﬁnition 1 is so general that almost all classiﬁcation problemsstudied in machine learning can be represented by it.
Deﬁnition
Given a ( X , Y , ϕ ) -classiﬁcation problem, (1) let X ×
X be the new input space, (2)let { 0 , 1 } be the new output space, (3) let ϕ † : X × X → { 0 , 1 } be the new target function, where∀ ( x i ,
x j ) ∈
X × X , ϕ † (( x i , x j ))
=
I ( ϕ ( x i ) =
ϕ
( x j )) , then ﬁnding the ϕ † is deﬁned as its adjoint(cid:0)
X × X , { 0 , 1 } , ϕ † (cid:1) -classiﬁcation problem.
Obviously, the ϕ † is an equivalence relation (ER) on X (see Deﬁnition 7 in Appendix A.1.2) and theadjoint (cid:0)
X × X , { 0 , 1 } , ϕ † (cid:1) -classiﬁcation problem is a binary-classiﬁcation problem.
Based on the above deﬁnitions, we have the following proposition.
Proposition 1 The ( X , Y , ϕ ) -classiﬁcation problem is equivalent to its adjoint (cid:0)
X × X , { 0 , 1 } , ϕ † (cid:1) -classiﬁcation problem, i.e. if one problem is solved, then the other problem will also be solved.
(See
Appendix A.3.1 for proof)
The Proposition 1 establishes the relationship between ER and classiﬁcation problem.
Forany ( X , Y , ϕ ) -classiﬁcation problem, we can always solve it equivalently by solving its adjoint(cid:0)
X × X , { 0 , 1 } , ϕ † (cid:1) -classiﬁcation problem.
Because ER ϕ † : X ×X → { 0 , 1 } is a special similarity 99 on
X .
Therefore, we can solve any classiﬁcation problem by ﬁnding a appropriate similarity.
2.2 Capturing Fuzziness of Concept
The mathematical tools used in this section are fuzzy set theory and binary fuzzy relation.
See
Appendix A.1.3 for a brief introduction.
A large number of theories and experiments in cognitive science have shown that fuzziness is anintrinsic property of concept [13].
A short mathematical argument is as follows.
For any ( X , Y , ϕ ) -classiﬁcation problem, we have |X| > |Y| .
So ∃ x i , x j ∈ X , x i 6 = x j , such that ϕ
( x i ) =
ϕ
( x j ) .
An 106 awkward situation arises naturally.
The membership degrees of x i and x j to the concept ϕ ( x i ) areequal even if x i and x j may be very different.
When fuzziness of concept is introduced, the dilemmacan be solved directly.
Because x i and x j are different, their membership degrees to the conceptϕ ( x i ) are different.
As shown in Figure 1, the membership degree of the 1st image to ‘cat’ is greaterthan the 3rd image, and the membership degree of 3rd image to ‘dog’ is less than the 6th image.
The Section 2.1 concludes that any classiﬁcation problem can be solved based on the special similarity,i.e. ∀ x i , x j ∈ X , either they belong to the same concept then ϕ † (( x i , x j ))
= 1 , or they don’t belongto the same concept then ϕ † (( x i , x j )) =
Obviously, this assumption is hard to satisfy in realworld because it does not capture the intrinsic fuzziness of concept.
Based on the above analysis, a new mathematical tool is needed to effectively deal with the fuzzinessof concept.
Fortunately, an effective solution is found in the fuzzy set theory that is introducedby Lotﬁ A. Zadeh [20] in 1965.
In it, the crisp deﬁnition of the set is extended to a fuzzy one.
Speciﬁcally, in set theory, the membership degree of an object to a set is crisp, i.e. either it belongs tothe set then membership degree is 1, or it does not belong to the set then the membership degree is 0.
Instead, in fuzzy set theory, the membership degree of an object to a set is fuzzy, which is representedby a real number in the interval [0 , 1] (see Deﬁnition 11 in Appendix A.1.3 and Figure 1).
To capture the fuzziness of concept, ER in Deﬁnition 2 is replaced by FER (see Deﬁnition 16 and
Theorem 3 in Appendix A.1.3), which leads to the following deﬁnition.
Deﬁnition
Given a ( X , Y , ϕ ) -classiﬁcation problem, (1) let X ×
X be the new input space, (2) let[0 , 1] be the new output space, (3) let ϕ
X × X → [0 , 1] be the new target function, where ϕ ‡ is a
FER on X , and ∀ ( x i , x j ) ∈
X × X , ϕ ‡ (( x i , x j )) be the degree that x i and x j belong to the sameconcept.
Then ﬁnding the ϕ ‡ is deﬁned as its adjoint (cid:0)
X × X , [0 , 1] , ϕ ‡ (cid:1) -FER problem.
To solve the ( X , Y , ϕ ) -classiﬁcation problem, it is more reasonable to solve the (cid:0)
X × X , [0 , 1] , ϕ ‡ (cid:1) -
FER problem than the (cid:0)
X × X , { 0 , 1 } , ϕ † (cid:1) -classiﬁcation problem.
Because it is fuzzy whether twosamples belong to the same concept in the former.
This assumption ﬁts well with the principle thatconcept is fuzzy.
For a learning task, solving the (cid:0)
X × X , [0 , 1] , ϕ ‡ (cid:1) -FER problem is ﬁnding a FERon X that can approximate ϕ by using D train .
The process can be formally described as follows
[Equation]
where L is the loss function that measures how well the f ﬁts the training data set D train .
2.3 Exemplar Theory Based Concept Representation
The main knowledge used in this section is concept representation theory.
See Appendix A.1.1 for abrief introduction.
Suppose we have obtained a FER f ∗ (the optimal solution of formula (1)) by using D train .
From theperspective of cognitive science, we have constructed the basis used for representing all concepts in
Y .
Next, we need to complete concept representation based on the basis.
In cognitive science, exemplar theory is a kind of theory about concept representation [13].
Its coreidea is that human represents a concept by remembering some objects that belong to it.
Therefore,we need to ﬁnd some exemplars that can represent every concept in Y .
In cognitive science, whichobject is suitable as an exemplar of a concept is still an open question, but it is an unequivocal factthat the similarity is the key of exemplar theory [13].
In a data-driven learning task, the exemplar setof every concept can be determined by the following deﬁnition.
Deﬁnition
Given a ( X , Y , ϕ ) -classiﬁcation problem and a training data set D train .
Let f ∗ be the 147 FER found by using D train .
∀ c ∈ Y , let X c train =
{ x i | ( x i , y i )
∈ D train , y i = c } .
And ∀ x ∈ X c train ,let µ ( x, c ) =
| X c train |
P x i ∈
X c train f ∗ (( x, x i )) .
Then, ∀ c ∈ Y , the exemplar set is deﬁned as
E c = { x | x ∈ X c train , µ ( x, c ) is the top − n exe largest value in { µ ( x i , c )
| x i ∈
X c train }} , where n exe is a manually speciﬁed parameter.
The above deﬁnition is intuitive.
Take Figure 1 as an example.
Among the 3 images of ‘cat’, the 1stimage is more suitable as an exemplar of concept ‘cat’ than the other 2 images, because the 1st imageis more similar to the 3 images of ‘cat’ than the other 2 images.
After obtaining the exemplar set of every concept, the predicting process can be described as
[Equation]
2.4 The Framework of FLM
Combining the Sections 2.1-2.3, the training and test processes of FLM are given as follows.
Input : The training data set D train , the number of exemplars of every class n exe .
Find f ∗ by solving optimization problem (1).
for c ∈ Y do Compute the exemplar set E c of class c by
Deﬁnition 4.
endOutput : f ∗ and E c , ∀ c ∈ Y .
[Table]
do
[Table]
predicting
[Table]
y j of sample x j by formula (
[Table]
endOutput : ( x j , ˆ y j ) , ∀ x j
[Table] test .
2.5 Relationship to the Existing
Methods
The existing classiﬁers [1] aim to learn a function f : X → Y , while FLM aims to learn a FER 159 f
FER : X ×X → [0 , 1] .
Therefore, the existing classiﬁers and FLM solve classiﬁcation problem from 160 different perspectives.
FLM is designed based on concept cognition, while most existing classiﬁersseldom fully consider concept cognition.
The distance metric learning [21, 22, 23], the siamese network [24, 25, 26], and the relation networkbased methods [27, 28, 29] both aim to learn a similarity s :
X × X → R .
And FLM aims to learn 164 a FER f
FER :
X × X → [0 , 1] .
On the one hand, the f FER is a special similarity.
So these three 165 methods and FLM are similar in this respect.
On the other hand, the f FER needs to satisfy fourproperties (i.e. normalization, reﬂexivity, symmetry, and transitivity), while the s usually cannotsatisfy these properties at the same time.
Therefore these methods can not fully capture the nature ofclassiﬁcation.
The detailed analysis about the above conclusions can be found in Appendix A.2.
NN-FLM: A Neural Network Based FLM on R d
In this section, the most popular input space, X = R d , is taken into account.
And the goal is toinstantiate optimization problem (1) and solve it.
The mathematical tools used in this section arebinary fuzzy relation and the transitive-closure.
See Appendix A.1.3 for a brief introduction.
Andthe code implementation is provided in supplementary material.
3.1 The Separability of Adjoint
[Equation]
Classiﬁcation Problem
To solve the adjoint (cid:0)
[Equation]
-classiﬁcation problem described in the Deﬁnition 2,classiﬁer needs to deal with pairs of samples on R d .
One of the simplest methods is to concatenatethe feature vectors of a pair of samples, i.e.
[Equation]
where ∀ a , b ∈ R d , [ a ; b ] ∈ R 2 d denotes the concatenation of them.
Then, a pair of samples can beconverted into a composite sample, which can be processed by classiﬁer designed for R 2 d .
However,the binary-classiﬁcation problem obtained by the above method is not linearly separable.
See thefollowing theorem.
Theorem 1 For a (cid:0)
R d , Y , ϕ (cid:1) -classiﬁcation problem and a training data set D train , if the sampleis deﬁned as formula (3) , then the corresponding
[Equation]
-classiﬁcation problem is notlinearly separable.
(see Appendix A.3.2 for proof)
Therefore, FLM on R d must be able to handle the linear inseparability of the training set.
3.2 The Design of NN-FLM
To solve the original (cid:0)
R d , Y , ϕ (cid:1) -classiﬁcation problem, a FER on R d (see Deﬁnition 3) should befound.
It is very difﬁcult to directly obtain a binary fuzzy relation that satisﬁes transitivity.
Soan indirect method is adopted.
First, construct a binary fuzzy relation that satisﬁes reﬂexivity andsymmetry, i.e. FSR (see Deﬁnition 15 in Appendix A.1.3).
Then convert the FSR into FER.
To fully implement FLM, the following requirements must be considered.
(a) The linear inseparabilityof the problem (see Theorem 1).
(b) The learned function must be a FER, i.e. the learned function isnormalized, reﬂexive, symmetric, and transitive. (c) The loss function must consider the fuzziness ofconcept. (d) An effective optimizer must be designed, and it should be efﬁcient especially for largescale training data sets.
Based on the above analysis, NN-FLM includes three components: (1) Fuzzy similarity relationnetwork, which responds to (a) and the ﬁrst 3 conditions in (b), (2) Fuzziness permissible loss, whichresponds to (c) and indirectly satisﬁes the transitivity in (b), and (3) Stochastic gradient descent basedoptimizer, which responds to (d).
An overall design of NN-FLM is shown in Figure 2.
[Figure]
[Figure] [Figure] [Figure]
[Figure]
First, the feature extraction network can be formally described as follows
[Equation]
(4) where R + is the nonnegative real numbers, d h is the dimension of the latent space, and Θ is theset of learnable parameters.
A multi-layer neural network with nonlinear activation function isrecommended, to deal with the linear inseparability of the problem.
Second, the cosine similarity is used as the skeleton of the binary fuzzy relation network, i.e.∀ h i , h j ∈ R d h + , g ( h i , h j )
=
h T i
h
j k h
i k 2 k h j k 2 .
Let f (( x i , x j ) ; Θ)
=
g (h ( x i ; Θ) , h ( x j ; Θ)) , ∀ x i , x j ∈ R d ,be the composite of h and g .
Obviously, f is a FSR on R d , i.e. f is normalized, reﬂexive, andsymmetric.
It is very difﬁcult to make f satisfy transitivity on R d .
However, it is easy to make f satisfytransitivity on the training sample set X train =
{ x i | ( x i , y i )
∈ D train } .
Let S ∈
[0 , 1] n × n , s ij = 213 f (( x i , x j )) , ∀ i, j = 1 , 2 , · · · , n, be the FSR matrix on X train (see Deﬁnition 14 in Appendix A.1.3).
And let
[Equation]
(5) be the transitive-closure of S (see Deﬁnition 20, Theorem 4 and 5 in Appendix A.1.3).
Then,
T ∈ [0 , 1] n × n is a FER matrix on X train .(2)
Fuzziness permissible loss For a pair of training samples ( x i , y i ) , ( x j , y j ) ∈ D train , let t ij
bethe predicting FER value of ( x i , x j ) by formula (5).
The ( α, β ) -insensitive loss is used to measurethe difference between the predicting value and the true value and can be written as
[Equation]
where 0 ≤ α < 0 .
5 < β ≤ 1 are two hyper-parameters.
The gap β − α ∈ (0 , 1] is used for controlling 221 the fuzziness degree of concept.
And the greater the value is, the less the fuzziness will be.
Therefore,the loss on the entire training data set is
[Equation]
However, the time complexity of converting the FSR matrix S into the FER matrix T is O
[Equation]
224 (see Remark in Appendix A.1.3).
In practice, it is not feasible to directly optimize the above loss,especially when the number of training samples n is large.
Fortunately, due to the intrinsic propertyof the transitive-closure, we can indirectly control the loss (7) by optimizing the following loss
[Equation]
See the following theorem.
Theorem 2 Given a training set D train = [Equation] x
[Equation]
, y i ) | x [Equation]
[Equation] ϕ ( [Equation]
[Equation]
n (cid:9) and a
FSR f on X train = { x i | ( x i , y i ) ∈ D train } . Let S ∈ [0 , 1] n × n , s ij =
f (( x i , x j )) , ∀ i, j = 1 , 2 , · · · , n , 230 be the FSR matrix on X train , and T = t FSR2FER ( S ) be the transitive-closure of S . ∀ 0 ≤ α < 231 0 .
{+5 < β ≤ 1 , if P ni = P nj =
 L α,β ( s ij , y i , y+} j ) = 0 , then
P ni =
{+P nj =
+}
L α,β ( t ij , y i , y j )
= 0 . (see Appendix A.3.3 for proof) According to the Theorem 2, when the loss (8) reaches its minimum 0 , the loss (7) also reachesits minimum 0 .
The loss (8) can be optimized without calculating the FER matrix on all trainingsamples.
Therefore, the loss (8) is used to complete the learning process, in this paper.(3) Stochastic gradient descent based optimizer To sum up, the learning process of NN-FLM canbe written as
[Equation]
The time complexity of calculating the entire FSR matrix S is at least O (cid:0) n(cid:1) , which still has a largecomputational burden.
Fortunately, to optimize the above loss, it is not necessary to calculate theentire matrix S at the same time.
So the above optimization problem can be solved efﬁciently usingthe stochastic gradient descent with mini-batch, even when n is large.
Experiments
4.1 An Example on MNIST Data Set
Experimental settings In this section, the MNIST data set [30] is chosen to demonstrate the workingmechanism of NN-FLM.
In NN-FLM, a 5-layer convolutional neural network is used as the featureextraction network, and the fuzzy parameters are ﬁxed as α = 0 .
2 , β = 0 .
See Appendix A.4.1 formore experimental details.
Interpretability analysis First, Figure 3a shows the FSR matrix predicted by NN-FLM on 10,000test samples.
The following observations can be seen from it.
The FSR value between samplesfrom the same classes is signiﬁcantly larger than that of samples from different classes.
The
FSR matrix exhibits an obvious block-diagonal structure, where each block corresponds to a class.
Combining (1) and (2), we can conclude that NN-FLM can learn a high-quality FSR from the trainingdata.
Based on the learned FSR, the concepts of 10 visual digits can be accurately represented.
Second, Figure 3b shows the exemplars selected from the training set according to Deﬁnition 4.
Thefollowing observations can be seen from it.
(1) The 5 exemplars of every concept are regular, with nointerrupted strokes, no distortions, well-proportioned parts of the digits, and the digits are located inthe center of the image.
It shows that the exemplars of the every concept selected by NN-FLM canaccurately capture the characteristics of every concept.
(2) There are certain changes among the 5exemplars of every concept, which ensures that the exemplars can cover the intra-concept variations.
In summary, we can conclude that NN-FLM can learn high-quality FSR from training data.
Andthen based on learned FSR, the high-quality exemplars can be selected to accurately represent thecorresponding concept.
[Figure]
Robustness analysis A total of 31 test samples are misclassiﬁed and they are shown in Figure 3c.
The following conclusions can be drawn from it.
(1) All the misclassiﬁed samples are irregular digitalimages.
(2) Among them, 26 images ( 3 images) are hit by the 2nd (3rd) class label predicted by
NN-FLM.
For the 2 images (1st and 3rd in row 3 of Figure 3c), none of the top-3 class labelspredicted by NN-FLM hits the annotated label.
These 31 images are so irregular that it is difﬁcult to give crisp and uncontroversial labels even for human.
On the contrary, the class labels predicted by
NN-FLM are more in line with human cognition.
In summary, we can conclude that NN-FLM can adequately capture fuzziness of concept.
Therefore,the samples with a high degree of fuzziness can be identiﬁed and assigned reasonable candidateconcepts by NN-FLM.
This veriﬁes that NN-FLM is robust to the controversial samples.
4.2 Comparison With 179 Classiﬁers on 121 Data Sets
Experimental settings In the literature [1], 179 classiﬁers from 17 families (see Table 3 in Appendix
A.4.2) are systematically compared on 121 benchmark data sets (see Table 2 in Appendix A.4.2).
Inorder to verify the generalization performance of FLM, NN-FLM is chosen and compared with theabove 179 classiﬁers.
In all experiments, NN-FLM adopts a 3-layer fully connected network as thefeature extraction network.
The fuzzy parameters are ﬁxed as α = 0 . 2 , β = 0 .
More details aboutthe experimental settings can been found in Appendix A.4.2.
[Table]
48 ∗ 1.
88 ∗ 14.
49 ∗ 1.
70 ∗          1.
91 ∗        1.
54 ∗              6.
36 ∗    7.
59 ∗                  63.
a) ID: The unique ID of every data set (see Table 2 in Appendix A.4.2) ) ∗ : The 12 small sample data sets (see Table 5 in Appendix A.4.2) c
The rank considers that two or more classiﬁers have the same test accuracy. For example, on data set 121 there are 129 classiﬁers with 100% test accuracy, so the ranks of these 129 classiﬁers are (cid:16)P
[Equation]
/ 129 = 65 . The top-10 classiﬁers The 10 classiﬁers with the smallest mean rank among 180 classiﬁers on 121data sets are selected for comparison analysis.
They are NN-FLM (1st), 3 classiﬁers from the randomforest family (2nd parRF-t, 3rd rf-t, 5th rforest-R), 3 classiﬁers from the support vector machinefamily (4th svm-C, 6th svmPoly-t, 8th svmRadCost-t), 1 classiﬁer from the decision tree family (7th
C5.0-t), and 2 classiﬁers from the neural network family (9th elm-kernel-m, 10th avNNet-t).
analysis The test accuracy of the 179 comparison methods are from the literature [1].
Werun NN-FLM on 121 data sets and record the test accuracy.
Then the accuracy of 180 classiﬁers areused for evaluating their performance.
The following conclusions can be drawn from it.
(1) Due tospace constraint, the accuracy of NN-FLM and other top-9 comparison methods are shown in Table4 in Appendix A.4.2.
And it can be seen that the mean accuracy on 121 data sets of NN-FLM is5.49% higher than the 2nd parRF-t and 6.00% higher than the 3rd rf-t. Therefore, NN-FLM achievesa signiﬁcant improvement compared with the 179 classiﬁers.
The rank of NN-FLM among 180 methods on 121 data sets is recorded in Table 1.
And it can be seen that NN-FLM ranks ﬁrst on 43data sets (data sets 1-43) and ties for ﬁrst place on 10 data sets (data sets 44-53), and ranks in thetop-3 on more than half of the data sets (data sets 1-68).
Therefore, NN-FLM achieves outstandingrank among the the 180 classiﬁers.
Comparative advantage analysis Figure 4a shows the proportion of the top-10 classiﬁers rankedin top- K ( K = 1 , 3 , 5 , 10 , 15 , 20 ) among the 121 data sets.
And Figure 4b shows the mean andstandard deviation of the rank of the top-10 classiﬁers on 121 data sets.
The following conclusionscan be drawn from it.
(1) NN-FLM ranks the top-1 on 43 / 121 ≈ 35 .
5% of the data sets and the top-3on 68 / 121 ≈ 56 .
2% of the data sets.
And NN-FLM ranks in top-20 on more that 80% of the datasets.
None of the other 9 comparison classiﬁers ranks in top-20 on more than 50% of the datasets.
(3) NN-FLM has the smallest mean rank 10 . 51 .
And the mean rank is signiﬁcantly superior tothe second-ranked classiﬁer, parFR-t, 33 . 50 .
(4) NN-FLM has the smallest standard deviation 14 . 86 .
And the standard deviation is signiﬁcantly lower than to the other comparison classiﬁers.
In summary,the performance of NN-FLM is signiﬁcantly superior to the comparison classiﬁers.
At the same time,it can perform well on various tasks.
In contrast, the comparison classiﬁers can only perform well onlimited tasks.
Therefore, the performance of NN-FLM is more stable than the others.
The possiblereason is that NN-FLM can adequately capture the nature of classiﬁcation.
NN - F L M pa r R F - t r f - t sv m - C
r
f
o r e s t -
[Figure]
Experiments on small sample data sets To verify the performance of NN-FLM on small sampleclassiﬁcation problem, we selected 12 data sets with small number of samples (see Table 5 in
Appendix A.4.2) from the 121 data sets.
In these data sets, the mean number of training samples foreach class is less than 15.
We selected the top-10 methods with the smallest average rank on the 12 small sample data sets forcomparison, see Figure 4c.
The following conclusions can be drawn from it.
(1) Compared Figures4b with 4c, it can be seen that the top-10 classiﬁers on the 12 small sample data sets differ from thaton the all 121 data sets.
Some classiﬁers (svmPoly-t, C5.0-t, svmRadCost-t, and avNNet-t) rank inthe top-10 on the all data sets but not on the small sample data sets.
The mean rank of NN-FLMon the 12 small sample data sets is still the best.
The rank of NN-FLM is signiﬁcantly superior tothe second-ranked method, fr-t ( 5 . 91 ± 7 .
44 vs 25 .
33 ± 24 . 62 ).
In summary, it can be concluded thatwhen there are fewer training samples, the superiority of NN-FLM is more signiﬁcant.
The reason isthat NN-FLM can capture the characteristic and intra-concept variations of concept through a smallnumber of exemplars.
Therefore, the exemplar theory based concept representation is reasonable.
Conclusion
A new learning machine based on concept cognition, fuzzy learning machine, is proposed.
The newlearning machine has the advantages of strong interpretability and solid mathematical foundation.
And it is of certain degree of robustness because it can effectively deal with the intrinsic fuzziness inclassiﬁcation.
At the same time, a large number of comparison experiments show that it can reach anadvanced level even with a simple implementation.
Checklist
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reﬂect the paper’s contributions and scope?
[Yes] (b) Did you describe the limitations of your work?
[No] (c) Did you discuss any potential negative societal impacts of your work?
[No] (d) Have you read the ethics review guidelines and ensured that your paper conforms tothem?
[Yes]
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[Yes] See AppendixA.3.
(b) Did you include complete proofs of all theoretical results?
[Yes] See Appendix A.3.3.
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experi-
mental results (either in the supplemental material or as a URL)?
[Yes] The code of
the proposed method is provided in supplemental material.
And the data sets can be
download from the links in Appendix A.4.
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?
[Yes] See Appendix A.4.
(c) Did you report error bars (e.g., with respect to the random seed after running experi-
ments multiple times)?
[No]
(d) Did you include the total amount of compute and the type of resources used (e.g., typeof GPUs, internal cluster, or cloud provider)?
[N/A]
If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators?
[Yes]
(b) Did you mention the license of the assets?
[Yes]
(c) Did you include any new assets either in the supplemental material or as a URL?
[Yes]
The code of the proposed method is provided in the supplemental material.
(d) Did you discuss whether and how consent was obtained from people whose data you’re
using/curating?
[N/A]
(e) Did you discuss whether the data you are using/curating contains personally identiﬁable
information or offensive content?
[No]
If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if
applicable?
[N/A]
[Table]
[Table]
[Table] [Table]
[Table] [Table]