Fuzzy Learning Machine
Authors: Junbiao Cui
Jiye Liang
Venue: NeurIPS 2022 Accept (2022)
Keywords: Classification
Cognitive Science
Fuzzy Set Theory

Abstract: Classification is one of the most important problems in machine learning and the nature of it is concept cognition. So far, dozens of different classifiers have been designed. Although their working mechanisms vary widely, few of them fully consider concept cognition. In this paper, a new learning machine, fuzzy learning machine (FLM), is proposed from the perspective of concept cognition. Inspired by cognitive science, its working mechanism is of strong interpretability. At the same time, FLM roots in set theory and fuzzy set theory, so FLM has a solid mathematical foundation. The systematic experimental results on a large number of data sets show that FLM can achieve excellent performance, even with the simple implementation.

Fuzzy Learning Machine
Junbiao Cui!
Jiye Liang' 1jy0sxu. edu. en
9455468998qq. com
Key Laboratory of Computational Intelligence and Chinese Information Processing of Ministry of Education, School of Computer and Information Technology, Shanxi University, Taiyuan 030006, Shanxi, China.
Abstract
Classification is one of the most important problems in machine learning and the nature of it is concept cognition. So far, dozens of different classifiers have been designed. Although their working mechanisms vary widely, few of them fully consider concept cognition. In this paper, a new learning machine, fuzzy learning machine (FLM), proposed from the perspective of concept cognition. Inspired by cognitive science, its working mechanism is of strong interpretability. At the same time, FLM roots set theory and fuzzy set theory, so FLM has solid mathematical foundation. The systematic experimental results on large number of data sets show that FL.M can achieve excellent performance, even with the simple implementation.
Introduction
Given an input space X, an output space (a finite set) V, and an unknown function yp + V. the goal of classification is finding the ip approximation to it. Classification is one of the most important problems in machine learning, dating back to the origins of machine learning. In classification, each member of the output space )' corresponds to a concept. In essence, the process of classification the process of concept cognition. Concept contains our knowledge about the world, and we use concept to understand and organize the world. Without it, there will be no human intelligence at all. That why classification plays an important role in machine learning even in artificial intelligence.
Over the past decades, a large number of classifiers have been designed and achieved success on different tasks. The literature [1] systematically summarizes 179 classifiers from 17 different families, which contains almost all existing classifiers. Usually, generalization performance is regarded as an important dimension for evaluating classifiers. With the increasing depth and breadth of machine learning applications. classifiers are expected possess more and more good properties, such as interpretability [2, 3, 4], robustness [5, 6, 7], and so on.
The existing classifiers follow different design principles (see Appendix A.2.1 for detailed analysis), but they are designed without full consideration for the nature of classification and how humans do it. For specific scenario, the existing classifiers often perform well on one evaluation dimension but poorly on others. For example, deep neural network has exceed the human level in very specific settings [8]. Unfortunately, its working mechanism is difficult be understood due the complexity of the network structure [9, 10], and it is also very vulnerable to be attacked [11, 12]. Instead, human can perform well on a variety of tasks and on different evaluation dimensions.
To implement a classifier that is easy to understand and interpretable, one effective approach is to draw on relevant research in cognitive science. As mentioned above, classifying is essentially the
CCoresponding author
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

process of concept cognition. In cognitive science, there is a lot of valuable theories about how human learn, represent and use concept [13]. However, few of the existing classifiers are designed based on these theories, which leads to that the existing classifiers are unable to take into account the performance on multiple evaluation dimensions. Inspires by this, we are encouraged to design a new classifier based on concept cognition, which can simultaneously perform well on different evaluation dimensions. In this paper, a new learning machine is designed based on the following principles.
Similarity is the basis of concept representation. Similarity is a fundamental concept within cognitive science and plays crucial role in the process of human classification [14]. Concepts are represented based on similarity for children, which is also basic choice for adults [15]. Based on this principle, we prove the equivalence between classification and equivalence relation (ER) via binary relation in set theory. Considering that ER is special similarity, we actually demonstrate that all classification problems can be solved based on similarity (see Section 2.1).
Concept is fuzzy. An important conclusion in cognitive science is that fuzziness the intrinsic property of concept [13]. The fuzziness and the randomness the most important sources of the uncertainty. In machine learning, probability theory is often used for dealing with the uncertainty, such Bayesian methods [16, 17], label distribution learning [18], etc. It is sufficient for probability theory to deal with the randomness rather than the fuzziness [19]. However, the intrinsic property of concept is just the fuzziness rather than the randomness. An intuitive example is shown in Figure 1.
dog
Figure I: The visual concepts of 'cat' and 'dog' are of fuzziness. In fuzzy set theory, the membership degree of the 3rd image to "cat' ('dog') ) can be 0.6 (0.5). In probability theory, the probability of the 3rd image being 'cat" ('dog") can be 0.6 (1 - 0.6 0.4). Obviously, the latter is unreasonable, because the 3rd image can not be 'cat' in some case and be 'dog' in the other case.
Based on this principle, fuzzy equivalence relation (FER) in fuzzy set theory [20] rather than ER is chosen the core component of the new learning machine (see Section 2.2).
Exemplar theory based concept representation In cognitive science, there are several theories of concept representation [13]. Exemplar theory is chosen to solve concept representation problem, in this paper. Because it not only has good celf-consistency cognitive science, but also it can be naturally integrated with the data-driven machine learning paradigm (see Section 2.3).
To summarize, the main contributions of this paper are four-folds below: (1) This paper revisits classification problem from the perspective of concept cognition, and proves the equivalence between classification problem and ER. Furthermore, considering that concept is intrinsically fuzzy, the classification problem is modeled as an FER problem.
(2) A new and general learning machine FLM is proposed based on concept cognition for solving FER problem.
(3) A new fuzziness permissible loss is designed. Given the new loss, this paper prove that FER can be approximated effectively by fuzzy similarity relation (FSR). And then neural network based FLM (NN-FLM) is designed.
(4) Extensive experimental results demonstrate the superiority of NN-FLM on different evaluation dimensions including interpretability, robustness, and generalization performance.
2 Fuzzy Learning Machine
Basic notations Let X' and y {cI, c2,,,,,.... c be the input and output spaces, respectively. Let
s X + )' be the unknown target function and Vx E X, sfxx) E y is the true class label of the sample x. The goal of classification is to approximate yp as closely as possible. Let D
{(I,, lz) E X. y = spx,)) E y,i 1,2,... ... ... and Dreat = xz Øy E X, = n + 1.n +
2.... n m } be the training and test data sets, respectively.

---

2.1 Classification Problem Can Be Solved Based on Similarity
Researches [14, 15] on cognitive science prove that similarity is the basis of concept representation. However, the relationship between similarity and classification (e.g. whether all classification problems can be solved by using similarity) still lacks a mathematical justification. This section will accomplish it. The mathematical tool used in this section is binary relation in set theory. See Appendix 1.1 for a brief introduction.
First, the formal definition of classification problem is given.
Definition For an input space X, an oulput space y = {ei, , C2, ,..,,,,,,,,,, ,,,,, and an unknown target function yp : X + y. The (X, XxY,, classsifffci problem is defined as: Vx E X, finding s(xx)
should be pointed out that the Definition 1 is so general that almost all classification problems studied in machine learning can be represented by it.
Definition 2 Given (X,Y, (K, classiifffiiatinn problem, (1) let X x X be the new input space, (2) ler {0, 1} be the new oulpul space, (3) let spl X x X - {0, 1) be the new larget fanction, where V (Iz, xz) E X' x N. ot ((x1, = [(p (x44 (Iz)), then finding the ip is defined as its adjoint (ahhs=)-ccassffcattiii classificcation problem.
(X x X, (0, 11. (")
Obviously; the spt is an equivalence relution (ER) on X (see Definition 7 in Appendix A.1.2) and the adjoint (X x X, {0, 1), : -classification problem a binary-classffcattin problem.
Based on the above definitions, we have the following proposition.
Proposition 1 The (X,Y, -classsifficatnn problem is equivalent to its adjoint (X x X, (0, (0,1),111))) classification problem, i.e. if one problem is solved, then the other problem will also be solved. (See Appendix 1.3.1 for proof)
The Proposition 1 establishes the relationship between ER and classification problem. For any (X, (p)
. classificattin problem, we can always solve it equivalently by solving its adjoint (X X,
X, {0, 1}, v²) -classification problem. Because ER pp X' x X' + {0, 13 is special similarity on X. Therefore, we can solve any classification problem by finding a appropriate similarity.
2.2 Capture Fuzziness of Concept
The mathematical tools used in this section are fuzzy set theory and binary fuzzy relation. See Appendix 1.1.3 for a brief introduction.
A large number of theories and experiments in cognitive science have shown that fuzziness is an intrinsic property of concepts [13]. A short mathematical argument is as follows, For any (X, (p) classification problem, we have | > [D] So Ez,, I, E X, 24 + Zj, such that yo (zx) = 4 (rz). The membership degrees of x; and zj to the concept y (z;) are equal even if x4 and Ty may be very different. When fuzziness of concept is introduced, the dilemma can be solved directly. Because 3; and xy are different, their membership degrees to the concept o (I;;) are different. As shown in Figure 1, the membership degree of the st image to "cat" is greater than the 3rd image, and the membership degree of 3rd image to 'dog' is less than the 6th image.
The Section 2.1 concludes that any classification problem can be solved based on the special similarity, i.e. Vzi, Ij X, either they belong to the same concept then se ((x;, 1, or they don't belong to the same concept then spt ((1i,
O. Obviously, this assumption is hard to satisfy in real world because it does not capture the intrinsic fuzziness of concept.
mathematical tool is needed effectively deal with the fuzziness of concept. Fortunately, an effective solution is found in the fuzzy set theory that is introduced by Lotfi A. Zadeh [20] in 1965. In it, the crisp definition of the set is extended to fuzzy one. Specifically, in set theory, the membership degree of an object to a set is crisp, i.e. either it belongs to the set then membership degree is 1, or it does not belong to the set then the membership degree is 0. Instead, in fuzzy set theory, the membership degree of an object to set is fuzzy, which is represented by real number in the interval [0, 1] (see Definition 11 in Appendix A. 1.3 and Figure 1).
Based the above analysis,
To capture the fuzziness of concept. ER in Definition 2 is replaced by FER (see Definition 16 and Theorem 3 in Appendix A.1.3), which leads to the following definition.
3

---

Definition 3 Given a (X.V. -lasssifffiiati problem, (1) let X x X be the new input space, (2) let [0, 1] be the new output space, (3) let y² X x X' [0, be the new target function, where sØ is an FER on X, and V (0i, E X x X, up ((I,,;z)) be the degree that H, and 2j belong to the same concept. Then finding the spt is defined as its adjoint (x x X, [0, 11, 1),,+t)-FER problem.
To solve the (X, claasiiifiio problem, it more reasonable solve the (X' x X, [0, 1], () FER problem than the (X' x X, {0, }}; -elaassssfficatio problem. Because it is fuzzy whether two samples belong the same concept in the former. This assumption fits well with the principle that concept is fuzzy. For a learning task, solving the (X' x X, [o, 1], - problem is finding an FER on X' that can approximate yp by using Dirain- The process can be formally described follows
f* arg min L (Derain f) + +R((f) s.t. f {glg is an FER on X):
(1)
where L is the loss function that measures how well the f fits the training data set Darain R. is the regularization term, and y > 0 is the tradeoff parameter.
2.3 Exemplar Theory Based Concept Representation
The main knowledge used in this section is concept representation theory. See Appendix A.1.1 for brief introduction.
Suppose we have obtained an FER [* (the optimal solution of formula (1)) by using Dtrain- From the perspective of cognitive science, we have constructed the basis used for representing all concepts in y. Next, we need to complete concept representation based on the basis.
In cognitive science, exemplar theory is about concept representation [13]. Its core idea is that human represents a concept by remembering some objects that belong to it. Therefore, we need to find some exemplars that can represent every concept in V. In cognitive science, which object is suitable an exemplar a concept is still an open question, but is an unequivocal fact that the similarity is the key of exemplar theory [13]. In a data-driven learning task, the exemplar set of every concept can be determined by the following definition.
Definition 4 Given (X,Y, --)-classification problem and training data set D Let f be the FER found by using Derain. Vc V, let Xirain = {xx] (I ys) E Dirain yi = c). And Va E Xiraiu let μ (I. c) X
Emexx f* ((x,x;)), Then, Vc E V, the exemplar set is defined
Eg fx: la E Xirain j (I,C) is the top-n largest value in {p (zi, c) Jx, Xirainh) where n. is manually specified parameter.
The above definition is intuitive. Take Figure I as an example. Among the images of 'cat', the 1st image is more suitable as an exemplar of concept 'cat' than the other 2 images, because the 1st image more similar to the images of 'cat' than the other images.
After obtaining the exemplar set of every concept, the predicting process can be described as
1
Σ fz ((2, Vo X.
arg max eEY
EE
Z,EE
2.4 The Framework of FLM
Combining the Sections 2.1-2.3, the training and test processes of FLM are given follows.
Algorithm 1:
The training process of FLM
The test process of FLM Input: The test data set D the learned f*,
Algorithm 2:
Input: The training data set Dtrains the number of exemplars of every class nexe, Vc E y.
and the exemplar set Ez, Ve E )
Find f by solving optimization problem (1).
for T Doest do
for e e e do
Compute the predicting label ú of sample Ij by formula (2).
Compute the exemplar set E, of class e by Definition 4.
end
end
Output: (I.j, M,), Vay E Drest-
Output: f" and Ec, Vc y

---

2.5 Relationship to the Existing Methods
The existing classifiers [1] aim learn a function f: X' - V, while FLM aims learn an FER FFER X' x X' + [0. 1]. Therefore, the existing classifiers and FLM solve classification problem from different perspectives. FLM designed based on concept cognition, while most existing classifiers seldom fully consider concept cognition. There is also large corpus fuzzy classifiers [21] that use fuzzy set theory for classification task. Compared with most existing fuzzy classifiers, FLM is more suitable for data-driven learning tasks.
The distance metric learning [22, 23, 24], the siamese network [25, 26, 27], and the relation network based methods [28, 29, 30] both aim to learn a similarity S: X' x X' + R. And FLM aims to learn an FER FFER X X + [0, 1]. On the one hand, the FFER is special similarity. So these three methods and FLM are similar in this respect. On the other hand, the fEER needs to satisfy four properties (i.e. normalization, reflexivity, symmetry, and transitivity), while the usually cannot satisfy these properties at the same time. Therefore these methods can not fully capture the nature of classification.
The detailed analysis about the above conclusions can be found in Appendix A.2.
3 NN-FLM: A Neural Network Based FLM on Rd
In this section, the most popular input space, X' Rd, is taken into account. And the goal is to instantiate optimization problem (1) and solve The mathematical tools used in this section are binary fuzzy relation and the transitive-closure See Appendix A.1.3 for brief introduction. And the code implementation is provided in supplementary material.
3.1 The Separability of Adjoint (IRd d IRd, (0,1),))))CClassfffiiaii Classifficato Problem
To solve the adjoint (I 12d, {0, 1}, :0)) -llsssification problem described in the Definition classifiers need to deal with pairs of samples on Rd. One the simplest methods is to concatenate the feature vectors of pair of samples, i.e.
Deoncat
{[xx;: x y.. I (y: b;)) (x,, Ut) (X,, Uj) E Dirain)
(3)
where Va, b E Rd, (a: bj E 2d denotes the concatenation of them. Then, pair of samples can be converted into a composite sample, which can be processed by classifiers designed for R2d, However, the binary-classification problem obtained by the above method is not linearly separable. See the
following theorem.
Theorem For (R. V. -classiiication problem and training data set Dwaine if the sample defined as formula (3), then the corresponding (II2d, { [0,1), (0) cllassfficcaai problem is not linearly separable. (see Appendix f.r proof)
Therefore, FLM on Rdd must be able to handle the linear inseparability of the training set.
3.2 The Design of NN-FLM
To solve the original (Rd, ,,, sp) -classification problem, FER on Rd (see Definition 3) should be found. It is very difficult to directly obtain a binary fuzzy relation that satisfies transitivity. So an indirect method is adopted. First, construct binary fuzzy relation that satisfies reflexivity and symmetry, i.c. FSR (see Definition 15 in Appendix A.1.3). Then convert the FSR into FER.
To fully implement FLM, the following requirements must be considered. (a) The linear inseparability of the problem (see Theorem 1). (b) The learned function must be an FER, i.c. the learned function normalized, reflexive, symmetric, and transitive.
(c) The loss function must consider the fuzziness of concept. (d) An effective optimizer must be designed, and it should be efficient especially for large scale training data sets.
Based on the above analysis, NN-FLM includes three components: (1) Fuzzy similarity relation network, which responds to (a) and the first 3 conditions in (b), (2) Fuzziness permissible loss, which responds to (c) and indirectly satisfies the transitivity in (b), and (3) Stochastic gradient descent based optimizer, which responds to (d). An overall design of NN-FLM is shown in Figure 2.
5

---

Feature Extraction Network
Binary Fuzzy
Fuzziness Permissible
Relation Network
Fuzzy Similantty Relation Input Space
Proximate Fuzzy Equivalence Relation on Training Data Set
Porward propagation
Backward propagation
Figure 2: The overall design of NN-FLM.
(1) Fuzzy similarity relation network contains two parts: feature extraction network and binary fuzzy relation network.
First, the feature extraction network can be formally described as follows
(4) where R is the nonnegative real numbers, d,, is the dimension of the latent space, and θ is the set of learnable parameters. A multi-layer neural network with nonlinear activation function is recommended, deal with the linear inseparability of the problem.
Vx E Rd (x: Q) E TR4,,
Second, the cosine similarity is used as the skeleton of the binary fuzzy relation network, i.e. h:',
Let f ((x;, x,): ;)) g (h (x,; h), (x,; 0)) Xj Rd Vh, hy 1 .g (h;, by) [h,
be the composite of h and g. Obviously, is an FSR on RRd, i.e. f is normalized, reflexive, and symmetric.
It is very difficult to make f satisfy transitivity on Rd. However, it is easy to make f satisfy transitivity on the training sample set Xtrain = {x: (x,.3)) Drrain). Let S [o, 1]*x,, 11 ,8 = I ((x;, x,)), Vi,j 1,2,... ,, be the FSR matrix on Xtrain (see Definition 14 in Appendix A.1.3). And let
(5) be the transitive-clouure of S (see Definition 20, Theorem 4 and 5 in Appendix A.1.3). Then, T T [0, 1| is an FER matrix on Xtrain-
T = UFSR2FER (S)
(2) Fuzziness permissible loss For pair of training samples (X.,W) (X,. My) E Dirain let lij be the predicting FER value (x, x;) by formula (5). The (a, B)-insensitive loss used to measure the difference between the predicting value and the true value and can be written
max (1i a. 0) Yi + yj
Ca (taj, Vs. vi) = max (3- 1,, y/ = yj
(6)
where S a < 0.5 < B < are two hyper-parameters. The gap 3 3 E (0, 1] used for controlling the fuzziness degree of concept. And the greater the value is, the less the fuzziness will be. Therefore, the loss on the entire training data set is
LEER >> La.s (taj, Vo. Uz)
However, the time complexity of converting the FSR matrix S into the FER matrix is O (n log, n) (see Remark Appendix 1.3.3). practice, is not feasible directly optimize the above loss, especially when the number of training samples is large. Fortunately, due to the intrinsic property of the cransitive-cloure, we can indirectly control the loss (7) by optimizing the following loss
CFSR
Law (8ij, Us. M,)
(8)
See the following theorem.
Theorem 2 Given training set Dusin { (xx, 21) |x, E Rd === =x)) 1,2, .n} and an
FSR on Xtrain {x,] (x 31) Drrain) }. Let S E [0, 1|nxn, 8i f((xx,x,))), Vi,j 1, 2,
be the FSR matrix on X Xirain and tFSR2FER (S) ()){ be the transitive S. < a <
0.5 < < if E EØ=1 La,9 (Sig, Ui. Vi. O, then -y La.s (t.j, Uo. v₁) 0. (see Appendix A.3.3 for proof)

---

According to the Theorem 2. when the loss (8) reaches its minimum 0, the loss (7) also reaches its minimum The loss (8) can be optimized without calculating the FER matrix on all training samples. Therefore, the loss (8) is used to complete the learning process, in this paper.
(3) Stochastic gradient descent based optimizer To up, the learning process of NN-FLM can be written as
min n
La ((x);xx)):)) yr. 11) + (R(().
The time complexity of calculating the entire FSR matrix S is at least O (n"). which still has large computational burden. Fortunately, to optimize the above loss, it is not necessary to calculate the entire matrix the same time. So the above optimization problem can be solved efficiently using the stochastic gradient descent with mini-batch even when n is large.
Experiments
4.1 An Example on MNIST Data Set
Experimental settings In this section, the MNIST data set [31] is chosen to demonstrate the working mechanism of NN-FLM. In NN-FLM, a 5-layer convolutional neural network used as the feature extraction network, and the fuzzy parameters are fixed as 0.2, 3 0.8. See Appendix A.4.1 for more experimental details.
Interpretability analysis First, Figure 3a shows the FSR matrix predicted by NN-FLM on 10,000 test samples. The following observations can seen from it. (1) The FSR value between samples from the same classes is significantly larger than that of samples from different classes. (2) The class.
FSR matrix exhibits an obvious block-diagonal structure, where each block corresponds Combining (I) and (2), we can conclude that NN-FLM can learn high-quality FSR from the training data. Based on the learned FSR, the concepts 10 visual digits can be accurately represented. Second, Figure 3b shows the exemplars selected from the training set according to Definition 4. The following observations can be seen from it. (1) The 5 exemplars every concept are regular, with no interrupted strokes, distortions, well- vell-proportioned parts of the digits, and the digits are located in the center of the image. It shows that the exemplars accurately capture the characteristics of every concept. (2) There certain changes among the exemplars of every concept, which ensures that the exemplars can cover the intra-concept variations.
the every concept selected by NN-FLM can
In summary, we can conclude that NN-FLM can learn high-quality FSR from training data. And then based on learned FSR, the high-quality exemplars can be selected to accurately represent the corresponding concept.
0
O
D
O
O
1
7
8
3 3
4
4
2
2
J
3
4
4
3
S
3
3
5
4
S
5
S
6
4
6
6
6
>
7
6
6
6
7
7
7
8
8
8 9 9
9
9
2
/
2
9
4
9
4
(a)
(b)
(c)
Figure 3: (a) The predicting FSR matrix the test set (the samples are sorted by class labels). (b) The exemplars of 10 classes selected from the training set. (c) The 31 errors of NN-FLM, with annotated label (up right) and top-2 top-3 predicting labels (down right).
Robustness analysis A total of 31 test samples are misclassified and they are shown in Figure 3c. The following conclusions can be drawn from it. (1) All the misclassified samples are irregular digital images. (2) Among them, 26 images (3 images) are hit by the 2nd (3rd) class label predicted by NN-FLM. (3) For the images (Ist and 3rd in row 3 of Figure 3c), none of the top-3 class labels predicted by NN-FLM hits the annotated label. These 31 images are irregular that it is difficult
7

---

to give crisp and uncontroversial labels even for humans. In this case, the class labels predicted by NN-FLM line with human cognition certain degree.
In summary, we can conclude that NN-FLM can adequately capture fuzziness of concept. Therefore, the samples with high degree of fuzziness can be identified and assigned reasonable candidate concepts by NN-FLM. This verifies that NN-FLM is robust to the controversial samples.
4.2 Comparison With 179 Classifiers on 121 Data Sets
Experimental settings In the literature [1], 179 classifiers from 17 families (see Table in Appendix A.4.2) are systematically compared on 121 benchmark data sets (see Table 2 in Appendix A.4.2). In order verify the generalization performance of FL.M, NN-FLM is chosen and compared with the above 179 classifiers. all experiments, NN-FLM adopts a 3-layer fully connected network as the feature extraction network. The fuzzy parameters are fixed as a 0.2, 3 0.8. More details about the experimental settings can been found in Appendix A.4.2.
Table 1: The rank NN-FLM among 180 classifiers on 121 data sets
ID
Rank
Rank
Rank
Rank
Rank
Rank
21
41
61
2
81
10
T01
21
2
22
42
62*
2
82
10.5
102
23
23
43
63
2.5
83
11
103*
23.5
4
24
44
1.5
64*
2.5
84
12
104
24.5
5
25
45
1.5
65
85
13
105
26
26
46
1.5
66
13.5
106
26
7
27
47
1.5
67
3
87
14
107
29
8
28
48"
1.5
68
88*
14.5
108
31.5
29
49"
1.5
69
3.5
89
15
109
32
10
30
50
1.5
70*
4
90
15
110
35
31
51
1.5
71
4.5
91*
15
111
12
32
52
1.5
72
5
92
15.5
112
38
13
33
53
1.5
73
5.5
93
15.5
113
39
14
34
54*
74
94
16
114
39
15
35
55
2
75
6.5
95
16
115
39
16
36
56
76
7.5
96
16.5
116
42
17
37
57
77
97
17.5
117
47
18
38
58
2
78
98
18.5
118
60
39
59*
79
19
20
40
60
2
80
10
100
20
120
63.5
121
65
(a) ID: The unique ID of every data set (see Table 2 in Appendix A.4.2). (b) The 12 small
sample data sets (see Table in Appendix A.4.2). (c) The rank considers that two or more classifiers have the same test accuracy. For example, data set 121 there are 129 classifiers with 100% test accuracy, so the ranks of these 129 classifiers are
129 rr /129 = 65.
The top-10 classifiers The classifiers with the smallest mean rank among 180 classifiers on 121 data sets selected for comparison analysis. They are NN-FLM (Ist), classifiers from the random forest family (2nd parRF-t, 3rd rf-t, 5th rforest-R), classifiers from the support vector machine family (4th svm-C, 6th svmPoly-t, 8th svmRadCost-O), 1 classifier from the decision tree family (7th C5.0-t), and classifiers from the neural network family (9th elm-kernel-m, 10th avNNet-t).
Accuracy analysis The test accuracy of the 179 comparison methods are from the literature [1]. We run NN-FLM on 121 data sets and record the test accuracy. Then the accuracy of 180 classifiers are used for evaluating their performance. The following conclusions can be drawn from it. (1) Due to space constraint, the accuracy of NN-FLM and other top-9 comparison methods are shown in Table 4 in Appendix A.4.2. And it can be seen that the mean accuracy 121 data sets of NN-FLM is 5.49% higher than the 2nd parRF-t and 6.00% higher than the 3rd rf-t. Therefore, NN- WNNFLMM achieves significant improvement compared with the 179 classifiers. (2) The rank of NN-FLM among 180

---

methods on 121 data sets is recorded in Table 1. And it can be seen that NN-FLM ranks first on 43 data sets (data sets 1-43) and ties for first place on 10 data sets (data sets 44-53), and ranks in the top-3 on more than half of the data sets (data sets 1-68). Therefore, NN-FLM achieves outstanding rank among the the 180 classifiers.
Comparative advantage analysis Figure 4a shows the proportion of the top-10 classifiers ranked in top-K K (K 3, 5, 10, 15, 20) among the 121 data sets. And Figure 4b shows the mean and standard deviation of the rank of the top-10 classifiers on 121 data sets. The following conclusions can be drawn from it. (1) NN-FLM ranks the top-1 on 43/ 121 R 35.5% of the data sets and the top-3 on 68/ 68/122 //12 = 56.2% of the data sets. And NN-FLM ranks in top-20 on more that 80% of the data sets. (2) None of the other 9 comparison classifiers ranks in top-20 on more than 50% of the data sets. (3) NN-FLM has the smallest mean rank 10.51. And the mean rank is significantly superior to the second-ranked classifier, parFR-t, 33.50. (4) NN-FLM has the smallest standard deviation 14.86. And the standard deviation is significantly lower than to the other comparison classifiers. In summary, the performance of NN-FLM is significantly superior to the comparison classifiers. At the same time, it can perform well on various tasks. In contrast, the comparison classifiers can only perform well on limited tasks. Therefore, the performance of NN-FLM is more stable than the others. The possible reason is that NN-FLM can adequately capture the nature of classification.
50
gevetts
40
20
20-
10
paeti
sime
Aemana
tenn agg
c301
emierdde
Homis
sent spd
C3.11
attstameei
Andeet
et
cnviemate
pettre
(e)
(b)
(c)
Figure 4: The statistics top-10 10 methods among 180 classifiers. (a) The proportion of top- K rank on 121 data sets. (b) The mean and standard deviation of the rank on 121 data sets. (c) The mean and standard deviation of the rank on 12 small data sets.
Experiments on small sample data sets To verify the performance of NN-FLM on small sample classification problem, we selected 12 data sets with small number of samples (see Table 5 in Appendix A.4.2) from the 121 data sets. In these data sets, the mean number of training samples for each class is less than 15.
We selected the 10 methods with the smallest average rank on the 12 small sample data sets for comparison, see Figure 4c. The following conclusions can be drawn from it. (1) Compared Figures 4b with 4c, can be seen that the top-10 classifiers on the 12 small sample data sets differ from that on the all 121 data sets. Some classifiers (svmPoly-t, C5.0-t, svmRadCost-t, and avNNet-t) rank in the top-10 10 on the all data sets but not on the small sample data sets. (2) The mean rank of NN-FLM on the 12 small sample data sets is still the best. (3) The rank of NN-FLM is significantly superior to the second-ranked method, fr-t (5.91 + 7.44 vs 25.33 + 24.62). In summary, it can be concluded that when there are fewer training samples, the superiority of NN-FLM is more significant. The reason is that NN-FLM can capture the characteristic and intra-concept variations concept through a small number of exemplars. Therefore, the exemplar theory based concept representation is reasonable.
5 Conclusion
A new learning machine based on concept cognition, fuzzy learning machine, is proposed. The new learning machine has the advantages of strong interpretability and solid mathematical foundation. And it is of certain degree of robustness because it can effectively deal with the intrinsic fuzziness in classification. At the same time, a large number of comparison experiments show that it can reach an advanced level even with simple implementation.

---

Acknowledgments and Disclosure of Funding
This work is supported by the National Key Research and Development Program of China (under grant 2020AAA01061000)
References
[1] Manuel Fernández Delgado, Eva Cernadas, Senén Barro, and Dinani Gomes Amorim. Do we need hundreds of classifiers to solve real world classification problems? Journal of Machine Learning Research, 15(1):3133-118, 2014.
[2] Cynthia Rudin. Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. Nature Machine Intelligence, 155::206-215, 2019.
[3] Zhifeng Kong and Kamalika Chaudhuri. Understanding instance-based interpretability of variational auto-encoders. In Advances in Neural Information Processing Systems, virtual, 2021.
[4] Aya Abdelsalam Ismail, Héctor Corrada Bravo, and Soheil Feizi. Improving deep learning interpretability by saliency guided training. In Advances in Neural Information Processing Systems, virtual, 2021.
[5] Cheng Zhang, Kun Zhang, and Yingzhen Li. A causal view on robustness of neural networks. In Advances Neural Information Processing Systems, virtual, pages 289-301, 2020.
[6] Shahd Safarani, Arn Nix, Konstantin Willeke, Santiago A. Cadena, Kelli Restivo, George H. Denfield, Andreas S. Tolias, and Fabian H. Sinz. Towards robust vision by multi-task learning on monkey visual cortex. In Advances in Neural Information Processing Systems, virtual. 2021.
[7] songwei Ge, Vasu Singla, Ronen Basri, and David W. Jacobs. Shift invariance can reduce adversarial robustness. In Advances in Neural Information Processing Systems, virtual, 2021.
[8] Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael S. Bernstein, Alexander C. Berg, and Li Fei- Fei. Imagenet large scale visual recognition challenge. International Journal of Computer Vision, 115(3):211-2555 2015.
[9] Alexey Dosovitskiy and Thomas Brox. Inverting visual representations with convolutional networks. In IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, pages 4829-4837, 2016.
[10] Aravindh Mahendran and Andrea Vedaldi. Understanding deep image representations by inverting them. In IEEE Conference on Computer Vision and Puttern Recognition, Boston, MA, USA, pages 5188-5196. 2015.
[11] Christian Szegedy, Wojeiech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian J. Goodfellow, and Rob Fergus. Intriguing properties of neural networks. In International Conference on Learning Representations, Banff, AB, Canada, 2014.
[12] Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. In International Conference on Learning Representations, Vancouver, BC. Canada, 2018.
[13] Gregory Murphy. The Big Book of Concepts. MIT press, 2004.
[14] Ulrike Hahn. Similarity. Wiley Interdisciplinary Reviews: Cognitive Science, 5(3):271-280. 2014.
[15] Heidi Kloos and Vladimir M Sloutsky. What's behind different kinds of kinds: effects of statis- tical density on learning and representation of categories. Journal of Experimental Psychology: General, 137(1):52-72 72, 2008
[16] Pedro M. Domingos and Michael J. Pazzani. On the optimality of the simple bayesian classifier under zero-one loss. Machine Learning, 103-11):033-33, 1997.
10

---

[17] Judea Pearl. Probabilistic reasoning in intelligent systems: networks of plausible inference. Morgan Kaufmann Series in Representation and Reasoning. Morgan Kaufmann, 1989.
[18] Xin Geng. Label distribution learning. IEEE Transactions on Knowledge and Data Engineering, 28(7):1734 17488 2016.
[19] Lotfi A. Zadeh. Is probability theory sufficient for dealing with uncertainty in AI: A negative view. In Anmual Conference on Uncertainty in Artificial Intelligence, Los Angeles, CA, USA, pages 103-116, 1985.
[20] Lotfi Asker Zadeh. Fuzzy sets. Information and Control, 8(3):338-353, 1965.
[21] Ludmila I. Kuncheva. Fuzzy Classifier Design, volume 49 of Studies in Fuzziness and Soft Computing. Springer, 2000.
[22] Eric P. Xing, Andrew Y. Ng, Michael I. Jordan, and Stuart J. Russell. Distance metric learning with application clustering with side-information. In Advances in Neural Information Processing Systems, Vancouver, British Columbia, Canada, pages 505-512, 2002.
[23] Dewei Li and Yingjie Tian. Survey and experimental study on metric learning methods. Neural Networks, 105:447-462, 2018.
[24] Mang Ye, Jianbing Shen, Gaojie Lin, Tao Xiang, Ling Shao, and Steven C.H. Hoi. Deep learning for person re-identification:: A survey and outlook. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021, DOI: 10. 1010TPAAM..2222.055555...
[25] Jane Bromley, Isabelle Guyon, Yann LeCun, Eduard Säckinger, and Roopak Shah. Signature verification using a siamese time delay neural network. In Advances Neural Information Processing Systems, Denver, Colorado, USA, pages 737-744, 1993.
[26] Gregory Koch. Siamese neural networks for one-shot image recognition. In International Conference on Machine Learning, Deep Learning Workshop, 2015.
[27] Davide Chicco. Siamese neural networks: An overview. Artificial Neural Networks, pages 73 73-94, 2021.
[28] Flood Sung, Yongxin Yang, Li Zhang, Tao Xiang, Philip H. S. Torr, and Timothy M. Hospedales. Learning to compare: Relation network for few-shot learning. In IEEE Conference Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, pages 199--108, 2018
[29] Binyuan Hui, Pengfei Zhu, Qinghua Hu, and Qilong Wang. Self-attention relation network for few-shot learning. In IEEE International Conference on Multimedia and Expo Workshops, Shanghai, China, pages 198 203, 2019.
[30] Xiaoxu Li, Jintao Yan, Jijie Wu, Yuxin Liu, Xiaochen Yang, and Zhanyu Ma. Anti-noise relation network for few-shot learning. In Asia-Paacifi Signal and Information Processing Association Annual Summit and Conference, Auckland, New Zealand, pages 1719-1724, 2020.
[31] Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(1 1):2278-2233 23244, 1998.
[32] Kenneth H. Rosen. Discrete Mathematics and Its Applications. McGraw-HEr Hill Education, 2018.
[33] Bernard De Baets and Etienne Kerre. Fuzzy relations and applications. In Advances in Electronics and Electron Physics, volume 89, pages 255-324. Elsevier, 1994.
[34] Ling Zhang and Bo Zhang. Quotient Space Based Problem Solving: A Theoretical Foundation of Granular Computing. Tsinghua University Press, 2014.
[35] W. Bandler and L. J. Kohout. Special properties, closures and interiors of crisp and fuzzy relations. Fuzzy Sets and Systems, 26(3):117-331, 1988.
[36] Corinna Cortes and Vladimir Vapnik. Support- vector networks. Machine Learning, 20(3):273- 297, 1995.

---

[37] J. Ross Quinlan. Induction of decision trees. Machine Learning, 1(1):81-106 1986.
[38] Yann LeCun, Yoshua Bengio, and Geoffrey E. Hinton. Deep learning. Nature, 521(7553):46--
444, 2015.
[39] Harry Zhang. The optimality of naive bayes. In International Florida Artificial Intelligence Research Society Conference, Miami Beach, Florida, USA, pages 562-567, 2004.
[40] Yoav Freund and Robert E. Schapire. A Decision-theorede generalization of on on-line learning and an application to boosting. Journal of Computer and System Sciences, 55(1):1 1911199133 139, 1997.
[41] Leo Breiman. Random forests. Machine Learning, 45(1):5 2001.
[42] Ryan M. Rifkin and Aldebaro Klautau. In defense of one-vs-all classification. Journal of Machine Learning Research, 5:101-141, 2004.
[43] Thomas M. Cover and Peter E. Hart. Nearest neighbor pattern classification. IEEE Transactions on Information Theory, 13(1)211-27, 1967.
[44] Siyuan Shan, Yang Li, and Junier B. Oliva. Meta-neighborhoods. In Advances in Neural Information Processing Systems, virtual, pages 5047-5057, 2020.
[45] Diederik Kingma and Jimmy Ba. Adam: method for stochastic optimization. In Interna- tional Conference on Learning Representations, San Diego, CA, USA, 2015.
Checklist
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's
contributions and scope? [Yes]
(b) Did you describe the limitations of your work? [No]
(c) Did you discuss any potential negative societal impacts of your work? [No]
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you are including theoretical results...
(a) Did you state the full set of assumptions all theoretical results? [Yes] See Appendix A.3.
(b) Did you include complete proofs of all theoretical results? [Yes] See Appendix A.3.
3. If you ran experiments..
(a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either in the supplemental material
URL)? [Yes] The code of the proposed method is provided in supplemental material. And the data sets can be download from the links in Appendix A.4.
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes] See Appendix A.4.
(c) Did you report error bars (e.g., with respect to the random seed after running experi- ments multiple times)? [No]
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, cloud provider)? [N/A]
4. If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators? [Yes]
(b) Did you mention the license of the assets? [Yes]
(c) Did you include any new assets either in the supplemental material or as URL? [Yes]
The code of the proposed method provided in the supplemental material.
(d) Did you discuss whether and how consent was obtained from people whose data you're 're using/curating? [N/A]
12

---

(c) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content? [No]
5. If you used crowdsoureing conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if applicable? [N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable? [N/A]
(c) Did you include the estimated hourly wage paid participants and the total amount spent on participant compensation? [N/A]
13


Reviews:
[{"review_id": "qHCfxKL5JHt", "reviewer": "Reviewer_qNtC", "summary": "In the paper \"Fuzzy Learning Machine\" the authors propose an approach to learn a classifier via a neural network forming a fuzzy equivalence relation. Deriving the approach from fuzzy set theory, the authors find their approach to perform particularly well across a number of datasets comparing the approach to various other classifiers.\n", "questions": "* In what regard does the proposed method really go beyond the already existing corpus of literature in fuzzy classifiers?\n* Is the classification problem really solved with definition 2? To me, it seems like the information to which class at least a representative data point belongs to is missing. Hence, the problem is only solved if the y's of some representative class members are known.\n* Is not it a problem that the model is not permutation invariant? At least it seems to me that it is not invariant to the order of inputs.\n", "limitations": "Limitations, except for runtime complexity to compute the FER matrix, are not really discussed. When does the approach fail and why does it fail?", "rating": 5, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "## Weaknesses\nThe idea of employing fuzzy set theory for classification tasks is not new at all and I am wondering what is now the methodological novelty of the approach. In general the idea of comparing instances / data points according to their similarity is the basic idea behind learners using kernel functions where the shape of a concept is specified via the respective kernel. However, there is also a relatively large corpus of literature on classifiers leveraging fuzzy set theory, even working exactly with neural networks and the idea of fuzzy equivalence relations. Still this related work is neither discussed nor cited in the paper. See for example the following references:\nAcharya, U. Rajendra, et al. \"Classification of heart rate data using artificial neural network and fuzzy equivalence relation.\" Pattern recognition 36.1 (2003): 61-68.\nMoser, Bernhard. \"On Representing and Generating Kernels by Fuzzy Equivalence Relations.\" Journal of machine learning research 7.12 (2006).\nMeier, Andreas, and Nicolas Werro. \"A fuzzy classification model for online customers.\" Informatica 31.2 (2007).\nSenge, Robin, and Eyke Hüllermeier. \"Top-down induction of fuzzy pattern trees.\" IEEE Transactions on Fuzzy Systems 19.2 (2010): 241-252.\nKuncheva, Ludmila. Fuzzy classifier design. Vol. 49. Springer Science & Business Media, 2000.\nSun, C-T., and J-S. Jang. \"A neuro-fuzzy classifier and its applications.\" [Proceedings 1993] Second IEEE International Conference on Fuzzy Systems. IEEE, 1993.\nUebele, Volkmar, Shigeo Abe, and Ming-Shong Lan. \"A neural-network-based fuzzy classifier.\" IEEE Transactions on Systems, Man, and Cybernetics 25.2 (1995): 353-361.\nIt is unclear to me how this part of the literature is widely ignored by the authors when they seem to come from that area.\n\nOverall, the paper has a good structure but could benefit from proofreading. Especially, a vs an is a frequent problem in the text, e.g., \"a input space\", \"a output space\", \"a FER\". Then, \"classifier\", \"concept\" and \"classification process\"  are used without an article. Some parts also seem overly complicated to me. For example, consider the proof that a non-linear model is needed to tackle the derived problem where the instances are concatenated. I do not know whether yet another proof for the fact that an XOR problem cannot be tackled via a linear model is really needed. This could have been simplified. Furthermore, I find that the example given in Figure 1 is not very well chosen. The concepts cat and dog have crisp biological borders and a human not being able to distinguishing the two categories is rather due to epistemic uncertainty than fuzziness of the concept borders. Personally, I would also argue that non of the three cats is more or less representative of the category or concept \"cat\".\n\nA claim that was made by the authors is that their approach indeed learns \"concepts\" instead of just assignments. However, there was no proof given in the paper that this is really the case. Especially, there is no presentation or demonstration of any particular concepts that were induced by fitting their model. I would even argue that from Figure 3 is rather becomes clear that it is learning not really any concepts as the FSR matrix shows more or less the same color for every cell not being on the main diagonal. If it was to learn real concepts I would also expect that a 0 would receive a lower membership score for the concept 1 than a 7 for example. A better overall performance is no proof for the claim that the method learns concepts.\n\nAnother branch of classification literature also tries to capture concepts for classification purposes: Analogy learning.\nBayoudh, Sabri, Laurent Miclet, and Arnaud Delhay. \"Learning by Analogy: A Classification Rule for Binary and Nominal Data.\" IJCAI. 2007.\n\n## Strengths\nSince most people in the machine learning community will not be that much familiar with fuzzy set theory, I liked it very much that all fundamental definitions were provided by the authors in the paper or supplementary material to make it self sufficient.\n\nAccording to the experiments the proposed method seems to perform very strong compared to a set of almost 200 classifiers. However, the way how the rankings were calculated is a little bit odd. Why are 65 learners sharing rank 1 with 100% accuracy receive a rank of 65? This will most likely also affect the average rank statistics compared for the ten classifiers later on. I would rather expect that performances with a tie receive the same higher rank, leaving free the next n-1 spots in the ranking.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "f1er7W5Bj-B", "reviewer": "Reviewer_iVDZ", "summary": "This paper proposes a new machine learning method for classification called Fuzzy Learning Machine. The paper draws from concepts from cognitive science to derive a method based on fuzzy similarity relations of examples on the input space. The training method learns a similarity function and selects a set of exemplars from each category used during the prediction phase to compute the similarity of new examples to the exemplars in each category and then assign it to the category with more similar examples.", "questions": "Below I provide a list of questions and points that could be improved in my opinion:\n1. Line 16: The term \"Concept cognition\" lacks reference and support in the literature. If this is a term coined in the paper, then first, it should be defined what is meant by \"Concept\" and \"Cognition\" in the views of the authors, since these terms are rather vague.\n2. Line 18: Why classification and not categorization? The former may be rather too specific and we do not know enough about human intelligence to pinpoint it as the approach used by it. For instance, see Jacob, Elin K.. “Classification and Categorization: A Difference that Makes a Difference.” Libr. Trends 52 (2004): 515-540. More specifically: do humans need supervision to learn?\n3. Line 28: It should be made clear that DNN may exceed the human level in very specific settings.\n4. Lines 29-30 and 33: The way humans do classifications is also difficult to understand and explain. Therefore, I see a conflict in pursuing human-like classifiers and classifiers that are easy to understand and interpretable at the same time.\n5. Line 30: I do not see how this can be direct and efficient because we first need to understand and explain how humans do classification and, although there exist theories, this is not settled in the literature.\n6. Figure 1: Please, describe where the numbers used in the legend are derived from.\n7. Line 74: Test data also need labels. Otherwise, the method can not be evaluated.\n8. Definition 1: Observe that \"phi(x) belongs to Y\" does not define any mapping in particular. Is that the intended definition? If so, it does not resemble a classification problem, and any arbitrary mapping would be a solution.\n9. Line 107: I do not see why an awkward situation arises from the definitions above.\n10. Equation 1: no regularization terms are considered. Is overfitting a problem?\n11. I see potential problems in the way the model chooses examples for the Ec set. Consider a class with high variability, such as \"vehicles\". In such a class, it is hard to select k exemplars that cover well the variability of the class (cars, trucks, planes, boats) and yet are similar among all of them.\n12. Line 269: \"On the contrary, the class labels predicted by NN-FLM are more in line with human cognition.\" it is hard to conclude this from the examples shown without adequate research with different subjects and in comparison with other methods.\n13. Please, provide a description of the hyperparameter tuning method used for the methods. How well were they adjusted?\n\nTypos and writing style:\n1. Line 23: possess instead of process?\n2. Line 26: humans instead of human?\n3. Line 42: \"A concept is\" or \"Concepts are\" instead of \"Concept is\" ? \n4. Line 75: The question mark is not needed because the phrase is not a question.\n5. Definition 1: In the notation used (X,Y, phi), phi seems rather an input than output or result. Consider something like phi = f(X,Y). (optional)\n6. Line 89: why double parenthesis in ((xi, xj))?\n7. Line 101: Concepts instead of Concept?\n8. Line 141: \"exemplar theory is about\" instead of \"exemplar theory is a kind of theory about\".\n9. Line 269: humans instead of human.\n", "limitations": "I do not see any limitations or potential negative social impact of this work.", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n\nThe method proposed is interesting and brings up a number of novelty elements. The method seems to improve significantly in relation to existing classification methods on a large number of data sets.\n\nWeaknesses:\n\nThe paper makes a lot of assertions about human cognition that are questionable. For instance:\n- \"In essence, the process of classification is the process of concept cognition\"\n- \"Concept contains our knowledge about the world, and we use concept to understand and organize the world. Without it, there will be no human intelligence at all.\"\n- \"Similarity (...) plays a crucial role in the process of human classification\"\n- \"Concept is represented based on similarity for children, which is also a basic choice for adults\"\n\nAlso, sometimes it is difficult to understand if the paper makes assertions about its own definitions or about human cognition as in \"the intrinsic property of concept is just the fuzziness rather than the randomness\".\n\nI do not see a problem in using assumptions based on cognitive science for building models. In fact, most models in AI do that somehow. However, care should be taken to not state these assumptions in the paper as settled truths. I rather see the paper provide in advance a list of theories, hypotheses, and assumptions considered along with references for them, and then describe the model proposed using them as a basis.\n\nFinally, without details on how well the other methods used for comparison were adjusted, it is hard to know if the comparison is fair.\n\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "VAAaNCS6n65", "reviewer": "Reviewer_55q9", "summary": "This paper proposes a new learning machine for the general classification problem, which is one of the most important problems in ML/AI. The new learning machine is based on the concept cognition theory in cognitive science and fuzzy set theory in mathematics science. So its working mechanism is highly explainable and has a solid theoretical guarantee. Meanwhile, a large number of systematic experimental results demonstrate the superiority of the proposed method.", "questions": "(1)\tThe title of the manuscript is FLM, but the performance of NN-FLM is shown in experiments. Please explain the difference between FLM and NN-FLM and why only the performance of NN-FLM is analyzed.\n\n(2)\tIt appears that the optimization model of NN-FLM is a general multi-layer neural network plus a special loss function. Please explain what good properties of the loss function are able to guarantee the performance of NN-FLM.\n\n(3)\tAccording to the descriptions in Appendix A.1, there are at least four different concept representation theories. Why choose the exemplar theory to complete the concept representation in FLM? And in some machine learning methods, ‘prototype’ is also usually used for representing a class, such as ‘means’ in ‘k-means’, ‘modes’ in ‘k-modes’, and ‘prototype ’ in ‘prototype network’. Please explain the difference between ‘prototype’ in these methods and ‘prototype’ in ‘theory theory’ and why not use prototype theory to represent the concept?\n", "limitations": "N/A", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "The manuscript focuses on the classification problem which is one of the most important problems in ML/AI.\n\nThe manuscript re-examines the classification from the perspective of concept cognition and reveals the essence of classification. And the manuscript provides a new view to interpret the structure of the classification problem by establishing the equivalence between binary classification problem and classification problem by employing equivalence relation in set theory. Furthermore, the manuscript realizes that fuzziness of concept is the main source of uncertainty in classification and then employs the fuzzy set theory to model this kind of uncertainty.\n\nBased on the above conclusions, the classification problem is modeled as a fuzzy equivalence relation problem, which well preserves the nature and intrinsic fuzziness of the classification problem. What’s more, the manuscript designs a clever model and loss function to approximate the fuzzy equivalence relation effectively and efficiently.\n\nTherefore, in this manuscript, the main proposals have the theoretical basis of cognitive science, and the key conclusions are proved mathematically. And extensive experiments (compared with 179 methods on 121 data sets) verify the rationality and superiority of the proposed method.\n\nOverall, the manuscript is clearly written and well organized with good clarity. To enhance the readability and completeness, it is suggested that some contents in the appendix should be moved to the corresponding part of the main manuscript. For example, the analysis of the working mechanism of the existing classifiers should be moved to the Introduction of the main manuscript. However, in the current manuscript, these contents are placed in Appendix A.2.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept