Human Cognition-Inspired Hierarchical Fuzzy Learning Machine
Authors: Junbiao Cui
Qin Yue
Jianqing Liang
Jiye Liang
Venue: ICML 2025 poster (2025)
Keywords: Classification
Cognitive Science
Quotient Space
Primary area: general_machine_learning->supervised_learning
TL;DR: We propose a human cognition-inspired hierarchical fuzzy learning machine,  achieving significant gains in interpretability and generalization performance.

Abstract: Classification is a cornerstone of machine learning research. Most of the existing classifiers assume that the concepts corresponding to classes can be precisely defined. This notion diverges from the widely accepted understanding in cognitive science, which posits that real-world concepts are often inherently ambiguous. 
To bridge this big gap, we propose a Human Cognition-Inspired Hierarchical Fuzzy Learning Machine (HC-HFLM), which leverages a novel hierarchical alignment loss to integrate rich class knowledge from human knowledge system into learning process. We further theoretically prove that minimizing this loss can align the hierarchical structure derived from data with those contained in class knowledge, resulting in clear semantics and high interpretability. Systematic experiments verify that the proposed method can achieve significant gains in interpretability and generalization performance.

Human Cognition-Inspired Hierarchical Fuzzy Learning Machine
Junbiao Cui
Qin Yue!
Jianqing Liang' Jiye Liang'
Abstract
cross-entropy loss widely adopted in deep neural networks (LeCun et al., 2015), etc. Despite their differences, these loss functions share common underlying assumption, i.e., the concepts corresponding to classes can be precisely de- fined. This assumption inherently implies that class relation are crispy, where any two classes are either identical or distinct, leaving no room for ambiguity.
Classification is comeestone of machine learning research. Most the existing classifiers assume that the concepts corresponding to classes can be precisely defined. This notion diverges from the widely accepted understanding in cognitive sci- ence, which posits that real-world concepts are often inherently ambiguous. To bridge this big gap, we propose a Human Iegnition-Inspired Hi- erarchical Fuzzy Learning Machine (HC-HFLM)) which leverages a novel hierarchical alignment loss integrate rich class knowledge from hu- man knowledge system into learning process. We further theoretically prove that minimizing this loss can align the hierarchical structure derived from data with those contained in class knowl- edge, resulting in clear semantics and high inter- pretability. Systematic experiments verify that the proposed method can achieve significant gains in interpretability and generalization performance.
In the classification problem, each class corresponds to concept. Unlike existing classifiers, human solves the classification problem mainly based on concept cognition (Murphy, 2004). Over the centuries, the classical theory of concept representation prevailed, positing that all concepts can be precisely defined. However, 1953, the philoso- pher Ludwig Wittgenstein challenged this proposition by exploring the concept of "game", questioning whether precise definition of such a concept truly exists (Wittgen- stein, 1953). This seminal inquiry sparked a paradigm shift, leading the philosophy community to widely accept that real-world concepts are often inherently ambiguous and resist precise definition.
Building on the above consensus, modern cognitive sci- ence has developed several influential theories for concept representation. Prototype theory (Rosch, 1975) models concept using idealized prototype that encapsulates its most representative features. Exemplar theory (Medin & Schaffer, 1978) represents concept through a set of typical and representative instances. Knowledge theory (Murphy & Medin, 1985) posits that concepts are deeply embedded within human knowledge system, asserting that human cog- nition of concept inseparable from the contextual and relational structure of this system. As the most sophisticate framework, knowledge theory not only aligns closely with human cognitive processes but also provides a compelling foundation for advancing machine learning models that aim to mimic human-liie like cognition.
1. Introduction
Classification stands of the most fundamental and extensively studied problems in machine learning. Over the years, vast array of classifiers has been developed (Delgado et al., 2014), with their methodologies broadly categorized into two main approaches. The first approach relies directly on the discrete 0-1 loss, as seen in classi- fiers such as k-nearest neighbors (Cover & Hart, 1967), decision trees (Quinlan, 1986), and naive Bayes (Domin- gos & Pazzani, 1997). etc. The second approach employs continuous functions as proxies for the 0-1 loss, including the hinge loss used in support vector machine (Cortes & Vapnik, 1995). the exponential loss adopted by AdaBoost (Freund& & Schapire, 1997; Friedman et al., 2000), and the
Recently, proposed fuzzy learning machine (FLM) inspired by concept cognition (Cui & Liang, 2022). FLM leverages fuzzy set theory (Zadeh, 1965) to capture the inherent fuzziness of concepts (McCloskey & Glucksberg, 1978; Marti et al., 2023) and employs exemplar theory to capture the typicality effects (Smith et al., 1974), achieving good interpretabilitty robustness, and generalization.
'Key Laboratory of Computational Intelligence and Chinese Information Processing of Ministry of Education, School of Com- puter and Information Technology, Shanxi University, Taiyuan 030006, Shanxi, China. Correspondence to: Jianqing Liang <liangiq exe.x.u.duu edu.e,, Jiye Liang <ljy @ sxu.edu.en..
Proceedings of the 4dnd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 20255 by the author(s).
However, FLM still faces limitations in understanding con-

---

Human Ceppittion-nssiire Hicrarchical Fuzzy Learning Machine
cepts. While it employs exemplar theory for concept rep- resentation, an approach well-suited to purely data-driven machine learning. Unlike knowledge theory (Murphy & Medin, 1985), it separates concept representation from hu- man knowledge system, limiting its ability to form pro- found understanding of concepts. Recent research in cogni- tive science further highlights that many concepts are under- stood through their relations with other concepts (Piantadosi et al., 2024).
noted D {( Vi) by E X, yt = Sc (n,) E E C
D. The goal is to use D to get fc X + y as close as possible to the unknown Lc. 4k E V, let Xx = {a; (z,, M6) E D, y: = k} be the set of training samples belonging to class k. Let X {æz (e4.9)) E D} be the set of all training samples.
Class knowledge. Let K. be human knowledge system, which contains various types of knowledge. In practice, each class in y corresponds to a concept, and the concept name corresponds to a word in natural language. Therefore, we can collect the class knowledge about y by concept name, denoted as K C K.
Meanwhile, the community has accumulated a wealth of digitized knowledge base. including the conventional knowl- edge graph, e.g., WordNet (Miller, 1995), ConceptNet (Speer et al., 2017). and the fashionable pre-trained language model, e.g. Word2Vectov (Mikolov al., 2013), Glo Ve (Pennington al., 2014), and GPT (OpenAI, 2023), etc. These resources make it feasible to develop the knowledge- driven method for concept cognition.
2.2. Preliminaries
Fuzzy similarity relation is the core notion of the proposed method. Specifically,
Definition 2.1. (Bandler & Kohout, 1988) Given a set A, the mapping F A A + [0, 1] is called a fuzzy binary relation on A.
Accordingly, this paper explores relations between concepts contained in human knowledge system and then leverages these relations to guide the learning of concepts from data, enhancing the model's ability to understand and represent concepts. The main contributions are follows.
Definition 2.2. (Bandler & Kohout, 1988) Given set A and fuzzy binary relation F on A. If F satisfies (1) reflexivity: Ya E A, R((a, a)) = 1, and (2) symmetry: Va, b E A, R((a, b)) = R[(6, a)). the is called a fuzzy similarity relation (FSR) on A.
We propose a unified modeling framework that rep- resents various types of knowledge as fuzzy similar- ity relation (FSR) on the class space, resulting in the knowledge-infused concept representation.
More preliminaries, including binary relation, fuzzy binary relation, and fuzzy quotient space are given in Appendix A.
We design novel hierarchical alignment (HA) loss in- spired by the principles of concept cognition, enabling the knowledge-infuse concept representation to guide the learning process.
2.3. Fuzzy Learning Machine
In FLM (Cui & Liang, 2022), we achieve concept cognition according to exemplar theory. Specifically,
We introduce the FSR-based quotient space theory, and then prove that minimizing the HA loss can align the hicrarchical structures derived from both data and knowledge in quotient space.
Learning. The training process of FLM is
n
min
Σ
LrP (ʃ ((x;, (S((,, (();, Mis w) (1)
11==,
where LFP and R are fuzziness permissible loss and regu- larization term, respectively. y > is trade-off parameter.
Extensive experiments verify the effectiveness of the proposed method in improving interpretability and gen- eralization performance.
I ((:)); O) is a FSR on X' with learnable parameters +.
The fuzziness permissible (FP) loss is defined as
LeP (Sij. Us. v;) = max (sij a, 0), Y/ # y
2. Notions and Related Works
(2)
max (3 Sij. 0),
y/ -
2.1. Problem Formalization
where Sty = f ((),, is the predicting similarity between H; and æj. 0 < a < B < 1 are hyper-parameters to control the degree of concept fuzziness.
Task. Given a fclacslassiiiiitooonn problem, X' C RRd, V. and fe : X + y' are input space, class space and un- known classification function, respectively. To solve the classification problem, we need to determine the unknown classification function fe- For the sake of discussion, the classes in y are numbered as .,,,....,17...
Concept representation. Let O be the local optimal solu- tion of formula (1). The FSR f ((-;)) θº) forms the basis of concept representation. According to exemplar theory, the set of exemplars of class i is defined as
Data. Let D= X' x y be data space. In machine learning, experience is often given in the form of training data, de-
æ:| E X l (1r, k) is the top-n largest value in {u (2z, k) 24 E Xx
} (3)
E =
2

---

Human Cognition-Inspiree Hicrarchical Fuzzy Learning Machine
where y (x, - J L J' 2(z; and tee is a manually specified parameter.
(0, 1I1)),) where Vi,j E V.
stj = - simli.jik), i i
==
Predicting. Given a test sample a E X, FLM predicts the class label of x according the following formula
(5)
where sim(i,]; E (0, is the similarity between classes and measured by class knowledge K. a is fuzzi- ness parameter. Appendix B.1 gives several types of class knowledge and the corresponding sim(i,,; K).
argmor KEY
Σ FFER&
f((2,2,):0")
(4)
E
To sum up, FLM captures the fuzziness of concept well. However, it ignores relations between concepts contained in human knowledge system, limiting its ability to concept cognition. This paper aims to alleviate the limitation.
In practice, the quality of class knowledge K varies signifi- cantly, so we need to further refine the class FSR. Let
case 1 K is of high quality T S(D,K), case 2 K" is of medium quality (6)
SK
3. Proposed Method
Sena ((D.)) case 3 K of low quality
The overall framework of the proposed HC-HFLM is shown in Figure 1, including method design and theory analysis. The method design is given below, and the theory analysis will be given in Section 4.
be the refined class FSR, where S is obtained by formula (5). case 1: S needs no refinement. case 2: S needs to be calibrated by training data, Appendix B.2 gives the procedure for computing ((D.K)', case 3: S needs be coarsened and then be calibrated by training data. Appendix B.3 gives the procedure for computing ((D,K)"
3.1. Method Design
We first introduce the following two definitions.
Definition 3.1. Given a E (0, 1). a set A, and a FSR S on A. If Va, E A, a + b, 0,0- < S(a, b) < a, then is called
In formula (6), T symmetrie matrix. Each row of T is dimeeennnll embedding representation of one concept, which is composed of relations between concepts. Meanwhile, T is mined from human knowledge system. Therefore, T can be regarded as the knowledge-infuss concept representation.
a-FSR on A.
Definition 3.2. Given finite set A { R. Va, E A. let sort(a;;A) A) E {1, } be the rank of a; in A by ascending order. Let b = sortvec(A) = E Rl4 be the vector formed by arrang- ing the elements in ascending order A. Obviously, the fol- lowing propositions are true.
(b1, b2. ...
b)a))" ..,...,)))))
(2) Learning Sample FSR from Data
We design novel hicrarchical alignment (HA) loss, which uses class FSR T, the knowledge-infuset concept represen- tation, to guide the learning of sample FSR from training data, enhancing model's 's ability to concept cogmition.
(1) Va,,a, az E A,i + j. a₂ aj iff sort (ag; A)
sort (az; A).
(2) Va, E A, a; = ())rt(aa:)-
(3) Vi= = 1,2, ... ,|A,, by E A and sort (b;; A) = i.
HA loss. Let T be the a-FSR obtained by formula (6). Let
(I) Mining Class FSR from Human Knowledge System
V = [tyl i,j E }},
U = V U (0,a,
According to knowledge theory (Murphy& & Medin, 1985), to understand concepts profoundly need obtain the knowledge-infused concept representation from human knowledge system K. It is well known that K. contains various types of knowledge. In this paper, we do not limit the type of class knowledge K C K. It can be the attribute deseription vector of concept name used in literature (Lam- pert et al., 2014). It can the embedding vector of concept name by pre-trained language models, such as Word2Vector (Mikolov et al., 2013), GloVe (Pennington et al., 2014), GPT (OpenAI, 2023), etc. It can be the knowledge graph, such as WordNet (Miller, 1995), ConceptNet (Speer et al., 2017), etc. It can also be human themselves.
u sortvec(U) (n,, U2, "
where 0 <
are fuzziness parameters. Y (I,, yi) (z), U1) E D. * zj. let sij = az), +) be the predicting similarity between x; and zj. Based on formula (7), the HA loss is defined as
Ur Sijs Sij S Ury
CHA(SIG, yo. (i11s19,,,100 Ho.
0,
Sij E (U Wra) (8)
Sij Uray.
Sij 2 ury
where raj = sort (L U). The underlying cognitive principles are as follows.
To ensure the universality, different types of class knowledge K are uniformly modeled as a a- a-FSR on V. i.e., S E
(a) the ideal case, samples x, and xy can perfectly repre- sent concepts y: and y/ respectively. The similarity s:y can
3

---

Human Cognition-Inspire Hicrarchical Fuzzy Learning Machine
Data Space >=
Human Knowledge System A.
Input sapco
sapco
Concept name
Different types knowledge
Method Design
Veutor
Kenoee Lede
LigoT
Learning
HA Loss
Mining
FSR on
FSR on
$= [0.1]
T=(0.1p0>1
Represent
Represent
Theory Analysis
D-representation
(D,K) representation
K X-representation
[6x)-44 Aad and 4x340) 4 (x 44) AxJ)
(leat (rieet). (don))
([awe)) (o and fh and (1s (x sill (proe (fr on 4x sill Mo Hy Aad tn ( 4o, N 15 Mo %oll
Align
Align
flot, (iger) Monel]
(font, famr, doeff
Quotient Space Q
Figure 1. The overall framework HC-HFLM
accurately depict the similarity between yi and Yj. There- fore, Stj should be equal to the similarity between concepts measured by class knowledge, i.e., 8ig 1y0..
θ R. is regularization term. y > 0 is trade-off parameter. Formula (9) can be solved efficiently by stochastic gradient descent method (Kingma Ba, 2015).
(b) In the real case, difficult that the sample perfectly represent the corresponding concept. For example, it is al- most impossible to find cat in the real world that perfectly represents the concept of "cat". The Siy cannot represent the similarity between concepts and perfectly. Therefore, Sty should be less than the similarity between concepts mea- sured by class knowledge, i.e., 8i) ty. = u. "sort(torr, :
Let @ be the local optimal solution of formula (9), then f ((,, :); 0*) forms FSR X. If loss of f (()-):0*)) is 0, then ((( O*) can effectively approximate the fuzzy equivalence relation (FER). In theory, we have
Theorem 3.3 (see Appendix C.I.1 for proof). For a ( clsssiifiii problem, Given (al) fueziness parameters
< 3 < (see formula (2))
(c) In the real case, although the similarity s,y cannot per- fectly represent the similarity between concepts y/ and yjj it hardly violates the rank. For example, it is difficullt to find cat, tiger, and a dog in the real world such that cat and dog are more similar than cat and tiger. Therefore, Sij should be too different from the similarity measured by class knowledge, i.c., $ij U (xort(bou,;:0)
(a2) training data D { M. Ma) E X,y; = fe(x,)
y) y)=r-
(a3) o-FSR T on y constructed from class knowledge K
(see Definition 3.1 and formula (6)).
(a4) the local optimal solution O formula (9) and the
FSR f ((-.)))-*) on X.
max W. wEU, arctpery
Let
{xs (I E D3 be the set of training sam- (bl) X
(d) Combining (a)-(c), when 8ty falls into the interval
ples.
[sort((uuRN -11 ceort "eort(txxw the loss is O. Otherwise, the nonzero loss should be generated.
(b2) S E [0,1]axn, Vi,j 1, n, 8ij I ((Ii, be the predicted FSR on X.
(b3) S traclo(S) be transitive-closure of (see Defini- tion A.19) and also be FER on X (see Theorem 4.1). (b4) L1 > Ej=1,f4 LUA (8ij, (8ij,Yis WiyBj) be the corre- sponding HA loss of (see formula (8)).
Training. Based on the HA loss, the class FSR T can guide the learning process. Formally, we have
min
Σ
Cha (f ((11, +,,1)) (9)
(b5) L2 = =En E5-1 LFP (Sij, Vis U;) be the corre- sponding fuzzy permissible loss of (see formula (2)). (b6) La Σn Σn E) Lrr (Bij, Vis Ub) be the corre-
(,1=1iøf
where / (() is FSR on with learnable parameters

---

Human Aegnition-Inspiree Hicrarchical Fuzzy Learning Machine
sponding permissible loss of $ (see formula (2)).
Definition 4.3. Given a finite set A and FSR on A.
2 (A, { {A/traclo (S/A)) X E [0, 1] is called the quotient space derived by S.
C = 0, then (1) C2 = 0, (2) C3 = O.
Theorem 3.3 indicates that the HA loss is more stringent
The meaning of Definition 4.3 is as follows.
than the FP loss and inherits the ability of PF loss in approx- imating FER.
(a) AA E 1), STAI the X-cut relation of S (see Definition
A.17) 17). According to Theorem A.18, SiX] is a SR on A (see Definition A.4).
(3) Concept Representation and Predicting
By replacing ((:,-)) :)) in formulas (3) and (4) with f ((.-): concept representation and predicting will be completed.
(b) traclo (S/AI) is the transitive-closure of ŞIAI (see Defi- nition A.19). According Theorem 4.1, traclo (S ER on A (see Definition A.5).
Both f (()))))") learned by FLM and f f((+-);;')
(c) A/traclo (S!A)) is the partition on A derived by traclo (SlAI) (see Definition A.6, A.7, and A.8).
learned by HC-HFLM are FSRs X. Guided by the knowledge-infused concept representation, the f ((:,);;0) integrates the class knowledge K, thus the cognition of concepts is profounder. Therefore, f ((.,)) ;))) can obtain better concept representation and predicting result.
(d) Combining (a)-(c), (A, is set of partitions on A that are derived by S.
(e) Given a finite set A and a FER T on A. AA E [0, 1), according to Theorem A.18, T(A is an ER on A. Ac- cording to Definition 19, traclo (TDI) = Tix Then (A, = { [A/r|A] À E [0, ]]] degenerate into the hier- archical structure given in literature (Zhang & Zhang, 2014).
Appendix B.4 gives the algorithm description of the pro- posed HC-HFLM.
4. Theory Analysis
The main properties of Definition 4.3 are as follows.
This section demonstrates how the knowledge-infused con- ceptual representation guides the learning process.
Theorem 4.4 (see Appendix C.2.3 for proof). Given a finite set A and FSR on A. Let V = {(, 6)) a, E A}, then QA, S) = { (A/traclo (SlA)] X V).
4.1. FSR-based Quotient Space Theory
Theorem 4.5 (see Appendix C.2.4 for proof), Given a finite sel and a FSR on A, then 2 (A, = 2 (A, traclo(S)),
Literature (Zhang & Zhang, 2014) introduces the FER-based quotient space theory. In practice, it is difficult to obtain FER directly. Hence, we leverage FSR as an alternative for modeling data and knowledge. Consequently, we need to develop the FSR-based quotient space theory.
Theorem 4.5 establishes the connection between the quo- tient space derived by FSR and FER. Due to the existence and uniqueness of transitive-closures of FSR (Bandler & Kohout, 1988), the theories developed based on FER in lit- crature (Zhang & Zhang, 2014) can be casily extended to the quotient space derived by FSR.
Literature (Bandler & Kohout, 1988) introduces the transitive-closure of fuzzy binary relation. This paper only focuses on the transitive-closure of FSR, which satisfies the following special properties.
Theorem 4.6 (see Appendix C.2.5 for proof). Given a finite set A, FSR S on A, and 1. x described in Definition A. The following propositions are true. (1) (2 (A, S). <) is
Theorem 4.1 (see Appendix C.2.1 for proof). Given a finite set A and a FSR S on A. The following propositions are true. (1) Vk
partially ordered set. VP, Q E Q (A, S), P + Q IP and
0, ..., S²" is FSR on A. (2) vk 0, I,... 52* C S2 (3) Sequence 1(k) 52", k 0, 1,,... converges to traclo(S). And traclo(S) is the transitive- closure of S. (4) traclo(S) is a FER on A.
Q are comparable under the sense of -.
Theorem 4.7 (see Appendix C.2.6 for proof). Given a finite
set A and a FSR on A. Let U = {S((a, b))l a, b E A}.
The following propositions are (rue. (1) VP, Q E (A, IfF * Q then [P] * [Q]. (2) Q (A, S)| < [A]. [f)
Based on Theorem 4.1 and the definitions of cut relation and partition (see Definition A.17 and A.8), we have the following theorem.
then EV C U, V = (A,S),, such that
Q = { ( A (4) If S satisfies transitivity, i.e., S is FER on A, then = [A(,
Theorem 4.2 (see Appendix C.2.2 for proof), Given a finite set A and FSR on A. VX E [0, 1]. traclo(S)MM (SAclo((AA).
Theorem 4.8 (see Appendix C.2.7 for proof). Given a finite set A and two FSRs R, S on A. If the ranking of elements in R is consistent with that in S, L.e. Va, c, E A,
Based on Theorem 4.2, we can construct a quotient space from a FSR as follows.
R((a, b)) = R((c, d)) iff ((aa b)) = S((,d)) {
R((a, b)) > R((c,d) iff (((a, b)) > S((c,)) : (10)
5

---

Human Irgnition-Inspirge Hicrarchical Fuzzy Learning Machine
then R)) = (A,,).
First, we align K- K-representation and (D, K -representttio by the following theorem.
Given the above properties, we can construct hierarchical structure in the quotient space derived by FSR as follows. Definition 4.9. Given a finite set A, a FSR S on A, and the < described in Definition A. According to Theorem 4.6, VP, Q E Q ( A. S), + Q2 P and Q are comparable under the sense of <. According to Theorem 4.7, 12 (A)
Theorem 4.11 (see Appendix C.2.8 for proof), Given the symbols defined in Theorem 3.3, let (Q +) and (Q (X, span (T, fer X)). <) be the K -rrepresentatton given by formula (12) and the (D, K rrepressennnn given by formula (13), respectively. Iy E 3, Br E X. such that fx(x)) = then the alignment between (2 (X, span (T. (c) 3) is as follows.
<) and
[A]. Without loss of generality, let Q (A.
[P.) 2(A.S))
1, IP; < P.41. The (2 (A, 5), <) = F1 < P2 ....... < <P/AAA))) is called hi-
and Vi = 1,2, ... 1 12 (A,5)
1 ( T) = 12 (X. span (T. X))l
erarchical structure derived by S.
(2) Vi -,,,.... (); T) the alignment re- lation between 02 and P. is (a) 40 E O, {-| E. fc(r) EO} E P, (b) vP E P, (c) |0,
4.2. Represent Data and Knowledge in Quotient Space
= PPal
Data representation. Guided by class FSR T. formula (9) learns a FSR/ f (() from D. Let S E [0,1]=x3 j 1,2, ........... n, Sty f zzz, (*) be the predict- ing FSR on training samples set X. According to Definition 4.3 and 4.9, the hierarchical structure derived by S is
Then, we align D-representation and (D, K)) -r)representtttonn by the following theorem.
Theorem 4.12 (see Appendix C.2.9 for proof). Given the symbols defined in Theorem 3.3, let (Q. x) and (2(X, span (T, Xe) -) be the D-representation given by formula (11) and the (D, (...)))rreseetation given by formula (13), respectively. If the HA loss of S is 0, then Vi 1,2,-.. .12 (X, span (T, X))], Ej e 21,2, (X, S) such that P. =Q²-
(2 [X, S) x) = [Qi < Q2 < ... - Qlexs)) (11)
denoted as D-representation.
Knowledge representation. In formula (6), class knowledge K is modeled as a FSR T on y. According to Definition 4.3 and 4.9, the hierarchical structure derived by T is
At last, we align D-representation and K K-representatton by the following corollary.
(Q, T) <) [01 < 02 ... @) (12)
denoted as K repressentation It has clear semantics and high interpretability. See Appendix B.5 for detailed analysis.
Corollary 4.13. Given the symbols defined in Theo- rem 3.3, let (2 (), z) and (Q(X,S) be the K- presentation given by formula (12) and the D-presentation given by formula (11), respectively. vy y, Er E X, such that fo(x) y. and the HA loss of S is 0, then
4.3. Align Data and Knowledge in Quotient Space
In Section 4.2, K K-representation (2(), x) and D- representation (2 (X, -) are defined on Y' and X, re- spectively. Due to the gap between Y' and X, it is not fea- sible to align these two representations directly. Therefore, we need to construct bridge between them.
(2 (), T) x)
Align
(2 (2(X, (X,, span (T, +) (14)
See Theorem 4.11
Align
problem, given a set of samples & C X C X and a fuzzy bi- nary relation F on y. Vz,, E X, let G ((x,, 2,)) = F (Ue (1:). Sc (xz))), G is called fuzzy binary relation on X that is spanned by F and for denoted as G span(F, F. for X).
Definition 4.10. For a f.)cfffsasiiiiicaaatt
(x. x).
See Theorem 4.12
Corollary 4.13 shows that minimizing HA loss can align the hierarchical structures of data and class knowledge. This means that the class knowledge contained in human knowl- edge system is effectively integrated into the learned sam- ple FSR f (())) Thus HC-HFLM can utilize human knowledge system to enhance its understanding of concepts.
For a (X fe) classsssifiiaa problem, given the set of training samples X and a class FSR T obtained by for- mula (6). According to Definition 401 4.3, and 4.9, the hierarchical structure derived by span (T, X) is
5. Experiments
(Q (X, span (T, (c),
(13)
5.1. Interpretability Analysis
[P1 < P2- x .... (XX.)xXpan(TTT...
We show the working mechanism of the HC-HFLM on the data set MNIST (LeCun al., 1998). The class space
denoted as (D, K)) repressetett......
6

---

Human Cognition-Inspaa Hicrarchical Fuzzy Learning Machine
Table 1. Class knowledge and its representation on handwritten digit classification task
121
127
C111
1220
S
S K
S(D,K)
0.
05
01
0.
95
05
5
8
8
11
6
8
0
2
6
(2 ()), S
S
y. Q
S
LD
K
consist of 10 digit characters. For the sake of display, 10 samples are randomly selected for each class to form a training data set. Ten volunteers are recruited to construct class FSR S Then the calibrated class FSR S(D.K) is obtained by formula (26). Appendix D.1 gives more experiment details. We analyze the experimental results from the following three aspects.
(d) S(,,0) is the optimal solution of formula (26)).
(o (v, s(D,) is the hierarchical structure derived by S Comparing S and (DD,K) ,te ranking of el- ements in them are the same. Comparing (2 (), )))
and (2 (V, S(D.K) ) 3), these two hierarchical structures are also the same (see Theorem 4.8).
(2) Data Representation
(I) Class Knowledge Representation
The left part of Figure 2 gives the hierarchical structure (Q (X, S) <) derived by S. S E [o, 10100x100 is the pre- dicting FSR by / O*) (2
Table shows the process of class knowledge representation. We have the following observations.
x) contains 100 partitions, which is equal to the training samples (see Theo- rem 4.7). In the finest partition Q1, each sample forms an equivalence class. In the coarsest partition Q100. all sam- ples form an equivalence class. Qu contains 10 equivalence classes. And each equivalence class is just the set of training samples of class.
(a) Sk is the class FSR constructed by 10 volunteers. (Q (V.S*).<
is the hierarchical structure derived by S It can be observed that each class forms an equiva- Ience class in the finest partition 01. In O1, the similarity between classes "6" and "9" is the biggest. They are merged to get the second partition O2.. And so on. Finally all the classes are merged into one equivalence class, resulting in the coarsest partition Oy.
(3) Alignment between Data and Class Knowledge
In Figure 2, the left part and the right part are the hierarchical structure of data and class knowledge in quotient space, respectively, i.e., (2 (X,S) <) and (2 (DD)( The alignment between them is as follows.
(b) SD is the estimated class FSR by the pre-trained f ((s,)); 0%) (see formula (25)). Comparing S and S it can be observed that the scale of S obtained from data and S obtained from knowledge is quite different. In addition, the ranking of elements is also different.
(a) the partition O2, each equivalence class corresponds to class. In the partition Q91, each equivalence class corresponds to a set of training samples belonging to class. The two correspond one by one, see the black dotted line.
(c) (Q (Y.S?). <) is the hierarchical structure derived by SD. Comparing (2 (), (D) z) and (Q (2, can be observed that the difference between S and S directly determines the difference between the hierarchi- cal structure derived by them. Therefore, the hierarchical structure can capture the difference of FSR.
(b) Based on the alignment between Q91 and C₁ the left and right equivalence classes are merged in the same way, forming coarser partitions Q92 and 02, respectively. There- fore, Q92 and C2 are aligned. Analogously, Qaa and Og.
7

---

Human Dignition-Inspirge Hicrarchical Fuzzy Learning Machine
(0(x.5).-)
(o(p.suaar) }-)
0
Figure2 2. The alignment between data and class knowledge on handwrilten digit classification task
Table 2. The basic information of data sets
DATA
CLASS KNOWLEDGE
TASK
Tree( Z2, E) DERIVED
DATA SET
SAMPLE FEATURE CLASS
CLASS DESCRIPTION VECTOR
FROM WORDNET
64-D
APY
15399
2048
32
18 LAYERS, 97 NODES
COARSE GRAINED CLASSIFICATION
DOMAIN EXPERT
768-D
IMAGENETIK
1331167
2048
1000
18 LAYERS, 1814 NODES
LLMS
85-D
AWA1
30475
2048
50
MEDIUM GRAINED
DOMAIN EXPERT
CLASSIFICATION
AWA2
37322
2048
50
85-D
17 LAYERS, 111 NODES
DOMAIN EXPERT
1024-D
FLO
8189
2048
102
N/A
FINE GRAINED
LLMS
CLASSIFICATION
CUB
11788
2048
200
312-D
N/A
DOMAIN EXPERT
Q91 and 0g are also aligned.
5.2. Generalization Analysis
(c) Based on the alignment between Qo4 and O4, the left
Data sets. This section verifies the generalization perfor- mance of the HC-HFLM on public data sets. Two types of knowledge, i.e., class description vector (CK,) and Word- Net (Miller, 1995) (CK2), are integrated into the HC-HFLM, denoted as -FFLM and CF2MH respectively. Ta- ble 2 gives the basic information of data sets, where NA denotes that the CK2 cannot be obtained for the data
equivalence classes are merged twice to get coarser par- tition Q9e. The right equivalence classes are merged once to get coarser partition Os, see the red dot line. Therefore, Qas and O5 is aligned.
(d) Based on the alignment between the partitions Qoo and Og. the equivalence classes on the left and right continue to merge the same way. Therefore, Ak = 1,2..... .4
Setting. 6 different classifiers are choose for comparison. Among them, k-nearest neighbors (KNN), decision tree (DT), support vector machine (SVM) and naive Bayes (NB) are classical classifiers. Cross entropy classifier (CEC) and FLM are based on deep neural network. Appendix D.2 gives more experiment details.
Q96+k and Os+a are aligned.
(e) According to (a)-(d), the substructure of the left hier- archical structure [Q91 .... Q91 < Q96 < Q100] is the common data-knowledge hierarchical structure (Q (X, spon (T, fe. X)), -) in formula (13), Theorem 4.11, and Theorem 4.12.
Results and analysis. Table 3 records the test accuracy of 8 methods on 6 data sets, where denotes that the result can not been obtained within 7 days. We have the following observations.
In summary, this experiment demonstrates that the HC- HFLM models the class knowledge and the traning data as hierarchical structures in quotient space and further aligns these hierarchical structures.
(a) Compared with the classifiers KNN, DT, SVM and NB,
8

---

Human Cognition-Inspire Hicrarchical Fuzzy Learning Machine
Table 3. The accuracy different methods (%)
APY
IMAGENETIK
AWA1
AWA2
FI.0
CUB
KNN
85.35
75.49
86.61
89.83
83.39
47.30
DT
63.13
63.47
70.09
42.65
21.64
SVM
84.54
84.26
89.06
86.56
43.89
NB
76.
73.66
84.67
87.68
85.53
60.27
CEC
89.08
75.62
88.48
91.67
93.58
61.49
FLM
89.42
76.02
89.95
92.79
94.05
66.19
CK: HFLM
90.23
76.16
91.10
93.59
95.06
68.78
CK HFLM
90.21
76.20
90.87
93.35
N/A
N/A
CEC achieves better performance due the powerful ex- pression ability of the deep neural networks. When the backbone network is the same, the performance of FLM is superior to CEC. This is because FLM designs more reasonable optimization objective according to the concept cognition principles.
this theory, we prove that the proposed method can align the hicrarchical structures of data and knowledge, which guarantees that the proposed method can leverage human knowledge system to enhance concept cognition. Exper- imental results validate the proposed method's ability to improve both interpretability and generalization. With its capability to learn from human knowledge system, the pro- posed method holds great promise for open-world learning tasks, such as zero-shot learning and continual learning.
(b) Compared with FLM, -HILM and CK2-HFLM achieve better performance, which shows that the concept cognition of the proposed method has been improved under the fine-
the guidance of class knowledge. Especially grained classification data set CUB, significant performance improvement is obtained by integrating class knowledge.
A Acknowledgements
This work is supported by the National Natural Sci- ence Foundation of China (Nos. 62376141, U21A20473, 62376142) and the UK.China Joint Laboratory of Security and Control on Smart Energy 202104041101020.
(c) On APY, AWAI, and AWA2, the performance of CK1- HFLM integrated with class description vector superior CKę-HFLM integrated with WordNet knowledge. On these three data sets, class description vector are constructed by domain experts. Compared with WordNet, the class description vector is more suitable for these tasks.
Impact Statement
This paper presents work whose goal is to advance the field of Machine Learning. There are many potential societal consequences our work, none which we feel must be specifically highlighted here.
(d) On ImageNerI the performance of CK2-HFLM in- tegrated with WordNet knowledge superior to the CK₁- HFLM integrated with class description vector. The class space is subset (Russakovsky et al., 2015) of words in Word- Net. Compared with the class description vector obtained by the pre-trained language model, the class knowledge contained in WordNet is more suitable for this task.
References
AlMousa, M., Benlamri, R., and Khoury, R. Exploiting non- taxonomic relations for measuring semantic similarity and relatedness in WordNet. Knowledge- Based Systems, 2 212: 212:106565, 2021.
In summary, the HC-HFLM obtains significant performance improvement benefiting from the guidance of the class knowledge. Meanwhile, the more class knowledge meets the requirements of the task, the greater the performance gain is obtained.
Bandler, W. and Kohout, J. Special properties, closures and interiors of crisp and fuzzy relations. Fuzzy Sets and Systems, 26(3):317331) 1988.
6. Conclusion
Boyd, S. and Vandenberghe, L. Convex Optimization. Cam- bridge University Press, Cambridge, UK, 2004.
Inspired by human cognition, we propose hierarchi- cal fuzzy learning machine. The method constructs the knowledge-infusedd concept representation by mining class fuzzy similarity relation from human knowledge system and employs novel hierarchical alignment loss guide the learning process. Meanwhile, we introduce the fuzzy similarity relation-based quotient space theory. Based on
Cortes, C. and Vapnik, V. Support-vector networks. Ma- chine Learning, 20(3):273-295 1995.
Cover, T. M. and Hart, P. E. Nearest neighbor pattern classi- fication. IEEE Transactions on Information Theory, 13 (1):21-27, 1967.

---

Human Argnition-Inspiree Hicrarchical Fuzzy Learning Machine
Cui, J. and Liang,.. J. Fuzzy learning machine. In Advances in Neural Information Processing Systems, pp. 36693- 36705, 2022.
Miller, G. A. WordNet: A lexical database for english.
Communications of the ACM, 38(11):39-41, 1995.
Murphy, The Big Book of Concepts. MIT Press, Cam- bridge, Massachusetts, USA, 2004.
Delgado, M. F., Cernadas, E., Barro, S., and Amorim, D. G. Do we need hundreds of classifiers solve real world classification problems? Journal of Machine Learning Research, 15(1):313333181 2014.
Murphy, G. L.. and Medin, D. The role of theories in conceptual coherence. Psychological Review, 92(3):289 316, 1985.
Domingos, P. M. and Pazzani, M. J. On the optimality of the simple bayesian classifier under zero-one loss. Machine Learning, 1032-331003-1300 1997.
OpenAL. GPT-4 technical report. https: /doi .org/ 10. 1..48550/arxX. wwt.exiiii....... 2303.08774, 2023.
Freund, Y. and Schapire, R. A decision- theorretc general- ization of on-line learning and application to boosting. Journal of Computer and System Sciences, 55(1):119- 139, 1997.
Pennington, J., Socher, R., and Manning, C. D. GloVe: Global vectors for word representation. In Conference on Empirical Methods in Natural Language Processing, pp. 1532-1543, 2014.
Friedman, J., Hastie, T., and Tibshirani, Additive logistic regression: statistical view of boosting (with discussion and rejoinder by the authors). The Annals of Statistics, 28(2):337-407, 2000.
Piantadosi, T., Muller, D. C., Rule, S., Kaushik, K., Gorenstein, M., Leib, E. R., and Sanford, E. Why con- cepts are (probably) vectors. Trends in Cognitive Sciences, 28(9):844-855 2024.
Kingma, D. P. and Ba, J. Adam: A method for stochastic optimization. In International Conference Learning Representations, 2015.
Quinlan, J. R. Induction of decision trees. Machine Learn- ing, 1(1):81-106, 1986.
Lampert, C. H., Nickisch, H., and Harmeling, S. Attribute- based classification for zero-shot visual object categoriza- tion. IEEE Transactions Pattern Analysis and Machine Intelligence, 36(3):433 465,466555556 2014.
Rosch, E. Cognitive representations of semantic categories. Journal of Experimental Psychology: General, 104(3):
192, 1975.
Rosen, K. H. Discrete Mathematics and Its Applications. McGraw-Hill Education, New York, USA, 2007.
Lastra-Diaz, J. and Garcia-Serrano, A. A novel family of IC-based similarity measures with detailed experi- mental survey on WordNet. Engineering Applications of Artificial Intelligence, 46: 112140015333 2015.
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S.,
Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M. S., Berg, A. C., and Fei-Fei, L. ImageNet large scale visual recognition challenge. International Journal of Computer Vision, 115(3):211-255) 2015.
LeCun, Y., Bottou, L., Bengio, Y., and Haffner, P. Gradient- based learning applied to document recognition. Proceed- 1998.
ings of the IEEE, 86(1
LeCun, Y., Bengio, Y., and Hinton, G. E. Deep learning. Nature, 521(7553):436 444, 2015.
Silla, C. and Freitas, A. A. A survey of hierarchical classification across different application domains. Data Mining and Knowledge Discovery, 22:31-72 2011.
Marti, L., Wu, S., Piantadosi, S. T., and Kidd, C. Latent diversity in human concepts. Open Mind, 7:79 92, 2023.
Smith, E. E., Shoben, E. J., and Rips. L. J. Structure and process in semantic memory: A featural model for se- mantic decisions. Psychological Review, 8133)2114-241 1974.
McCloskey, M. E. and Glucksberg, S. Natural categories: Well defined or fuzzy sets? Memory Cognition, 6(4): 462-472 1978.
Medin, D. L. and Schaffer, M. M. Context theory of classi- fication learning. Psychological Review, 85(3):207-238, 1978.
Speer, R., Chin, J., and Havasi, C. ConceptNet 5.5: An open multilingual graph of general knowledge. In AAAI Con- ference on Artificial Intelligence, pp. 4444-4451, 2017.
Mikolov, T., Chen, K., Corrado, G., and Dean, J. Efficient estimation of word representations in vector space. In International Conference on Learning Representations, 2013.
Wang. Y., Hu, Q.. Zhu, P., Li, Lu, B., Garibaldi, J. M., and Li, X. Deep fuzzy tree for large-scale hierarchical vi- sual classification. IEEE Transactions on Fuzzy Systems, 28(7):1395-140) 2020.
10

---

Human Cognition-Inspiree Hicrarchical Fuzzy Learning Machine
Wang, Y., Wang, Z., Hu, Q., Zhou, Y., and Su, H. Hierar- chical semantic risk minimization for large-scale classifi- cation. IEEE Transactions Cybernetics, 52(9):9546- 9558, 2022.
A. Preliminaries
A.1. Binary Relation
To unify the expression, we adopt the form of mapping to define binary relation and fuzzy binary relation.
Wittgenstein, L. Philosophical Investigations. Blockwell Publishing, 1953.
Definition A.1. (Bandler & Kohout, 1988) Given a set A, the mapping R: A A {0, } is called binary relation A.
Xian, Y., Lampert, C. H., Schiele, B., and Akata, Z. Zero- shot learning- comprehensive evaluation of the good, the bad and the ugly. IEEE Transactions on Pattern Anal- ysis and Machine Intelligence, 41(9):2251-2255. 2019.
Definition A.2. (Bandler & Kohout, 1988) Given set A and a binary relation R on A, the three properties of R are defined as:
Zadeh, L. A. Fuzzy sets. Information and Control, 8(3): 338- 338-353, 1965.
(1) reflexivity: Va E A, R((a, a)) = 1,
(2) symmetry: Va, b E A, R[(a, 6)) R((b,a))
(3) transitivity: Va, c A, if R((a, b)) = R((b,)()))))))))))))))))
Zhang, L. and Zhang, B. Quotient Space Based Problem
1, then R((a,c))= =1
Solving: A Theoretical Foundation of Granular Comput- ing. Tsinghua University Press, Beijing, China, 2014.
Obviously, transitivity
be equivalently defined as
()), Ya, b E A, [[(aa) ()) 2 max min[[[((a, c)), R((,
Definition A.3. (Bandler & Kohout, 1988) Given set A.
Let R and S be two binary relations on A.
(1) R C S iff Va, b E A, R( b)) S S((a, b)),
(2) R C S iff R C S and 3a, b A, such that R((a, 6)) <
S((a, b))-
Definition A.4. (Bandler Kohout, 1988) Given a set A and a binary relation R on A, if satisfies reflexivity and symmetry, then R is called a similarity relation (SR) on A. Definition A.5. (Bandler & Kohout, 1988) Given a A and a SR R on A, if R satisfies transitivity, then R is called equivalence relation (ER) on A.
Definition A.6. (Rosen, 2007) Given twos set A and P = (P,, ...... Pmt. m 2 1, if P satisfies the following
properties: EP Φ.
(2) UR. A,
(3) VP, E e if j. then P; P3 =
then P is called partition A. Partition is special type of quotient set.
Definition A.7. (Rosen, 2007) Given set A and an ER R on A. Va E A, [4]n [b] b E A, R((a,b)) = 1) is called an equivalence class of a derived by R.
Definition A.8. Given a set A and an ER R on A. Obvi- ously, A/ = R[ a E A} is a partition on A. And A/R is called a partition on derived by R.
Remark A.9. According to Definition A.6, A.7 and A.8. given set A, the ERs on A and the partitions on A can form a one-to-one mapping. Based it, in Proposition of literature (Cui & Liang, 2022), we model the classifi- cation problem as the equivalent relation problem on the input space, which establishes the equivalence between the classification problem and binary classification problem.
Definition A.10. Given a set and two partitions P., Q on A, VP E P. EQ E Q, such that P C Q, we call that Q

---

Human Cognition-Inspan Inspired Hicrarchical Fuzzy Learning Machine
coarser than P. denoted as P < Q. Equivalently, call that IP is finer than Q denoted as Q PP. Additionally, if IP Q and 3P E P., 3Q E Q, such that P C Q. we call that Q is coarser than P strictly, denoted as IP < Q, Equivalently, we call that is finer than Q strictly, denoted as Q > P..
According to formula (19) and (15), P Q. It contradicts with P o Q. Therefore, the original proposition is true.
In this paper, we use O to represent the partition class space V use P and Q to represent the partitions on the set of training samples X. Based on them, we use O E O, P E P, and Q E Q to represent the equivalence classes in partition O, P., and Q, respectively.
Remark A.11. According to Remark A.9 and Definition A. the following propositions are true.
(1) Given set A and two ERs R. S on A, if R C S, then
A/R < A/S.
(2) Given set A and two partitions IP, Q on A. Let R and
A.2. Fuzzy Binary Relation
S be the ERs corresponding to P and Q respectively. If
Definition A.14. (Bandler & Kohout, 1988) Given a set A, the mapping F A x A + [0, is called a fuzzy binary relation on A.
P 3 Q, then R C S.
Remark A.12. Given a set A. Let P be the set of partitions on A. Obviously, the < described in Definition A.10 satis- fies the following properties.
Comparing Definition A.I and A. the binary relation is special type of fuzzy binary relation.
(1) Reflexivity: VIP E P, P I.
(2) Antisymmetry: VP, Q C.ePPii P. Q and Q < I., then
Definition A.15. (Bandler & Kohout, 1988) Given a set A and a fuzzy binary relation on A. If F satisfies
P Q.
(3) Transitivity: VP, Q R. if Q and Q 3 R, then
(I) reflexivity: Va E A, R((a
P IR.
(2) symmetry: Va, b A, R[(a, 6)) R((b, a)).
In summary, I is a order relation on P.
F is called Fuzzy Similarity Relation (FSR) on A.
Lemma A.13. Given a set finite A. Let P and Q be two
Definition A.16. (Bandler & Kohout, 1988) Given a set A and a FSR S on A. If F satisfies transitivity: Va, E A, F( c)) 2 max min[F ((a, b)), F((b, e))], F is called a ba
partitions on A. IfIP + Q and IP Q then [P] + [Q].
Proof. Reduction absurdity.
fuzzy equivalence relation (FER) on A.
Assume that |P| Q
The following definition intuitively establishes the connec- tion between binary relation and fuzzy binary relation. Definition A.17. (Zhang & Zhang, 2014) Given set A and a fuzzy binary relation F on A, AA E [0, 1], the binary relation, Ya, b E A.
Because |P| = [Q| m, without losing generality, let P = (Pi, Pg. ..... Pm
, ). Q = {Q1, Q2.... Qm Meanwhile,
because P Q without losing generality, let
Vi = 1,, P. S Qi
(15a)
Vi = 1,2, ..,,,...... m, 1m.1 < 1Qil, iee IQ.1 - |P| 2 0. (15b)
plal b)) [[F((a, b)) ( 1, F((a, 2
(0 otherwise
Because P and Q are both partition on A, according to Definition A.6.
is called the X-cut binary relation, A-cut relation for short.
Based on Definition A.17, the following theorem further establishes the connection between FSR (FER) and SR (ER), Theorem A.18. (The proof intuitive and omitted.) Given set A and a fuzzy binary relation F on A, AX E [0, 1), let
| =
P.l.
(16a)
i=1 |A|
m
Σ 1Q
(16b)
P be the X-cul relation of F.
(1) If F is a FSR A, then F(A] is SR on A.
Subtracting the left and right sides of formula (16a) and (16b) respectively,
(2) If F is a FER on A, then F(A] is an ER on A.
Definition A.19. (Bandler & Kohout, 1988) Given a set A and fuzzy binary relation on the fuzzy binary relation G on A satisfies
m
0 Σ (1Qa) (P!)-
(17)
(1) transitivity.
According to formula (17) and (15b),
(2) F G.
(3) V fuzzy binary relation H on A, F F H. if H satisfies transitivity, then G C
Vi = 1,2, .2, IQ:1 - PP, = 0, i.e., IQil = |P]]. (18)
G is called the transitive-clouure of F, denoted as G satisfies transitivity, then
According to formula (18) and (15a).
traclo(F). ()). Obviously, traclo F) F.
Vi = 1,2, m, P: = Q,
(19)
12

---

Human
Hicrarchical Fuzzy Learning Machine
Literature (Bandler & Kohout, 1988) gives general defini- tions of P-closure. The Definition 19 is special case among them. Moreover, this paper only focuses on the transitive-closure of FSR.
fuzzy binary relation that satisfy transitivity. Therefore, we develop the FSR-based quotient space theory (see Section 4.1). Based on it, the class knowledge and data are repre- sented as hierarchical structures in the quotient space (see Section 4.2). And by designing a appropriate loss, we align these two hierarchical structures (see Section 4.3), As a result, the understanding of class concepts contained the class knowledge is integrated into the model.
Definition A.20. (Bandler & Kohout, 1988) Given set A. Let R and 5 be two fuzzy binary relation on A. The product fuzzy binary relation of It and S is defined as T = R 8 S, Va, b E A, T((a,b) b)) maz min[R((a, e)), S((c, b)]] c
Given 8, we can define the power of fuzzy binary relation as follows.
B. Details of Method
B.1. Knowledge Modeling
Definition A.21. Given a set A and a fuzzy binary relation F on A. F = F, F2 Fl s0 F, F F2 s F..., and
This section discusses different types of class knowledge and the corresponding methods for calculating class FSR.
so on.
Definition A.22. Given a finite set A and a fuzzy binary relation F on A. Without losing generality, we can sequen- tially number the elements in A as 01 42. ".", qAi- Based on it, F can be equivalently represented by a matrix, i.e.,
Algorithm Generate Class Tree Structure from WordNet, dDenoted W WordN et2ClassTree((..
Input: (1) Knowledge base WordNet.
(2) Class space y. Vk E y. let wy. be the concept name of class kt Vy,. 1/2 E Y, there is no hypernym relation between
E [0, 11 XIM), j 1.2, f1 = ((ar,
myy and 1yy-
Output: tree structure Tree (2, F)) on Y. where is the set
Definitions A. 19.9...2 are also applicable to binary relation. In this paper, we use matrices SK, SD, S(D.K), T E [0, ||>d| to represent the FSRs on class space V. use matrix S E [0, 1]mx* to represent the FSR on the set of training samples X, and use [((:))); O) represent the parameterized FSR input space X.
of nodes, E is the set of edges, and y C 2 is set of leaves.
I: Initialize the set of words corresponding to class space V:
W +- [wyly E )):
#/ (1) Mark the nodes corresponding the words in W.
2: Initialize queue: Q + {W ooo.ode..ettnn ent..ettttnnn.. Sepiiity..00 no.titt....1;; // erdily.n..01 is the common hypernym of other words.
3: while Q is not empty do
4: node <- Q.oul():
Out of the queue
A.3. Fuzzy Quotient Space
5: if node.word belongs to W then
6: Mark node and its hypemyym nodes as valid, recursively; else
Based on the preliminaries given in Appendix A.I and A.2, literature (Zhang& & Zhang, 2014) systematically develops the basebased fuzzy quotient space theory. We list the contents used in this paper as follows.
7:
8:
Mark node as invalid;
9:
end if
10:
node set +- hyponnhyponmms(; // The set of hyponyms
of node
Theorem A.23. (Zhang & Zhang, 2014) Given a set A, the
11:
for mode E node_set do
following assertions are equivalent.
12:
Q.In(node)::
|/ Entering the queue
(I) Given a FER on A (see Definition A.16). (2) Given a normalized isosceles distance d on A, where
13:
end for
14:
end while
1/ (2) Delete the invalid nodes.
normalized: Va, be E A, d(a.b) E [0,1),
15: Initialize queue: Q <- [Worddeel.eliityy...l
16:
while is not empty do
isosceles: Va,b,e b, E A, d(a,b). b). d(b, c), and d(a, c) form a an isosceles triangle and and its congruent legs are the longest side.
17: node + Qoou(()
18: if node is valid then
19:
node_set + mode.hyponymest
20:
for node E node_set do Q.In(node):
21:
(3) Given a hierarchical structure on A, ie..
22: end for
23: else
24: Delete the edge between node and its hypernym node;
a set partitions on A, and
25:
end if
26:
end while
these partitions form an ordered queue under the sense of < described in Definition A.10.
27:
Return the tree with WordNet.entity.n.00 as the root;
According to Definition A.1 FER needs to satisfy transi-
a, E Rd i }}, where a, is a description vector of class can be designed by domain experts.
= k
tivity. In practice, is relatively difficult to directly obtain
13

---

Human Dognition-Inspiree Hicrarchical Fuzzy Learning Machine
Algorithm 2 Generate Hierarchical Structure from Class Tree Structure, denoted as Class[rre2HHeSSruu
larity between classes i and can be calculated follows?.
sim(i,j;;) = Ct WWordNet (wi, wy) E a), (21)
Input: (1) Class space y.
(2) Class tree structure Tree (2, B), where 2 is the set of nodes, E is the set of edges, and y/ C Z is set of leaves.
where w; and wj are the words corresponding to class and J. respectively. WordNet (w, wy) the semantic similarity between wi and wy. And is the normalization factor.
Output: A set of partitions on y/
1: Initialize the set of partitions If <-;;
2: Initialize the partition P + 6:
3: Initialize the set of nodes NN < (''rre (2, root}:
In addition, WordNet stores the hypernym relation to repre- sent the "Is-A" relation. For example, the hypernym of 'cat' is "animal', the corresponding to the semantic is that "cat is type of animal".. Based on the hypernym relation, we can obtain the tree structure on class space by Algorithm 1.
4: while |P|| < | do
5: P' + Φb N" + d;
6: for node E .V do
7:
P' + P-P' Ufnode.lea set}; N' + N' Unode. set;
8:
9:
end for
10:
P+ P P.,N < N'
T Tree (2 F) = W WordNet2Class (WordNet }"), (22)
11: H H U(P): The duplicate partitions are delcted.
12:
end while
where 2 is the set of nodes, E is the set of edges. In this paper, we assume that there is no hypernym relation between classes in y. Obviously. y C 2 is the set of leaves. Given a Tree( Z, E) on V, we can generate FERs on y from Tree 22 E), see Algorithm 2 and
Algorithm 3 Generate FER from Class Hicrarchical Struc- ture, denoted as ClossHieStru2FFB E.....)
Input: (1) Class space V.
(2) hierarchical structure H on y.
(3) In many applications, there is natural hierarchical structure between classes (Silla & Freitas, 2011; Wang et al., 2022). The class knowledge K can be represented as a tree structure Tree(Z, E') on V, where Z is the set of nodes, E is the set of edges. If there is no hypernym relation between classes in V, then y C 2 is the set of leaves. By Algorithm 2, we can drive a hierarchical structure on y from Tree( F.), i.e.,
(3) Fuzzy parameter 0 < a < 1.0.
(4) A strictly monotonically increasing function f
f1,2,... [ - 1} (0,a).
Output: A a-FSR on V.
1: Initialize the FSR S E 10)1>>>>>
2: Sort the partitions in H according to "x" described in Defini-
tion 10 Let P1 <.... x IP #n be the sorted result;
3: for i 1.2, ... |) do
4: 8i: < 1:
5:
for j = + 1. 1
| do for = | ||| 2, ... ,1 do
6:
H CCasstree2iiSSSttttheee.a.
(23)
7:
1AAA E PR.. such that )
then
8:
S;y f [[II% D. 85c +- f (P&
9:
break;//Theorent //Theore 2.5 in (Zhang & Zhang, 2014).
Given the above hierarchical structure H on V/ according to Theorem A.23, we can obtain a a-FER on ), i.e.
10:
end if
11: end for
12:
end for
ClassHHeStruuFERR(I (see Algorithm 3). (24)
13:
end for
Let S S, which is the required a-FER on V
The following example demonstrates how to use the hyper- nym relation in WordNet to construct class FSR.
It can also be obtained from pre-trained language models. Given K, y E ), í + j. the similarity between classes and j can be calculated as follows!.
Example B.1. Given the class space y
(car, airplane, tiger, cat, dog} and WordNet 3.0. Fig- ure shows the process of sequentially calling Algorithm
sim(i.,;:) K) = a ( + a
a² a
E (0, a). (20)
1, 2, and to obtain a FSR on V
2
2 x |
(1) As shown in Figure 3(a), the WordNet tree with "entity. n.01" as the root node has total of 96308 nodes, of which
(2) Ki is knowledge graph, c.g., WordNet (Miller, 1995). Each class label corresponding to a word in WordNet. Cur- rently, a large number of methods have been proposed to cal- culate the semantic similarity between two words based on WordNet (Lastra-Díal & Gareía-Serrano 2015; AlMousa et al., 2021). Given the WordNet, Vi,j E V, # j, the simi-
74897 are leaf nodes, and the depth is 20.
(2) Input y and WordNet into Algorithm 1. By deleting nodes that are not related to V, Algorithm obtains a tree structure Tree 2, E) on ) (see Figure 3b)). Its depth is 15, and there are 31 nodes and 5 leaves. Each leaf corresponds to a class in y. In Tree 2, E), the position of classes can
In practice, we can introduce small constant e to make the value of sim(;, :)-) fall into the interval (0, a).
"Sme as
14

---

Human Dngnition-Inspinee Hicrarchical Fuzzy Learning Machine
bod kyy
oppestt
Peljerrneed jrew foc
Oyn ondeel)
layer
opuiber ofl
h. =jooseeeedd haruss fecl
F=]jer (eppeed seell
My oyelje
A-jerr jrodent
supelaad onlice
adalt
Mascaat
Aqait
(jon (undens Prencns ste
10b
sil
Your derieepue dun Peall
Foad panillepaan Jeppand dig Jeyl!
12b
Jvee ylwed Suomoe keell
15t laye
Py Soeel deppier (ien (eel
14th
sll
()] WordNit (owdNee/CCoo77r(-11
ClanTreeHiieSmm
(oSSin22E8((-...) +00)
(b) Trae(2.E)
Figure 3. The exccution processes of Algorithm 1, 2, and
reflect the semantic similarity between them. For example, the path between "cat" and "tiger" is shorter than the path between "cat" and "dog", which indicates that the semantic similarity between "cat" and "tiger" is bigger.
In the biology, "whales" and "sharks" belong to two major categories, "mammal" and "fish", respectively. Therefore, it is necessary to calibrate S obtained by formula (5) before using it guide the learning process.
(3) Next, we input (2, E) into Algorithm We show intermediate results of Algorithm between Figure 3(b) and Figure 3(c). There are different partitions among layers in T'ree E).
B.2. 1. 1. OPTIMIZATION MODELING
By solving the formula (1), FLM can learn FSR on input space X' from the training data D. Let O be the (local) op- timal solution of formula (I). We can use f ((.-);0% and training data D to estimate the similarity between classes, i.e., S E [0, 1 21, Vi e.
P = P4- (fear, airplane, tiger, cat, dog));
F3 P6 {(car airplane), (tiger, cat, dog]),
P7 {fcar), { {airplane), (tiger, cat, dog}}
Ps P13 {car)), (airplane), (tiger, cat), {dog)),
1. { EASEx, Emex M((=) [X,]x|X,
{
i=
P4 = P1s = ( [car) (airplane), (tiger), {cat], [dog]).
D 8
(25)
i i
Obviously, the above 5 partitions form ordered queue under the sense of < (see Figure 3(c)).
Then, we design the following optimization problem
(4) We input H into Algorithm 3. We show intermediate results of Algorithm between Figure 3(c) and Figure 3d)) Given the hierarchical structure H and the set of thresholds (A:)) by executing lines 3-12 of Algorithm we can obtain a FSR S on y, as shown in Figure 3(d). Obviously, S is also a FER on )
min S(D.K)
||((D.K) S
s.t.
s Sh
(D.K) (D.K)
Vi,j,k, y, sk =
(.K) > 8/1
(D.K), Vi,j,kllE) 8 > Sk1 (26)
(.K) =1 Vi
(..)) = 85
(D.K), Vi,j E),
"j (D.K)
B.2. Knowledge Calibration
0 0< 8ij < Vi,j E E0i
In practice, the sources of data D and class knowledge K are different, so the scales by which they characterize class similarity are not entirely the same. In addition, there may be certain differences in the class similarity reflected by data and knowledge (Wang et al., 2020). For example, "whales" and "sharks" have significant visual similarities.
For formula (26), we have the following statements.
(1) The objective function measures the closeness between variable (D. and D
IIn practice, small constant 0 be introduced ensure strict inequality constraints
15

---

Human
Hicrarchical Fuzzy Learning Machine
(2) The first two constraints ensure that the ranking of el- ements in optimization variable (D. is consistent with that in Sk which can maintain the hierarchical structure derived by S (see Theorem 4.8 and Definition 4.9).
may be caused by insufficient quality of K, by the char- acteristics of formula (5). It is necessary to pre-processing S appropriately before using it.
First, we introduce coarsening function food V + [0, 1] which needs to satisfy the following conditions.
(3) The last three constraints require that optimization vari- able S(D.K) is a a-FSR on ) (see Definition 3.1).
(4) It can be transformed into a standard convex quadratic programming problem (Boyd& & Vandenberghe, 2004).
Vu, U E V u < v. Scaa S Scou(u),
(30a)
3u, E K u < v, such that Scon (u)a(u) = Scoa (30b)
B.2.2. OPTIMIZATION SOLVING
Second, we use seea to coarsen S as follows.
In formula (26), the size of the optimization variable and constraint condition are O ((D)2) and O (D>)*), respec- tively. When | is large, solving it directly faces huge time costs and space costs. Fortunately, we can transform it into the following.
S E [0, 11 DI), Vi,j E V scoas, = Scoa (s.)) (31)
coa
Third, we replace S in formula (26) with the coarsening result S
coa. According to Theorem B.2, we can obtain
min
Dt - 25' Dt
min FERlVK| s.t.
r' Dr -
tEglvA s.t.
t, < 1, Vi,j 1,2,... |K| -1 (27)
r; < 1,41, Vi,j 1,2,
VK VR
1 (32)
0ct < a, vi= = ,,, [K]
0 < ri a, vi= 1,2, ...
where
where
Vk = { E, it1),
(28a)
Vk = fcoa (s/5)] i E V, it
(33a)
Vi= 1,2, ... |
Vi = 1,2, ... |vbaa
(28b)
4 = [(k,1) k, E y, sort (s VK) =
(33b)
i, = 1( |R: E ssort (Feos (s ; vVK roa))
1)
Vi 1,2, |V 55 EiS,EE DD
Σ E(priel
sD
14i
(28c)
Vi = 1,2, |Vooa 8i
Spa
(33c)
1
8 (51, S2, "(vx)) T E &
(28d)
= (š1, 82, ... Sp 1²² E 1Rl vodel,
D E R | j 1.2, ... v
(33d)
d, = ( [T, 0, i + j
i=j
(28e)
D E R v.k |x| xx|vakal, Vi, = 1,2, ...
|Vk
i i=
(33e)
dg =
According to (28), v < 1/2 1//2. In practice, class knowl- edge K is usually expressed by higher-order semantic in- formation i.e., | < D Therefore, formula (27) can be solved more efficiently. Importantly, by solving formula (27), we can obtain the optimal solution of formula (26).
0, i itj
Compared with formula (32) and (27), the size of opti- mization variables and constraints in formula (32) are both O O coal ). According to formula (30b), (31), and (33a), VK < [K In practice, by setting appropriate coarsen- ing functions feou. we can achieve | << |V*] This can not only reduce the scale of optimization problem, but also make more rational use of class knowledge K.
Theorem B.2 (see Appendix C.3.1 for proof). Let t² be the optimal solution of formula (27). Let ((D,K) E [o, 1] [])*) Vi.je y.
1.
j= i $
(D.K)'
(29)
At last, let r' be the optimal solution of formula (32). Similar to (29), we can obtain a calibrated a-FSR on y s(D.K) E [0, 10.11211111, Vi,j e V.
ij
The following propositions are true.
(1) SDD.Y is a a-FSR on y.
(2) ((DKY the optimal solution of formula (26).
1, sorr (Sorn (xk)
i i=j
coarse
i ij
(34)
B.3. Knowledge Coarsening and Calibration
(D.K)*
In practice, the S obtained by formula (5) may contain a large number of elements with small differences. These
Scoa
is not the optimal solution to the formula (26).
However, ((D.K) selectively preserves the more robust
16

---

Human
Hicrarchical Fuzzy Learning Machine
sorting information contained in S In addition, let S
When threshold À [0, we obtain the coarsest partition O3-
be obtained by formula (5). And let U $69 k, E )). According to Theorem 4.4 and 4.7, if [U] D then there are at least |4| - | redundant elements in U. Therefore, the coarsening of SK-> is reasonable.
From the above process, can seen that the ">" relation of class similarity is transformed into the "x" relation be- tween partitions in quotient space. And the set partitions forms a hierarchical structure in quotient space. According to Theorem 4.8, the same class similarity ranking can lead to the same hierarchical structure. Therefore, the hierarchi- cal structure can effectively capture the understanding of concepts contained in class knowledge K.
B.4. Algorithm Description of HC-HFLM
Algorithm 4 shows the workflow of the proposed method.
B.5. The Interpretability of Knowledge Representation
The following example demonstrates the FSR and hierarchi- cal structure derived by class knowledge.
In addition, each element in the hierarchical structure is partition on V, and each element in the partition is subset of y, both of which have clear semantics and high interpretability.
Example B.3. Given a class space y = (cat, tiger, dog}-
(1) Based on the concept names corresponding the classes, we can collect class knowledge K related to from human knowledge system V. Assuming that the class FSR con- structed from class knowledge K is
C. Proofs
C.1. Proofs of Theorems in Section 3
T((cat, cat)) (((iger, tiger)) = T((dog, dog)) =
C.1.1. PROOF OF THEOREM 3.3
T((cat, tiger)) = T'tgger cat)) = 0.7 >
T((cat, dog)) = T(dog, cat)) = 0.6 >
Proof. For proposition (1).
T (((tgeer, dog)) = T(doog tiger)) = 0.3.
For sake of discussion, we copy formula (7) as follows.
The matrix representation of T is
V [tyll i = V
0.7
0.6)
T
0.7
1
0.3
u sortvec(U) (u1, u2. (1ee
0.6
0.3
1
Because T is a-FSR on)' therefore vi E y, tu = 1 and
Vi,j E V, i j. 0 < 4ij = 1/3 < o, therefore 0 R V, a $ V. 3 Based on it, according to Definition 3.2,
(2) The view of Euclidean space. Given the class FSR matrix T, each row of T corresponds to a description vector of a concept. And the feature of every dimension corresponds to a concept, which has clear semanties. Therefore, the FSR matrix T has high interpretability.
sort (1; U) [U],
LIll 1,
(35a)
sort (B:U) |U| 1, M B,
(35b)
sort(a; U) = |UI - 2, 41l-2 =
(35c)
(3) The view of quotient space. Given the class FSR T, according to Definition 4.3 and 4.9, we can obtain the hier- archical structure derived by i.e.
sort U) 1
#1 = 0,
(35d)
Vu E V, 2 S sort(u; U) < [U].
(35e)
0 = { {cat], (tiger), {dog}} x
According to the definition of L1, L1 0, iff
(Q(IT)),>) = O2 = (cat, tiger), {dog} x
Og = [[cat, tiger, d
Liu Eij= CHA (Sij, Vs. UU) 0,
For (D,T)) <), we have the following observations.
Lib = Eij LHA (Bij, Vis M1)
When threshold À ==1 1, obtain the finest partition 0 3//tracoo (T())). In 01, each class forms equivalent class.
Because Lia = 0. according to (8), Vi.j
1, 2, ... n, i + j, = Mj. LHA (Sij, Wi. Uj) = 0, i.e.,
When threshold X = 0.7, "cat" and "tiger" are merged to obtain an equivalent class {cat, tiger). So we obtain the coarser partition 02
8ij u (soor((o,,, ()) - (t,,
Because T is a FSR on V. therefore tw = 1. According to formula (35a) and (35b),
When threshold À = 0.6, "cat" and "dog" are merged to obtain an equivalent class (cat, tiger, dog}. So we obtain the coarsest partition 03--
Wsort(1;U)=1 1100-1 soort (orrt(veensU--1 = Usart(1:17)
8
u (oot((v:0221))
= 1.
17

---

Human Cognition-Inspan Inspired Hicrarchical Fuzzy Learning Machine
Algorithm 4 Human AeggiitionIIsspieee Hierarchical Fuzzy Learning Machine, denoted HC-HFLM
Input:
Task. The (x, v. fe claassissiiin problem.
Data. The training data D = (( vi) 24; E &
(2;)) 2-) and the test data D1= {xz] xy &))=
Knowledge. The class knowlcdge K' related to the concepts in class space y/
Setting, The hyper-parameten for controlling the degree of fuzziness 0 < < B < 1. The network architecture of FLM
[((:,-);)) θ) (((((;; Q), (o; O))). The regular term R(-). The trade-off parameter 7, The number of examples required for each concept Vk E y, Nexes The way to pre-processing K, i.e. the value of "case" in formula (6).
Output: The predicting result (z,, 03) [2 E D
// (1) Pre-training FLM (see Section 2.3)
1: Initialize the learnable parameters θ randomly;
2: Using stochastic gradient descent method solve formula (1) and obtain the local optimal solution O
(2) Mining class FSR from class knowledge K (see Section
3: Initialize class FSR T +- LU:
4: Construct class FSR S from class knowledge K by formula (5) and Appendix B.1;
5: if case casel of formula (6) then
6: + S
7: else
8: Compute S from D and (((,-));" by formula (25);
9: if case case2 of formula (6) then
10:
Construct formula (27) from S and S by formula (28); Solve formula (27) and obtain optimal solution t*;
12:
Construct S (DD.K) from t² by formula (29);
13:
T ((D)))
14:
else if case case3 of formula (6) then
15:
Coarsening 5 by formula (31) and obtain S Construct formula (32) from S and S by formula (33);
17:
Solve formula (32) and obtain optimal solution r":
18:
Construct (D) from r' by formula (34);
T (DD,))
19:
20:
end if
21:
end if
(3) Learning sample FSR, guided by T (see section
22: Initialize learnable parameters <- Oº
23: Using stochastic gradient descent method to solve formula (9) and obtain local optimal solution θ";
// (4) Concept representation and prediction (see section 3)
24: Using O")) to seleet exemplar set for each class in Y by formula (3) and obtain Ex, wk )
25: Vay E Dro. using ():)); 07) and Eg, Vk E ) obtain the predicted class label ú; by formula (4);
Therefore Sij E [B, 1]. According to formula (2), fore
Lpp (8i5, Nb. Mj) = max [3 Stj, 0] 0. Therefore
Cza
CFP (Sij. Vs. Uz) = 0.
L2b Σ
LeP (Sij. Vi. Ui: = 0.
1=1 j 1i
iml j= pll.iiii
Because Lib O, according formula (8), Vi,j
1, n, + j. yi + Vj. LHA (suj, Vis 31) = O, i.e.,
Sig " (I -1 " (ort((n
Therefore, L2 L2a + L2s = 0 0. So proposition (1) is proven.
Because T is a-FSR on V. therefore 0 < tyy According to Definition 3.2 and formula (35e), we have 2-1 < sort (ty. - < sort (twiri U) < sort (a: U) and = un E " "sort((w.:::0-11 < "eort((s.n. Usort(o:U) = a, Therefore Sig E [0, a). According to for- mula (2), Lrp (Sij, Vh. 1)) mar (si a, 0] There-
For proposition (2).
Because L1 1, according to proposition (1) L22 = 0.
According to Theorem 2 in literature (Cui & Liang, 2022), C3 = 0. So proposition (2) is proven.
In summary, proposition (1) and (2) are proven.
18

---

Human Cognition-Inspiree Hicrarchical Fuzzy Learning Machine
C.2. Proofs of Theorems in Section
F((b,a)), then Ve, d E A,
C.2.1. PROOF OF THEOREM 4.1
F²((c,))
maz (i))
(I) We give some properties of the @ operation described in Definition A.20.
maz min(F((e,d)) F((c,e))
eEA
maz min(F((d,e)) F((e,c))
aEA
Lemma C.1. Given set A and fuzzy binary relation F on A. The following propositions are true.
c(d,cc)
i.c,, F² satisfies symmetry. So proposition (3) is proven.
(1) If F satisfies reflexivity, then F2 F 2 F satisfies
reflexxiityy.
For proposition (4).
1f F satisfies reflexivity, then S F F.F.
F F Va,b E A,F(((,b) 2 (F, b)) max min (F(((F)))) i,e., proposition (4) is CEA proven.
(3) F satisfies symmetry, then F22 F F satisfies
symmetry.
(4) F satisfies transitivity F² FFF
In summary, proposition (1)-(4) proven.
Lemma C.2. Given a set A and two fuzzy binary relation R, S on A. If R S, then R²2 S
Proof. For proposition (1).
Proof. Because C S, according to Definition A.3, Va, E A, R((a, b)) - S((a, b)). Therefore de A.
If F satisfies reflexivity, i.e., Va E A, a((a,a)) = 1, then
vb E A.
((e may min [R(cc e)) R((ee
((b)
eEA
max min (F c)), ((((,))))
max min(s((c.e)) S ((c,d)) A
max min min|F((b, e)), ((c,)((())] cEA,opth minn[F(b.b b), F(b, b))]
max
i.c., R² S
max min [(b,, c)), F((c,b)) }} cEA,ce
(2) Based on these conclusions, we prove Theorem 4.1.
max
Proof. Using mathematical induction prove proposition (I).
i... F2 satisfies reflexivity. So proposition (1) is proven. For proposition (2).
When k = 0, S² = = is FSR on A.
When k = 1, 1, = = S Because $ i' a FSR on A, therefore S satisfies reflexivity and symmetry. According to proposition (1) and (3) Lemma C.1 S2 satisfies reflexivity and symmetry. Therefore $2 is FSR on A.
If F satisfies reflexivity, i.e., Ve E A, F((e,c)) = 1, then Wa, 6 A,
(() mar min(F(ii,aFaaaa minIF(((..c),FF(((c.)
When k = > 1, assume that S is FSR A.
When & = m + 1, S2 S t X S Because S is FSR on A, therefore S2n satisfies reflexivity and symmetry, According to proposition (1) and (3) of Lemma C.1 52n satisfies reflexivity and symmetry. Therefore S2 is FSR on A. So proposition (1) is proven.
rEA,cya max min min F((,,)))
mar
(a)) ()),
max eEA,caa
mar
min F((,
For proposition (2).
mar min min[F((a,c)), (((c,)))))) F((a,b)) cEA.oça
V4: 0,1,2, S2 S2 @ S2* Because S is
2 F((a,b))
FSR A, according proposition (1), 52t is FSR
Therefore S²* satisfies reflexivity. According to propo-
i.c., F FCC. F2, So proposition (2) is proven.
sition (2) of Lemma C.I 52 C 52"
@ 82* S2 So
For proposition (3).
proposition (2) is proven.
If satisfies symmetry, i.e., ba A, F((a,b)
For proposition (3).
19

---

Human Aegnitioo-Inspiree Hicrarchical Fuzzy Learning Machine
First, let's prove the convergence. According to proposi- tion(1) Vt 0, ...,, ((k) to Definition 14, the sequence t(k) has an upper bound. According to proposition (2), the sequence t t(k) is monotoni- cally increasing. Therefore the sequence ((k) is convergent.
in graph G, there is path (a, and the length of path (a, b) is 1. When h
is FSR A. According
2, 52 - s Ya,b b E A, ()), 6)] = = EEA [mm)) (T[S((a,)) = A = 1|)= = in graph
(aa = = = mar
+
EEA
1] = 1 =
Second, let's prove that the sequence ((k) converges to the cransitive-closure of S. Without losing generality, assume that t(As t(ks + 1), i.e., 52 = S2 @ S2 (a) Because S2 = S²* therefore gat S S2f. Ac-
o
G, there are two edges (a, c) and (c, + in graph G, there is a path (a, c, b) and the length of path (a. (o.c,b) is 2.
cording to proposition (4) of Lemma C.I, 5 satisfies transitivity.
When h m > 2, assume that Va, b E A, STH ((a, b)) = 1
+ in graph G. there is path (a.... b) and the length of
(b) According to proposition (2), S S ge
path (a. (a..... .b) is m.
(c) A fuzzy binary relation T on A, assume that S S T and 7 satisfies transitivity. Because S C T, according to Lemma C.2
When k
m + Sm+1 = 8th
R S.
A. (a, = 1 max ((ins""((a,c)),si))
Va, b E
= 1
"EA
S CT
Se C T (36)
V (())))
= 1] 1
b
(T|STM ((a,c) 1] A I[S((c,)) b)) 1]) = 1
Meanwhile, because T satisfies transitivity, according to proposition (4) of Lemma C.1, T2 S T. According to Lemma C.2,
c
in graph G. there is a path (a. (a...., c) and the length of is m and in graph G, there is edge (c. + in graph G, there is a path (a, (a.... b) and the length of path (a.... .) is m + 1.
(a. (a..... c)
1 -
- (37)
In summary, the lemma is proven.
Combining formula (36) and (37),
Remark C.4. Comparing Definition A.1 and A.14, it can be seen that binary relation is a special type of fuzzy binary relation. Therefor, Definition A.19, A.20, and A.21 are applicable to binary relation. Based on it, Lemma C.I and C.2 are applicable to binary relation. Therefor, Theorem
5² T C T C
(d) Combining (a)-(c), 5e satisfies the three propositions in Definition A.19. Therefore 525 is the transitive-closure of S. So proposition (3) is proven.
4.1 is applicable to binary relation. According to Theorem
4.1 given a finite set A and a similarity relation (SR) S on
For proposition (4).
A, the the transitive-closure of S can also be written as S2".
According to proposition (3), traclo(S) can be written in
the form of 5²* And then, according to proposition (I), traclo(S) is a FSR on A. Because traclo(S) satisfies tran- sitivity, according to Definition A.1 traclo(S) is FER
(2) Based on the above lemma, we prove Theorem 4.2.
Proof. According to Remark C.4, 3k E {0, 1,2, ... ... such that traclo(S) S2" Because S FSR on A, ac- cording to Theorem A.18, AA [0, 1), S(Al is a SR on A. According to Remark C.4, 31 E 1,2, ...,., such that tracio trA)) SINi" Let m mar(2' 21). According to Theorem 4.1 traclo(S) = Sm and traclo(S(A)) S!Ai".
on A. So proposition (4) is proven.
In summary. proposition (1)-(4) are proven.
C.2.2. PROOF OF THEOREM 4.2
(I) We introduce the following lemma.
Therefore, to prove
Lemma C.3. Given a finite set A and SR matrix S on
AA E [0, 1], (racol(S)(A (racol(S(A)),
A. Let G = (A, S) be the unweighted graph, where A is
the set nodes and S is the adjacency matrix of the graph.
rk E 1, 2, Va, b E A, S*((a,b) (() = 1 + in graph G, there is path (a......0 b) and the length of (a....,b0 is k.
we need prove
AA E 00 11, smml1) = -!AA"
Proof. Using mathematical induction to prove.
we need to prove
When k = 1. S* = S = S. Va E A, S(a, b)) = < in graph G, there is edge (a, b) =
AE E (0, 1), Va, b E A, smal (ma = -|11" ((a,
20

---

Human Cognition-Inspiree Hicrarchical Fuzzy Learning Machine
According to Definition A.17 YA E [0, Va,b E A, smmll ((a,b) E {0, Sta"" ((a, ()) E 10, Therefore,
AA E [0, A & V. Let min v. Obviously, A < w "EV, A>d
and w E V.
we need to prove
Next we prove formula (39) in two cases. Va, b E A.
MA E [0, 11, Va,bE E A,
(a) (T b) 2 X) = i.e. S(a, b) 2 A. Because À & V and S((a, b)) E V, Therefore S(a,b) > X, Therefore
5mlh ((a, b)) = -1 + stall = (38)
S((a, 2
min v = w, i.e. T(S(a, b) 2 w) =1
Next, we prove the formula (38).
NEV セsト
Given the unweighted G = (A, SIN).. Va, b E A,
(b) (S b) 2 A) = 0, i.e. S((a b)) < X. Because À < w, therefore (( b)) < X < w. i.e. I(S(a, b) 2 w) = 0.
according to Lemma 2 of literature (Cui & Liang, 2022), ((a,)) = I [ ((a, b)) 2 X] = 1 + (ST b)) 2 x in graph G. there is a path (a,...,,) and the length of path (a..... b) is m. Mcanwhile, accord- ing to Lemma C.3, in graph G there is a path (a... (a......)) b) and the length of path (a..... b) is m = S(X) ((a. = That is to say, the formula (38) is proven. So the theorem is proven.
Combining (a) and (b), the formula (39) is true. So the theorem is proven.
C.2.4. PROOF OF THEOREM 4.5
Proof. According to Definition 4.3,
Q (A.5)
= { A/tracll(((AA) A E [0. 1]]
Q tA,troo)) A/traccoo (tracio(S) [1) X E [0.
C.2.3. PROOF OF THEOREM 4.4
{ A/trrcco(()))) A E 1].
Proof. According to Definition A.14 Va,b E A.
Therefore, to prove 2 - 2 (rcco((c)))))), ((4,)))o we need to prove AX E [o, 1), A/traclo(S(N) = A A/traclo(S))A) According toDefinition A.8. we need to prove AA e [0. 1], tracio(S!A)) = traclo(S)(A, According to The- orem 4.2, the above formula is true. So the theorem is proven.
S((a, b)) E [0, 1]. Therefore V = =(S((a, b, E A}
[0, 1]. And then, according to Definition 4.3, 2(A,5) =
{ [A/traclo (SINI) A E U[A/traado (SiNI) /A E [0,1]
- -V. Therefore, to prove
@ (A4 = { A/tracco
C.2.5. PROOF OF THEOREM 4.6
we need to prove
Proof. For proposition (1).
{ A/traclo (s XE [0.1] s {
According to Definition 4.3, (A, is a set of partitions
on A. Meanwhile, according to Remark A. < is a order relation on the partitions on A. Therefore (2 (4, S) (). :) is
A/trracoo
(siN) X }
partial ordered set. So proposition (1) proven.
we need to prove AA (0, 1]. A & V, Iw E V, such that
For proposition (2).
A/traclo (SlI)) A/traclo ( (swwl)
Let T = tracol
According to Theorem 4.5 (A, = (A, = { A/TIA X E [0, 1]]. According to Theorem 4.1, is a FER on A. According to Theorem
According to Definition A.8, we need to prove AA E [0, A & V. E such that
A. At E [0. TMl is an ER on A. YO < À < A2 S 1, obviously, T' C T According to Remark A.1I A/Tial < A/TIAi,, So proposition (2) is proven.
traclo (SIM) traclo (s/wl)
In summary, the theorem is proven.
According to Definition 19 we need to prove
C.2.6. PROOF OF THEOREM 4.7
AA 10, V, Be e V, such that S(Al = slu|.
Proof. For proposition (1).
According to Definition A.17, we need to prove AX E [0, 1 A & V, 3s E V. such that Va, b A,
Because P, Q E Q (A, S) and PP + Q, according to proposi- tion (2) of Theorem 4.6, IP and Q are comparable under the sense of x. Without losing generality, assume that IP < Q
(la b)) = T(S((a, b)) 2 = T(S((a, b)) w)) = ((a) b)).
then PP A Q. According to Lemma A.13, [PP * [Q] So proposition (1) is proven.
(39)
21

---

Human Dngmition-Inspiree Hicrarchical Fuzzy Learning Machine
For proposition (2).
For proposition (4).
According to Definition 4.3, VP E Q (A, IP is a partition on A. And then, according to Definition A.6,
According
{ A/traclo to Theorem 4.4, (
therefore |0| 2 (A,
Therefore, to prove |0| = (A, we need to prove VdL, A2 EU,1 < X2.
1 = ((H) < |P| < H(a) a E A}I |A|.
(40)
Combing proposition (1) and formula (40). Q (A, S) N
A/traclo (sthul) + A/traclo (slbel).
| So proposition (2) is proven.
Because S is a FER on A, according to Theorem A.18, Yu U. Slul is an ER on A. Therefore Stul satisfies transi- tivity. According to Definition A. traclo (slu)) SSu,, therefore we need to prove
For proposition (3). Let
C= { P 2 (A, }
(41a)
Ve E (() {u e U. A/traclo (sla)) =c}. (41b)
AY1, A2 E EUU < X2, A/S * A/S
v= max CEC] (ET(C)
(41c)
According to Definition A.8, we need to prove
V2 E U,A < X2, slAil + staal
First, we prove [ 12 (A, According to proposition
(1) CC Q |. And then, according to formula (41),
we need to prove VA1, A2 E U, < 22.
|C|= V. Therefore, |V
= [/|=1Q(4) (A)
3a, b E A, such that Si b)) & sla=)((a,)))
Second, we prove V C U. According to formula (41), V C U. Because |U] > 0 (4) = [V], therefore, V C U.
Because A1 E U. therefore, Ea, b E A, such that S((a, b)) = A1. In this case,
At last, we prove SA { A/traaoo (StAI) X V).
According to Theorem 4.4. (A,
((a b)) = [S((a, b)) 2 A1] = T[A 2 A1 =1 * slael b)) [[S((o, b)) 2 A2 II Ael =
{ (A/traclo
Therefore, we need to
prove
(A/traclo (SNN))EEU} = A/tracoo (SAI) A AEV]
therefore proposition (4) is proven.
In summary, propositions (1)-(4) are proven. So the theorem is proven.
we need to prove
{ AAttraclo ((o))\{EUff)
( (ftraa///traoooo (s(t)) E v).
C.2.7. PROOF OF THROREM 4.8
(42a)
Proof. Let
{ A/lrracoo ( A/traclo (sia1) A E v].
U = { [R((a, b))l a, b E A},
(42b)
u sortvec(U) (i,, u2 uu
V {S((a, ((((la a, E A},
Because V C U, therefore, (42a) is truc.
v = sortvec(V) = (ui, U2 )v)) TT
Next, we prove formula (42b). To prove formula (42b), we need prove Vu U, 3e E V, such that
According to formula (10),
[0] = =VV, Va, E A, sort( (o)))
(43a)
A/traclo (siu) A/traclo (slvt)
(43b)
Va,b,c, A, R((a,b)) < d)) iff b((a < (((c,d).
Yu E U, 3c E C. such that |/ttrccoo (s(ri) = c. Accord- ing to formula (41), Ee E V such that A/traclo (s(ri)|= c, i.c., Yu e U. 3v e V such that A/traclo (stul) =
(43c)
According
Theorem { A/traclo (RAI) AEU),
4.4. (A,
|A/traclo (s According to the inverse negation of proposition (1), A/traclo (S(ul)- = A/traclo (5lvl), i.e.,
Q
{ A/traclo (5/A) AE Therefore,
to prove
formula (42b) is true.
2 Q(A, R) = 2 (A, S), we need to prove
So far, formula (42a) and formula (42b) are proven. So proposition (3) proven.
A/ A/traclo (RAI) X xεU] = AA/tradoo
22

---

Human Cignition-Inspired Hicrarchical Fuzzy Learning Machine
need to prove
X described in Definition 4.10. The following propositions are true.
AAttraclo
( A E uJ (A/traclo (stNI)
A AEV] v]
(1) If S satisfies reflexivity, then span (5, fo: X) satisfies reflexivity.
(44a)
{ AA/traclo (Sti) x X xev] (A/traclo (RIN) A EU]
(2) If S satisfies symmetry, then span (S, fer X) satisfies symmetry.
(44b)
(3) If S satisfies transitivityy then span (S, for X) satisfies transitivity.
First, we prove formula (44a). To prove formula (44a), we need to prove Vu E U, 3v E V. such that
(4) AA E [o, 1}. span (S, fer fe. span (S) fcr X).
A/traclo (RlWl) A/traclo (slv))
(5) The span(;;'s) can be directly applied to binary relation and proposition (1 (-(44) are also true for binary relation.
according to Definition A.8, we need to prove Vu E U, Bu E V, such that
Proof. For proposition (1).
traclo (R(tI)= = traclo (slv:),
Because S satisfies reflexivity, i... Vy E y, S(y,,)) =
Vz E X, span (5, X) ((x,z)) = S ((f(()) =
1, i.e., span (o, X) satisfies reflexivity. So proposition
according toDefinition A.19, we need prove
(1) is proven.
Yu E U. Bu V, such that Ri stal
For proposition (2).
according to Definition A.17, we need to prove Yu E U. Ev E V. such that b E A, ((a s (l,,
Because S satisfies symmetry, i.e., Vy1,82 E y S((11, 92)) = (192, v1)). V21, E X,
spon fc, ((F1,2)) ((fo(1)), f((x2))
I(R((a, b)) u) = I((((a,)) b)) 2 v).
(45)
((fc(z)) fc(z2))
= span (S, Sc. X) (2)),
According to Definition 3.2 and formula (43a), sort(u;U) U) E f1,2,..., |U/} = f1.2. [V]], therefore, let v = ()) E V. Vu E U,
i.e. span (S, X) satisfies symmetry. So proposition (2)
is proven.
For proposition (3).
Because u E U. therefore, 3c, d E A, such that = R((c,d)) d)). According to (43b), sort(S((c, V)); sort(B((c, U) = sort(u; U)- sort(u; V), therefore (aort(u;(") (((e)((()))) S((c,)).
Because X c X, therefore Y = (Je(z) a E C
V. Because S satisfies transitivity, i.c. Vyl, Va E
V. S((11,85)) 2 mar min min[S((31 (2)), ((())))))) wEY
Next, we prove formula (45) in two cases.
V2 23 X,
(a) 1 (R((a, b)) 2 u) = i... R((a, b)) 2 R((c,d)) According to (10), b((a,b)) 2 S((c,d) = v. i.e., (S((aa, b)) 2 v) 1.
span (8, Ser X) (5),
((s (z)) fe (sa)))
maz min [S ((fo ( ((ve.
wey
(a) 6)) 2 u) = 0, ie., R((a,b) b)) <u= R((c,d)) According to(43c), we have S((a, b)) S(((, d)) u. i.e., ((S((a,)) b)) 2 v) = 0.
max min [S ((fa (x1) w2)) , ((22 fo (xa)))
y=E
maz min [S ((fe (æ1) fe (zz))) S ((fo (r2) fo (2)))))
Combining (a) and (b), formula (45) is true. So formula (44a) is true. Similarly, we can prove formula (44b). So far, formula (44) is proven. So the theorem proven.
iec span (S, X) satisfies transitivity. So proposition (3)
is proven.
0
For proposition (4).
C.2.8. PROOF OF THEOREM 4.11
AX E [0, 1], Vo1, E X,
(I) We introduce the following lemmas.
span (S, fa, x) (x11
Lemma C.5. For a -....)-laasifffaatti problem, given
I(pan (S, Jar X) (11, 2 X)
a set of samples & C. X C X and a fuzzy binary relation on y. Let spon(S S, fc, X) be the fuzzy binary relation on
= (f fc (22)) 2))
= Si (fa (x) ) fo (Iz)) = span (s|A), fcr Sc, X),
23

---

Human Dngnition-Inspiree Hicrarchical Fuzzy Learning Machine
i.e., proposition (4) is proven.
According Lemma C.6,
For proposition (5).
Vy J Fr E X, such that fc(2) y + spon (S. fer x)² span (5² fc: X) span (S, fcr X)² = span (5t, , X)
Obviously, the above proof' of proposition (1)-(4 are also valid under the sense of binary relation.
In summary, proposition (1)-(5) are proven. So the lemma proven.
span (S, fcr x)²m span (52m-1 xc. + span (5, Se, x)2 = =span (52TH Scc X),
Lemma C.6. For a (o) (.f_--)-aasffication problem, given a set of samples @ C. X S X and a fuzzy binary relation S on y. Let span(S. fc. X) be the fuzzy binary relation on X described in Definition 4.10. Vy E V,, 3x E X, such that fa(x)= = y. then span (S, X) span (52, X..
i.e., formula (46) is true. So the lemma is proven.
Lemma C.8. For a (X.Y, classsfiiiiiati problem, given
a set of samples & C X S X and an ER on Y. Let O Y R be the partition y derived by R. Let span sc, X) be the binary relation on X described in Definition 4. 10. According to Lemma C.5, spon R. X) is an ER on X. Let IP X/span R. fc. X) be the partition on X derived by span R,, fe:X) X). If E y, 3x E X such that fc(x) = y, then the following propositions are true.
Proof. Because Vy E V. 3x E X, such that fe(x) y. therefore, Y {f((()) x E X} = y I3 E X,
span (S, x)² ((F1,x33)
(1) OA E 0. [xl= (
span (S. fc) ( (111,,2))
mar min xxEX
(2) VP E P. (fc(z)) x #E E0
span (S, fey X) (2)2
max min xEE
[S fc
((fo(1z), ((dc
(3) |0 = IP]
max min [S ((fo(zz)) S (c(sa))))
=EY=Y
Proof. For proposition (1).
= (e((t)))
= =span (S², fe. X) ((11,
4O E O, obviously, & + 0 C V. Vy" E O. according to Definition A.7 and A.8.
i.e., the lemma is proven.
0 = -y'l = (v yy:yc yu:yEEJ y, ,9)) = =1.. (47)
Lemma C.7. For a (X, V, -al-ssssiiiiiaaiio problem, given a set of samples @ C Xx S X and a FSR S on y. Let span(S, X) be the fuzzy binary relation on X described in Definition 4.10. Vy E V, Ex E X, such that fx( y, then tradlo (span (c)) span (traclo (8), Sc. X).
Because wy E y. Rr E X such that fe(x) = y. Without losing generality, assume that x² E X and fx(x')= = y². And then,
[x"] (p.
{x x E X, spon (R, fc, (I),,)) =
=
= {a x X, B(ly", ff(x))) = =}
{x x E X, (,
Proof. Because S is a FSR on y, therefore, S satisfies reflexivity and symmetry. According to Lemma C.5, span (S, fe, X) satisfies reflexivity and symmetry, i.e., span (S, X) is a FSR on X.
= {o x f,((x) E [="]==O},
According to Definition A.8, (s'lspa((R.fo.)))) E P. i...
proposition (1) is proven.
According to Theorem 4.1, traclo (S) can be written
For proposition (2).
in form of S2", k 2 0, and traclo (span (S, fe, x)) can be Written in form of span (S, x)² >>
vP IP, obviously, o + P X, Vz- E P. according to Definition A.7 and A.8.
0. Let m = max (k, then traclo (S) = S²" traclo (span fc, span (S, f),
P = [a=] [epan(R. {x x E X, spon RR, fcr X) .z)) = 11
(48)
Therefore, to prove
Let Y" fc(x*), obviously, y² E EY, then
traclo (span (S, fc)) = span (traclo fc.
(v)n n = =(ylye y V.lyE RO),R(("" =1
= =(yl E,, (())()))))))))))))) y)) = =1
we need to prove
= ={f(z) I 2 (xX,,(((((('))) f(f())) = =1}
= =(f(x) x
x R, Sc. X)((x) x)) =1}
span (S, Sc, x)²" = span (5 Scs.) (46)
= -(f(x) 2E ( Lepann = P].
24

---

Human Dignition-Inspiree Hicrarchical Fuzzy Learning Machine
According Definition A.8, Wr'R E O, i.e., proposition (2) is proven.
Next, we prove formula (50). Because 2(), R) 2(' according to Theorem 4.5, 2(),, traclo( RR) Q(V, traclo(S)). Because R and S are both FSR on y. according to Theorem 4.1, traclo(R) and traclo(S) are both FER on y. Based on it, accord- ing to Definition 4.3, {)/traclo( X [0. 1]] ()/tracl((S) w E [0, 1]}- Because traclo R) and traclo(S) are both FER on V' according to Theorem A.18, Vy E [o, 1], traclo(R) and traclo(S)) I are both ER on V. According to Definition A.8,
For proposition (3).
Proposition (1) and proposition (2) actually establish a one- to-one mapping between the equivalence classes in O and the equivalence classes in IP, therefore, [O] = -IP|
In summary, proposition (1)-(3 are proven. So the lemma is proven.
Lemma C.9. For (X (c) claasiftcaatt problem, given set of samples 6 C X S X and two FSRs R, S on Y. If Q(), Rt = 20%,5 S) and vkk E V Ex E X, such that fe(x) = k. then Q (X,span span (R, X)) = Q (X, span (S, X))
(racdo(13))) A [o, ( w E [0. 10.113.
i.e., formula (50) is proven. So formula (49) is proven. So the lemma is proven.
Proof. To prove Q(X, span fe,
2X, (S, fo: X)), according to Theorem 4.5, we need to prove (X, (raclo(span (R, fa. Q (X, traclo (span (S. fe,x))), Because Vk E y.
(2) Based on these conclusions, we prove Theorem 4.11.
Er X, such that fc(x)= k, according to Lemma C.7, we need to prove
Proof. Let traclo(T) be the transitive-closure of T (see
Definition A.19). Let
2 (X, span (Iraclo [e,
(49)
Q (X, span (traclo fc.
U (Tracl((T)ss i,j E }},
(51a)
spon (traclo(T), Sc, is)) T4 E xx]. (51b)
Because R. and are both FSR on V. According to The- orem 4.1, traclo R) and traclo(S) are both FER V. According to Lemma C.5, span (traclo (ft), fe, X) and span (traclo (S), fe, X) are both FER on X. According
And then,
to Theorem A.18, Vp E [0, 1]. spon (traclo (R) fe, Xe X) and span (traclo (S), for x)l4 are both ER on X. Ac- cording to Definition 4.3, to prove formula (49), we need to prove
V span (traclo( (T). X)as 2i E EX]
(raesa)()) Fy By E EX] traclo(T) pirs Ys. Vj EV} U.
(52)
X/ X/span (traclo (R), | S/spon (traclo ( (w)
A xE10,113 10.11-
Because is a FSR on V, according to Theorem 4.1, traclo(T) FER on V. According Theorem A.18, AA [0, 1], traclo(T)) an ER on V. And then, accord- ing to Definition A.,9,
we 10.111
According to Definition A.8, we need prove
span (traclo (R) for x [A]
X [0. [0.1]=
AA [0, 1), tradlo (traclo(T)(A) traclo(T) (53)
span (traclo (s,, x) x
e 10.1
According to Lemma C.5, we need to prove
Meanwhile, because (raclo(T) is FER on y, accord- ing to Lemma C.5 span (traclo(T), f, fer X) s a FER on X. According to Theorem A.18, AA E [0, span (traclo(T), fe, x) is an ER on X. And then, ac- cording to Definition A. VA E [0, 1).
{ span (traclo (R) Scr x)
X [0. 11]
{ span (traclo (S) Sor x
i E [o. 11.
According to Definition 4.10, need prove
traclo (span (traclo (T), fcr fe, x)(1 span (tracloo)))) X)
(54)
(tracto (RT E 100.13/frrccoo
1 ftraclo (5) wE (50)
25

---

Human Lognition-Inspiree Hicrarchical Fuzzy Learning Machine
(Q (X, span (T, (c, x) =
Given V. 2 ()'; T) can be written follows
Pi X/span (racco (T) (T) Ael .Sc X 7
2 T)
(y, traclo(T))
P2= = X/span trracoo (T) [20) (T) .So= X x
(58)
//traclo traclo (T) (T) ( A [0.1]
...
y //braooo (T) E 1]
PIVI= = X/spoon (tracto (TT) .ss x)
(55)
y //traclo (T))
A XEU 1 //traalo (T) [A]
In formula (57) and (58), because T is a FSR on }, ac- cording to Theorem 4.1, traclo(T) a FER on y. And then, according Theorem A.18, Vi = 1,2, [V]. traclo (T) Au) is an ER on y. Meanwhile, because Vy E y. E E X such that fc(x) = y. Based on it, Vi = 1, 1... ...
A E v].
Given V. 2 span,pannTT, X)) can be written as follows
VV, according to Lemma C.8, O and P. satisfy the three conditions in proposition (2). So proposition (2) is proven.
@X,, spon X))
Q (X, traclo (span (T, fcr X)))
Q (X, span (traclo (T) Sc, X))
In summary, proposition (1) and (2) are proven. So the theorem is proven.
X X/traclo (span (traclo (T), fcr X) x) A [0.1]
X //spon (traclo (T) X) [A|
E (0, 1}
C.2.9. PROOF OF THEOREM 4.12
X/span (traclo (T), x
A 1.11
Proof. For sake of discussion, let T = spon (T, fer X). According to Definition 4.3 and 4.9, to prove Vi = 1,2, 2 (X, T) Fj E (1,2,... 112 (X, S)]), such that P, = Qj we need to prove VP E @X, T), EQ e
X X/span (traclo Je: X) |
A }
X/spon (traclo
fe,, fc, X |
A 10
1111
X/ X/span ttraclo (T) [1] fe; x)
\Ev].
(X,S), such that P = Q. we need to prove
(56)
@ (x, T) S (X.S).
(59)
For proposition (1).
Next, we prove formula (59). For sake of discussion, we copy formula (7) as follows
In formula (55), because (T) is a FER on V, accord- ing to Theorem 4.7, (),, T) [V. In formula (56), because spon (traclo(T), fe, X) is FER on X, according to Theorem 4.7, Q (X, span (T, scr X)) [V] Sopropo- sition (1) is proven.
V = {tal i E E),, U=VU
sortvec(U) = (u1, u2 "0o)" ,01))
According to Definition 4.10, let
For proposition (2).
V= = Ey-=pan (T, fcr (t =ty.s ú,j i.j=1... -,.... .m)
Without losing generality, we arrange the ele- ments in the set V in descending order to obtain
A1 > Ag > > lalk, According to Defini- tion A.17, Vi
and V C V. And then, according to Theorem 4.4,
|V| 1, traclo (T) [A]] c
((X,T)= = X X/truclo (TWI) X E v).
O' D/ttraclo (T) Di44l span trado (T)) Se,
C
span (traclo (rn) x).
Therefore, to prove formula (59), we need to prove AA V.
And then, ac- cording to Theorem 4.6, ///traclo (A) y //ttraloo (T) (rtt), X/span (tracoo (T) [x)] X) X/span (traclo (T) (T))) Scr x). Based on it, according to Definition 4.3 and 4.9.
Ew E [0. 1]. such that X/traclo (TI = X/traclo (S!w)) According to Definition A.8, we need to prove A\ E V.
Ew E [0, 1J. such that traclo (Tll) traclo (Sl=)). Ac- cording to Theorem 4.1, we need to prove AA E V.
Ew E [0, 1], such that, T|a = s(wl, According to Defi-
nition A.17, we need to prove VA E V, F E [0, 1]. such that Vi.j = 1,2, ing to Definition 4.10, Vi,j 1, 2....... n, Eay ty. Therefore, we need prove AA E V, FF E [0, 1). such that Vi,j = 1,2,.... nn.
n, 11 [T, 2 >] = [[[st 2 w]. Accord-
0 D//traclo (T) [Ar] +
0g D//traclo (T) (T))aa) 4
(57)
Dlv ///traclo (T) (T) [1111]
II [(Mrus 2 X]
[[8 2 w|
(60)
26

---

Human Dngnition-Inspiree Hicrarchical Fuzzy Learning Machine
Next, we prove formula (60). AA E V ç V. according to formula (35e), 2 v sort (2,U) & [T]. Therefore x sort (A, U)- -U-1=- S | Let (Aort(A,U)-1- according to formula (35d) and (35a). 0 = 41 S w < X - hlh = 1, i.e., w E [0, 1). Because In = O, therefore, vi,j = 1,2, n,, + j. CHA (sig. Ne, Myi a, T) = i.e.."
Let J2(1) (1- 8) Tz(s) = شحكك xt) mss then [1(r) = m.]2(t) + [s)s.. Substituting them into the for-
mula (62), we obtain the following optimization problem
min
[i(r) = m 8;)
s.t.
Ti =i= l441, Vi = 11,, ,
a a<t<
Sig (uoor((twm 11 sont( (mnnn
(61)
+ min
mJ2(1) + Js(s)
s.1.
a<t<b
Next, we prove that Vi,j = 1,2, ... ,,, 1yy., 2 A|
min
mJz(t)
II [sg 2 w].
s.t. a a<t<b.
(a) If j. Because T is a FSR on y. S is a FSR on X, therefore, Il 2 A| = I[1 2 ^|
1[1 2
Obviously, the last optimization problem in the above equa- tion is convex quadratic programming problem. Let t" be the optimal solution of it. Then
[sa 2 w].
(b) i + j. 1 |tness 2 1| = 1, ie., tyn,, N A. Accord- ing to formula (7), sorl Ub) 2 sorl (A; U), then sort (fway; -) -1)-12 2 sort (A; U)= 1. And then, according 2
r'
E
R Vi = 1,2, ... m,
,, r = =t,,
to formula (61), 8ij u scort( (1)) Wsort(>;U)-1
is the optimal solution of formula (62).
w, i..., [s.y 2 w| 1.
(2) Based on the above derivation process, we prove Theo- rem B.2.
(c) If * j. TI [Lu 2 2| = i.e., fuv, < X. According to formula (7), sort (tywy < sort (A;U), therefore And then, according to formula (61), Sij " ((or(()))))))) (2ort(>;0)-11 w,
sort (twwy; (i] S sort U);
Proof. For proposition (1).
According to formula (29), ((D.K) satisfies reflexivity and symmetry. Because ts is the optimal solution of formula (27), therefore,
i.e., [st, 2 w = O.
Combining (a)-(c), formula (60) is true. Therefore, formula (59) is true. So the theorem is proven.
vi= vi=1,2, 1,,,,...,VVKK,, < 17 <o.
C.3. Proofs of Theorems in Appendix B
According to formula (29). Vi.j E V,i # j. 0 < $i
(D.K) < a. Therefore, ((D,K) is a-FSR on y. So-
C.3.1. PROOF OF THEOREM B.2
proposition (1) is proven.
(I) To prove the theorem B.2, we introduce the following derivation process.
For proposition (2).
Remark C.10. Given the following convex quadratic pro- gramming problem
Because S and S both satisfy reflexivity and symmetry. Therefore, we can eliminate the third and fourth constraints and ignore the elements of the main diagonal and lower triangle positions of the matrix of the objective function in the formula (26). Therefore, we obtain the following equivalent optimization problem
min
(i) = Em (r:
81)²
s.l.
n = 1++1, Vi = 1,2, ... m -
(62)
a < ri- < b, vi = 1,2, ..2,...,m..
m, let t- ri. Mcanwhile, let D
Vi= 1,2, .... And then
Eie (D.K) (1 -0-K)=p)) -²?)²
min k),,K)
[r)
= Σ m
i=1 (r: - 8;)²
s.t.
(D.K) (D.K)
<
Vii,j, Vii <j <k
Σ
m
i= (r? - 2r;s; + s?)
<
(D.K) [,,j,e,,iij<+ (63)
Σm r² 22-22 i=1 128;
+ m
1==1
8
(D.K)
Syh
mf 2(" t +1++ 2==1 $7
mt² 2mst 1=11
jk
m 2 25t) +
0 < si
(D.K) aa Vi.j E i
11=1
= m (t- - 3)² - ms²
1=1 8
In formula (63), the S (D.)) satisfies the first two constraints iff the sorting of the triangular elements of S (DDK) is consis- tent with the sorting of the triangular elements of SK Let
*According to (8), sy should fall into the close interval. In practice we can introduce a small constant e > into formula (8) such that sy falls in the open interval.
27

---

Human
Hicrarchical Fuzzy Learning Machine
vk= = { i,j E j} Considering the situation
D. Details of Experiments
where the triangular elements of Sk have duplicates, let
D.1. Details of Interpretability Analysis Experiments
4 - {(: 1) | E V < I, sort (#f) v = = Substi-
tuting v and 1, into the formula (63), we can obtain the following equivalent optimization problem
(1) Task. To show the working mechanism of the proposed method, we conduct the experiments on handwritten digit classification task. The class space is y {0, 10.1. .9. which is familiar 10 numbers for human beings. There- fore, it is possible to evaluate the experimental results with common sense.
E E(ADEr (D.K) 1)
|
min S(D.K)
($k)
$k1
8.t.
Spy < Srs Vi,j 1,2
(D.K)_
(D.K)
|
<< (p.9) E (r,s)) 1
(64)
(D.K) = Srs , |VK
(2) Data. We adopt handwritten digit data set MNIST (Le- Cun et al., 1998), consisting of 60,000 training samples and 10,000 test samples. Each sample is a 28x28 pixel grayscale image containing a handwritten number. For the convenience of presentation, 10 samples were randomly selected for each class to form the training data set.
Spy
(p.g), (r,s) E (p.9) + (r.s)
(D.K) <a, Vi,j E v.iij..
<< 81g
Vi 1.2. V let 8, Eioores, 22 [4,1
According to Remark C.10, we can can climinate the equality constraints in the formula (64) and obtain the following equivalent optimization problem.
(3) Class Knowledge. In this experiment, we recruit 10 volunteers. each volunteer is asked to construct a 10 x 10 rating matrix that is used to measure the similarity between different digits. The score range {0, 1,2, ,103. The constructed rating matrix is averaged and normalized, de- noted as S The human knowledge system IC is the under- standing of visual features of digital characters in the minds of 10 volunteers, and class knowledge K is the rating matrix constructed by 10 volunteers.
Ji(t) = | [t; 3;)
min teglv s.t.
VK -1 (65) VK
t < 1+ Vi _,j==,,,...
0 < 44 < a, Vi= = 1,2,
Substituting s given in formula (28) and D into formula (65), we can obtain the following optimization problem
(4) Settings. Feature extraction network h 28 - R100 adopts 5-layer convolutional neural network. FSR network g : R x R - [0, 1] adopts cosine similarity. We adopt case 2 in formula (6) to obtain the final FSR T. The fuzzi- ness parameters = 0.7, 3 = 0.9. No regularization term was used in the experiment. We adopt Adam (Kingma & Ba, 2015) to solve formula (1) and (9), where the learning rate is set as 10-3, We implement the code based on Pytorch and conduct all experiments on an NVIDIA A100-PCIE-40GB GPU.
(t) = Dt - 25 Dt + D's
By removing the unrelated 5" Ds and let J2(t) = Dt
25 Dt, we can obtain the following optimization problem
min tERlvA |
Tz(t) = Dt - 25 Dt
s.t.
4 < 1,41, Vi.j = 1,2, 0 < 4 < a, Vi = 1,2, VK
VK
(66)
In formula (66), according formula (28), D is a symmet- ric positive definite matrix. Therefore (t) is a convex function w.r.t. t. Meanwhile, only linear inequality con- straints are involved in the constraint conditions of formula (66). Therefore, formula (66) is a standard convex quadratic programming problem. Let 1 be the optimal solution of for- mula (66). And then, let S(D.K)" E <R>I,, Vi.j E V, we have
D.2. Details of Generalization Analysis Experiments
To verify the effectiveness of the proposed method, we conduct experiments on 6 public data sets compared with 6 different classifiers.
(I) Task. APY and Image-NetIK K data sets contain 32 and 1000 classes respectively, involving different classes such as animals, furniture and vehicles. The differences among the classes are large, which are soarse-grained classification tasks. AWA1 and AWA2 data sets share the same class space, which contains 50 species of animals, such as horses, whales, tigers, etc. There are certain differences between the classes. Thus, they are medium granularity classification tasks. FLO and CUB data sets contain 102 different species of flowers and 200 different species of birds, respectively.
(D.K)' $ij
-
-
1. e
i= i j
Obviously, (D, satisfies the all constraint conditions of formula (26). Meanwhile, according to Remark C0 S(D,K) is the optimal solution of formula (26). So proposi- tion (2) is proven.
In summary, proposition (1) and (2) are proven. So the theorem is proven.
'nttps://pytorr org/
28

---

For each method, different parameter settings are adopted and the best experimental results are recorded. For the first four methods, we adopt the implementations by the Scikit-learn library". The remaining methods are imple- mented based on Pytorch. We conduct all experiments on an NVIDIA A100-PCIE-40GB GPU. More details are listed in Table 4. Human Cognition-Inspired Hicrarchical Fuzzy Learning Machine
Table 4. The comparison methods and their settings
Method
Abbreviation
Setting
K-Nearest Neighbor
KNN
We adopt Fuclidean distance. The number of nearest neighhors is selected within 5, 7.9.111
We adopt two eniteiinn, T.c., Entropy and Gini, to select split feature, We adopt stratogles, I..., global search and random split, to determine the split threshold.
Decision Tree
DT
We adupt one ress to deal with mulipple classilication problem. We adopt Caussian kernel lunction the parameters kernel function are default. The trad-offf parameters selected within {0.01, 0.1, 1. 10, 100).
Support Vector Machine
SVM
We adopt lour ditlerent distribution ussumptions to estimate the probubiliit density, Le., Gaussian, Multinorial, Complement
Naive Bayes
NB
and Bernoulli.
Feature extraction network is designed as h(x; 6) = RELU (ReLU (xW1 + J + by) Wa + bg. where è = [W, E R2248x1024 by E R4024, Wa E R Rl024x512 x122 by E R812, Wa E R612x 1"1, by E R is the set of leamable parameters. We adopt cross entropy loss Softma (h(x)) and true labels train network. We adopt Adam (Kingma & Ba, 2015) as optimizer. The size of batch set 256. The number of epoch is set to 200. The initial leamning rate is set to 1o-3 and decays by 0.5 every 20 spochs.
Cross Entropy Classifier
CBC
x Tol (8)li!!!!! where yo (-) @) = Sigmoid (h (-: @)), h is the same as in CEC. We adopt Adam (Kingma & Ba, 2015) as optimizer. The size of batch is set to 1000. The number of spoch is set to 200. The initial learning rate Regularization term is not used in the experiment.
The overall network is f ((++) :))) = Tw =(.ee) p(..))
Fuzzy Learning Machine
FLM
to and decays by 0.5 times every 50 epochs. The fuzziness paramoters
0.2, B 0.8.
The overall network the same as FL.. The pretrained FLM used the initaal point. We adopi Adam (Kingma Ba, 2015) as oprimizer. The size of batch set to 1000. The number of epoch is set to 50. On ImageNet1 K data set, the initial learning rate is set to 1c-9. On the remaining data sets, the initial learning rare is sct to 1e-3. The learning rate decays by 0.5 times every 30 epochs. For class description vector, adopt formula (20) in Appendix B.1 calculate the FSR S We adopt case 3 in formula 6 to get final FSR T. On ImageNetIK K data set, S R is coarsened as 50 levels. On the remaining data sots, S is coarsened 100 levels.
HC-HFLM with K₁
-FFM
Difierent from CK2 -HFLMM Algurilhu 1, and 3 called successivelyy to get FSR S (see Appendix B. The number of levels is determined by knowledge graph (see Table 2). Based on it, we adopp case2 in formula 6 to get the final FSR T.
HC-HFLM with K₂
CK2-HFLM
There are small differences between classes. Thus, they are fine-grained classification tasks.
Appendix B.1) are called successively to get the class FSR
S The class spaces of FLO and CUB data sets are the sets of different followers and different birds, respectively. This makes difficult to match with WordNet and take advantage of the knowledge provided by WordNet.
(2) Data. For ImageNet1 K data set, we adopt common data partition, which contains 1,281,167 training samples and 50,000 test samples. We record the test accuracy of all meth- ods. The remaining 5 data sets are evaluated using 5-fold cross-validation for each method, and the mean accuracy are recorded.
(4) Comparison Methods and Settings. In this experiment, we adopt 6 different classifiers comparison methods. K- nearest neighbor (KNN), decision tree (DT), support vector machine (SVM), and naive Bayes (NB) are classical and non-deep neural network methods. Cross entropy classifier (CEC) and FLM are methods based on deep neural network. Because the input features are extracted by the pre-trained deep neural network, we adopt fully-connected layers as backbone for CEC and FLM.
As is known to all, the way of extracting features from im- ages seriously affects the performance of image classifica- tion. The focus of this paper is how to integrate knowledge to improve the classification performance rather than de- signing a better method of extracting features. In order to eliminate the influence of the feature extraction process, the 2048 dimensional features extracted by literature (Xian et al., 2019) are used in the whole experiments for all data sets. The detail information of data sets is summarised in Table 2.
In the proposed HC-HFLM, two types of knowledge K₁ and K₂ are integrated into FLM, respectively, denoted as CK₁- HFLM and CK2-HFLM. For fair comparison, the network architecture is the same as the FLM.
(3) Class Knowledge. In this experiment, we adopt two types of knowledge, denoted Ki and K2.
Kj represents the class description vector. For ImageNet| data set, the class description vector is obtained by Glo Ve (Pennington et al., 2014). For the remaining 5 data sets, the class description vectors are designed by domain expert. The detailed descriptions can be found in literature (Xian et al., 2019). We adopt formula (20) in Appendix B.1 to calculate class similarity relation S
K2 represents knowledge graph. WordNet (Miller, 1995) is adopted in this experiment. Algorithm I, and 2 and 3 (see
Sttpps//ssikttt-eem.orgg//
html
29


Reviews:
[{"review_id": "FEsDD1kUAU", "reviewer": "Reviewer_CTni", "summary": "This paper advocates solving classification problems from the perspective of concept cognition. Inspired by human cognition, this paper utilizes the relationships between concepts embedded in the human knowledge system to guide the learning process. This deepens the model's understanding of concepts. In addition, this paper develops the fuzzy similarity relation-based quotient space theory and then analyses the working mechanism of the proposed method. Meanwhile, the experimental results demonstrate that the proposed method can improve the generalization performance by incorporating human knowledge system.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "# 0. General Response\n\nWe sincerely appreciate the reviewer’s positive feedback and valuable comments. Below, we provide a point-by-point response to each comment.\n\n# 1. Response to “Weakness”\n\n- In the experiments, to directly highlight the performance improvement achieved by incorporating class knowledge contained in human knowledge system, the proposed method does not include the regularization term.\n- In practical applications, appropriate regularization term can be introduced based on domain knowledge, which is expected to further enhance performance.\n\n# 2. Response to “Other Comments Or Suggestions”\n\nTo improve the overall coherence and readability, we will incorporate the relevant content into the main paper in the final version.\n\n# 3. Response to “Questions For Authors”\n\nThe primary goal of the experiments is to demonstrate that the proposed method improves model performance by incorporating the class knowledge contained in human knowledge system rather than merely aiming for higher accuracy. As the reviewer pointed out, employing more complex and advanced neural networks could potentially yield better accuracy. However, the subject lies beyond the scope of this paper.", "claims_and_evidence": "The main claims in this paper are supported by theoretical analysis.", "methods_and_evaluation_criteria": "The proposed method is effective for aligning data and class knowledge and then improving the generalization performance.", "theoretical_claims": "I have checked the main conclusion, i.e. Corollary 4.13. The relevant proof process should be correct. Meanwhile, Figure 2 in Section 5.1 provides a typical example of this conclusion.", "experimental_designs_or_analyses": "The experimental results in Section 5.2 demonstrate that the proposed method can deepen the model's understanding of concepts by incorporating human knowledge system, thereby improving the model's testing accuracy.", "relation_to_broader_scientific_literature": "Many practical problems can be abstracted as aligning two pieces of information from different sources. Most existing methods achieve the alignment in Euclidean space. Unlike it, this paper achieves the alignment of two pieces of information in quotient space, and this alignment has clear semantics and strong interpretability. Therefore, the proposed method provides a new promising approach for aligning information from different sources.", "essential_references_not_discussed": "The references are complete and sufficient for understanding this paper.", "other_strengths_and_weaknesses": "Strengths:\n\n1.\tIn human cognition, there are rich relationships between concepts. Inspired by this, this paper integrates these relationships between concepts contained in human knowledge into the model, which improves the interpretability and generalization performance. It provides a new inspiring way to develop human cognition-inspired machine learning methods.\n\n2.\tThis paper has a solid theoretical analysis, which guarantees the effectiveness of the proposed method.\n\n3.\tThe proposed method models different types of knowledge as fuzzy similarity relation on class space, which has good universality.\n\nWeaknesses:\n\n1.\tIn formula (9), the specific form of regularization term R (\\Theta) and the value of the hyper-parameter \\gamma are not given, which affects the reproducibility of the experiments.", "other_comments_or_suggestions": "In order to enhance the readability of this paper, some content in the Appendix should be placed in the main text, such as the definition of “finer” and “coarser”, and the definition of “cut-relation”, etc.", "questions_for_authors": "Section 5.2 adopts fully connected networks as the backbone network for the deep neural network methods, the fuzzy learning machine, and the proposed method. As is known to all, the fully connected network is the simplest network structure. Why not", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "jygpsOuKGc", "reviewer": "Reviewer_DRfi", "summary": "The authors propose a human cognition-inspired classifier. The method first mines the fuzzy similarity relation between concepts from human knowledge system. And then the authors design the hierarchical alignment loss based on the principles of concept cognition. Using this loss, the fuzzy similarity relation between concepts guides the learning process. At the same time, the authors demonstrate that minimizing the hierarchical alignment loss can achieve hierarchical structures in class space and sample space, thereby aligning data and knowledge in the quotient space. Finally, the experiments verify the advantages of the proposed method in interpretability and improving the generalization performance.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "# 0. General Response\nWe thank reviewer for the appreciation of our work and valuable comments. Below, we provide a point-by-point response to each comment.\n\n# 1. Response to “Weakness”\n\n- The fuzzy equivalence relation (FER) is a more restrictive form of the fuzzy similarity relation (FSR). \nIn practical applications, obtaining the FER is generally more challenging than obtaining the FSR. \nTherefore, we employ the FSR to model class knowledge and sample similarity. \n\n- The existing fuzzy quotient space theory is based on the FER and is therefore not suitable for analyzing the method proposed. In response, we develop a quotient space theory based on the FSR.\n\n- **Theorem** 4.5 establishes the connection between the quotient spaces derived by FSR and FER, which allows the existing \n theoretical results in FER-based quotient space to be extended to FSR-based quotient space.\n\nWe will incorporate the above content in the final version.\n\n# 2. Response to “Other Comments Or Suggestions”\n\n1. In the final version, we will incorporate the relevant content into the main paper to improve the overall coherence and readability.\n\n2. In the final version, we will enlarge the figures in this paper to enhance readability and provide greater convenience for the readers.\n\n# 3. Response to “Questions For Authors”\n\nUnlike fuzzy learning machine, the proposed method further incorporates the class knowledge contained in human knowledge system into the sample fuzzy similarity relation. As a result, the proposed method enhances the model's understanding of concepts, leading to the performance gain.", "claims_and_evidence": "The claims in this paper are supported by clear and convincing evidence.", "methods_and_evaluation_criteria": "The proposed method makes sense for the problem at hand. So dose the evaluation criteria.", "theoretical_claims": "I checked the relevant proofs. The authors provide detailed proof steps. Given these proofs, the conclusion of the paper can be verified. Meanwhile, the lower part of Figure 1 presents a clear example. As stated in the main conclusion i.e., Corollary 4.13, the hierarchical structure corresponding to data is aligned with the hierarchical structure corresponding to class knowledge.", "experimental_designs_or_analyses": "The experimental results in Section 5.1 verify the theoretical conclusions in Section 4. By minimizing the proposed hierarchical alignment loss, the hierarchical structure on the training set is indeed aligned with the hierarchical structure on the class space.", "relation_to_broader_scientific_literature": "The proposed method achieves the alignment between hierarchical structures of data and knowledge. What's more, each component in these hierarchical structures has clear semantics, so this alignment has strong interpretability. In many practical applications, aligning data and knowledge is a common requirement. Therefore, the proposed method is expected to achieve successful applications in more scenarios.", "essential_references_not_discussed": "As far as I know, no key reference is missing.", "other_strengths_and_weaknesses": "**Strengths**\n\n1.\tThe proposed method is novel. The authors use cognitive science principles to guide the design of the model, improving its interpretability and providing a new solution for the development of humanoid intelligence.\n\n2.\tThe theoretical analysis is sound. The authors propose a new hierarchical alignment loss to align training data and class knowledge, and this loss has the theoretical guarantee in aligning data and knowledge.\n\n3.\tThe experimental results are sufficient. The authors not only provide an example of hierarchical alignment in the quotient space, but also verify the effectiveness of the proposed method on the public datasets.\n\n**Weaknesses**\n\n1.\tThe authors develop the fuzzy similarity relation-based quotient space theories. In order to help readers understand the thought process of the paper, the connection between these new theories and existing quotient space theories as well as the process of developing these new theories should be further discussed.", "other_comments_or_suggestions": "1.\tIn order to make readers better follow the idea of the paper, some content in the Appendix should be adjusted to the main text, especially the relevant processing procedures involved in formulas (5) and (6).\n\n2.\tThe font in Figures 1 and 2 should be enlarged. The same issue also appears in Figure 3 of the Appendix.", "questions_for_authors": "As stated in the paper, the fuzzy learning machine also solves the classification problems from the view of concept cognition. Why can the proposed method significantly improve the performance of the fuzzy learning machine?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "qw7kLjh9Zj", "reviewer": "Reviewer_2rzm", "summary": "In general terms, the paper presents a new and innovative method called Human Cognition-Inspired Hierarchical Fuzzy Learning Machine (HC-HFLM), which is aimed at improving interpretability and performance in classification tasks. This method is based on human cognition and takes into account the ambiguity present in real-world concepts, which cannot always be precisely defined.\n\n## Update after rebuttal:\nThank you to the authors for their detailed responses; my doubts and questions have been resolved.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "# 0. General Response\nWe thank reviewer for the appreciation of our work and valuable  comments. Below, we provide a point-by-point response to each comment.\n# 1. Response to “Methods And Evaluation Criteria” \n(1) We add the standard deviation of accuracy for all methods based on 5-fold cross-validation, except for ImageNet1K dataset. The experimental results are as follows. \n||APY|AWA1|AWA2|FLO|CUB|\n|:--:|:--:|:--:|:--:|:--:|:--:|\n|KNN|85.35$\\pm$0.46|86.61$\\pm$0.51|89.83$\\pm$0.28|83.39$\\pm$1.00|47.30$\\pm$1.07|\n|DT|63.13$\\pm$0.76|63.47$\\pm$0.45|70.09$\\pm$0.56|42.65$\\pm$0.81|21.64$\\pm$1.07|\n|SVM|84.54$\\pm$0.55|84.26$\\pm$0.56|89.06$\\pm$0.12|86.56$\\pm$1.26|43.89$\\pm$0.74|\n|NB|76.13$\\pm$1.06|84.67$\\pm$0.70|87.68$\\pm$0.18|85.53$\\pm$1.22|60.27$\\pm$0.27|\n|CEC|89.08$\\pm$0.72|88.48$\\pm$0.14|91.67$\\pm$0.35|93.58$\\pm$0.91|61.49$\\pm$2.47|\n|FLM|89.42$\\pm$0.36|89.95$\\pm$0.29|92.79$\\pm$0.31|94.05$\\pm$0.83|66.19$\\pm$0.92|\n|**CK$_1$-HFLM**|**90.23$\\pm$0.42**|**91.10$\\pm$0.28**|**93.59$\\pm$0.17**|**95.06$\\pm$0.65**|**68.78$\\pm$0.27**|\n|**CK$_2$-HFLM**|90.21$\\pm$0.32|90.87$\\pm$0.27|93.35$\\pm$0.26|N/A|N/A|\n\nThe results demonstrate that the proposed method outperforms all compared methods and exhibits a small standard deviation, indicating the stability of the performance.\n\n(2) For binary classification, the hierarchical structure on class space is unique. In this case, the proposed method degenerates into fuzzy learning machine, where class knowledge only guides the selection of fuzzy parameters. Additionally, when the quality of class knowledge is poor or not relevant to the task, the performance gain brought by the proposed method will be limited.\n# 2. Response to “Experimental Designs Or Analyses”\n(1) For fairness in comparison, the 2048-dimensional features and class description vectors used for all datasets are sourced from (Xian et al., 2019) without any additional preprocessing.\n\n(2) See item (1) in Section 1.\n\n(3) Due to time constraints, we conduct label noise experiments on the APY dataset. We introduce label noises with 4 different ratios, i.e., 5%, 10%, 15%, and 20%. For each ratio, label noises are added randomly and the process is repeated 5 times. The experimental results are as follows.\n||5%|10%|15%|20%|\n|:---:|:---:|:---:|:---:|:---:|\n|KNN|85.17|84.80|84.22|83.47|\n|DT|59.05|56.22|52.91|49.20|\n|SVM|77.44|76.57|75.70|74.72|\n|NB|75.07|74.72|74.46|74.17|\n|CEC|88.81|87.42|85.79|83.94|\n|FLM|88.93|86.57|85.70|85.57|\n|**CK$_1$-HFLM**|**89.75**|**89.26**|**88.62**|**88.02**|\n|**CK$_2$-HFLM**|89.61|89.17|88.53|87.87|\n\nThe experimental results show that as the label noise ratio increases, the proposed method experiences the least performance degradation, highlighting its robustness against the label noise.\n# 3. Response to “Essential References Not Discussed”\nThank you for the important literatures. Recent methods have successfully integrated structured knowledge with neural networks for knowledge graph representation and completion task. We will add the related discussions in the final version.\n# 4. Response to “Weaknesses”\n(1) See item (3) in Section 2.\n\n(2) See item (1) in Section 1.\n\n(3) See item (2) in Section 1.\n\n(4) We report the average time (s) for running one epoch (mean over 50 epochs) for CEC, FLM, and the proposed method. Due to challenges in utilizing GPU acceleration for the other comparison methods, their running times are not comparable. All experiments are conducted on an NVIDIA A100-PCIE-40GB GPU, and the results are as follows.\n||FLO|CUB|APY|AWA1|AWA2|ImageNet1K|\n|:---:|:---:|:---:|:---:|:---:|:---:|:---:|\n|CEC|0.29|0.33|0.51|0.99|1.10|88.29|\n|FLM|0.37|0.52|0.72|1.33|1.63|86.17|\n|**CK$_1$-HFLM**|0.78|1.12|1.55|2.92|3.52|169.50|\n|**CK$_2$-HFLM**|N/A|N/A|0.94|1.77|2.18|113.61|\n\nCEC computes the loss based on single sample, while FLM calculates the loss based on sample pair, leading to a longer runtime compared to CEC. The proposed method builds on FLM by incorporating class knowledge and further refining class similarities into different levels, leading to a longer runtime than FLM. Overall, the runtime of the proposed method is approximately 2-3 times that of CEC, which remains within an acceptable range.\n\n# 5. Response to “Other Comments Or Suggestions”\nThank you for your thoughtful correction. We will incorporate the corresponding modifications in the final version. Additionally, N/A denotes the absence of the corresponding type of knowledge for these datasets, and we will include this explanation in the main paper.\n# 6. Response to “Questions For Authors”\n(1) The proposed method effectively models the hierarchical structure, positioning it as a promising approach for knowledge graph representation learning. It is also applicable to structured prediction tasks, such as ranking and hierarchical classification. Additionally, it provides an interpretable solution for aligning two information sources, making it well-suited for multimodal learning tasks.\n\n(2) See item (2) in Section 1.", "claims_and_evidence": "During the development of the paper the authors demonstrate theoretically that aligning data and knowledge structures improves interpretability and performance. Furthermore, the authors present experiments on various datasets and using various classifiers for comparison. The experiments show promising results compared to traditional classifiers, suggesting that it may have potential applications in open-world learning.", "methods_and_evaluation_criteria": "In this paper, the methods and evaluation criteria are appropriate and well justified for the problem studied. The authors also make use of several datasets and a comparison with several classifiers to validate the effectiveness of the proposed method, however it would be beneficial to include, in addition to accuracy, an evaluation of sensitivity, precision and F1-score. Finally, it would be very interesting to discuss limitations and scenarios in which the method might not be as effective.", "theoretical_claims": "The paper contains several formulations and demonstrations, which are developed in detail in the annexes. Personally I have briefly reviewed the main theoretical claims and have not found any issues. Several demonstrations are extensive and demand a detailed review, which is time limiting.", "experimental_designs_or_analyses": "In relation to the evaluation of the robustness and validity of the experimental designs and analyses presented, the paper uses well known public datasets such as MNIST, APY, ImageNet1K, AWA1, AWA2, FLO and CUB. These datasets are suitable for the classification task that is studied, but it should be mentioned if any preprocessing was performed, as it may affect the results. Further, the paper compares HC-HFLM with several traditional and also deep neural network based classifiers, this provides a solid benchmark to evaluate the performance, however the variability of the results is not discussed. Additionally, the results show an improvement in generalization, however it would be useful to discuss how the method performs on noisy data to better evaluate the robustness of HC-HFLM.", "relation_to_broader_scientific_literature": "The paper is based on theories of cognitive science, especially focusing on the ambiguity of concepts and context dependence (Wittgenstein, 1953; Rosch, 1975). Within the context of ML, the authors expand the idea of using human cognition of concepts to improve performance, as explored in previous work (Cui and Liang, 2022) with their Fuzzy Learning Machine (FLM). In addition the paper introduces fuzzy similarity relations (FSR) to model class knowledge, based on the fuzzy set theory (Zadeh, 1965), this allows capturing the ambiguity of concepts. The proposed hierarchical alignment loss integrates this knowledge into the learning process. This extends previous work on hierarchical learning (Silla & Freitas, 2011; Wang et al., 2020). Finally the paper develops the FSR-based quotient space theory for modeling data and knowledge, this expands previous work on quotient spaces (Zhang & Zhang, 2014).\n\n\nReferences:\n\nWittgenstein, L. Philosophical Investigations. Blockwell Publishing, 1953.\n\nRosch, E. Cognitive representations of semantic categories. Journal of Experimental Psychology: General, 104(3): 192, 1975.\n\nCui, J. and Liang, J. Fuzzy learning machine. In Advances in Neural Information Processing Systems, pp. 3669336705, 2022.\n\nZadeh, L. A. Fuzzy sets. Information and Control, 8(3): 338–353, 1965.\n\nSilla, C. N. and Freitas, A. A. A survey of hierarchical classification across different application domains. Data Mining and Knowledge Discovery, 22:31–72, 2011.\n\nWang, Y., Hu, Q., Zhu, P., Li, L., Lu, B., Garibaldi, J. M., and Li, X. Deep fuzzy tree for large-scale hierarchical visual classification. IEEE Transactions on Fuzzy Systems, 28(7):1395–1406, 2020.\n\nZhang, L. and Zhang, B. Quotient Space Based Problem Solving: A Theoretical Foundation of Granular Computing. Tsinghua University Press, Beijing, China, 2014.", "essential_references_not_discussed": "The references are good but more recent work integrating structured knowledge bases into neural networks is not mentioned. For example, representation-enhanced neural knowledge integration (Liu et al., 2024) and logical neural networks for knowledge base completion with embeddings and rules (Sen et al., 2022). In addition, there is an interesting and recently published paper on hierarchical representation learning (Yu et al., 2024) which also aligns representations learned via embedding hierarchical tree structures of concepts, it would be good to include it. \n\n\nReferences:\n\nLiu, S., Cai, T., and Li, X. Representation-Enhanced Neural Knowledge Integration. arXiv preprint arXiv:2410.07454, 2024.\n\nP. Sen, B. W. Carvalho, I. Abdelaziz, P. Kapanipathi, S. Roukos, and A. Gray, “Logical Neural Networks for Knowledge Base Completion with Embeddings & Rules,” in Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, Abu Dhabi, United Arab Emirates, Dec. 2022, pp. 3863–3875.\n\nYu, J., Zhang, C., Hu, Z. and Ji, Y. Embedding Hierarchical Tree Structure of Concepts in Knowledge Graph Embedding. Electronics. 2024", "other_strengths_and_weaknesses": "Strengths:\n\n- The methodology presented by the authors has a considerable degree of originality as it uses fuzzy similarity along with human cognition for classification.\n\n- The paper contributes significantly to the use of fuzzy learning machine, quotient spaces and knowledge based approaches for better classification methods.\n\n- The paper addresses a problem present in classification methods, which is to assume that classes are well defined.\n\n- The authors present well-defined experiments on 6 datasets, showing an improvement over classical classification methods.\n\n- The presentation of the paper is well articulated and includes demonstrations that support its formulation.\n\n\nWeaknesses:\n\n- It would be useful to discuss how the method performs with noisy data and thus better evaluate its robustness to HC-HFLM.\n\n- It would be beneficial to include, in addition to accuracy, an evaluation of sensitivity, precision and F1-score. \n\n- It would be very interesting to discuss limitations and scenarios in which the method might not be as effective.\n\n- It would be useful to include details on training efficiency, computational requirements and an explicit comparison of execution time with other methods.", "other_comments_or_suggestions": "- In the introduction section, near line 051, “.. employs exemplar theory to capture the typicality effects (Smith et al., 1974; Rosch, 1975)”, you should quote (Medin & Schaffer, 1978) instead of (Rosch, 1975).\n\n- Figure 2 looks very small and misses the details, which makes it difficult to understand, it is recommended to present a larger image.\n\n- In section 5.2, near line 371, it says “WordNet (Miller, 1995) (CK1)”, but it should read “WordNet (Miller, 1995) (CK2)”. \n\n- In section 5.2, near line 376, it is recommended that you name the techniques instead of their abbreviations (KNN, DT, SVM and NB). They are described in the supplementary material but it would be good to write their full names when mentioning them for the first time.\n\n- In Table 3, the symbol “--” means that it takes more than 7 days for training? Also, what does N/A mean in this context?", "questions_for_authors": "- The proposed method is intended for classification, but could it be expanded for other machine learning tasks?\n\n- What are the limitations of the proposed method and under which scenarios is it not so effective?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "grBpKDNdTK", "reviewer": "Reviewer_qLb8", "summary": "The paper extracts the similarities between concepts from the human knowledge system and uses these similarities to guide the learning process. As a result, the similarities between concepts are integrated into the sample similarity, thereby improving model performance. Meanwhile, the paper guarantees the effectiveness of the proposed method through theoretical analysis.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "# 0. General Response\nWe sincerely appreciate the reviewer’s positive feedback and valuable comments. Below, we provide a point-by-point response to each comment.\n\n# 1. Response to “Weaknesses”\n(1) CLIP (Contrastive Language-Image Pre-training) is a self-supervised learning method that learns representations of images and texts by aligning them in a shared latent space. The proposed method differs from CLIP in several aspects.\n\n- **Learning Paradigm.** CLIP is a self-supervised learning method, whereas the proposed method follows a fully supervised learning paradigm.\n\n- **Interpretability.**  CLIP maps images and texts into a shared Euclidean space with specified dimensions and then aligns them. However, the individual dimension of this space lacks explicit semantic meaning, making the alignment process less interpretable. In contrast, the proposed method establishes alignment between the hierarchical structures derived by class knowledge and data in the quotient space. Each component in the quotient space carries a well-defined semantic meaning, leading to a  interpretable alignment process.\n\n- **Scaling Effect.** CLIP is trained on large-scale image-text pairs, enabling good performance across various downstream tasks. Similarly, the proposed method can be extended to large-scale datasets to learn more generalized representations, enhancing its ability to handle diverse downstream tasks effectively.\n\n- **Application Scenario.** The success of CLIP is largely attributed to the widespread availability of image-text pairs on the Internet. However, in many real-world applications, acquiring large-scale data along with corresponding textual descriptions is challenging. In such cases, leveraging class knowledge to enhance the model’s ability to understand concepts becomes crucial.\n\n(2) Formula (25) defines a class of functions, where any function satisfying the given conditions can serve as a coarsening function. This function eliminates subtle difference in the original class fuzzy similarity relation, preserving a robust and salient ranking of class similarities.\n\n# 2. Response to “Other Comments Or Suggestions”\nIn the final version, we will enlarge the fonts in Figures 1, 2 and Table 2 to enhance readability and provide greater convenience for the readers.\n\n# 3. Response to “Questions For Authors”\nSee item (1) in **Response to “Weaknesses”**.", "claims_and_evidence": "Yes. The claims made in the paper are supported by theoretical and experimental evidence.", "methods_and_evaluation_criteria": "Yes. The proposed method is effective. And the evaluation criteria can verify the advantages of the proposed method", "theoretical_claims": "Yes. I checked the main theoretical results of the paper, including Theorem 3.3, 4.11-4.13, and these conclusions are correct. Meanwhile, Section 5.1 provides a toy example which validates the conclusions in Theorem 4.11,4.12 and Corollary 4.13 step by step.", "experimental_designs_or_analyses": "Yes. I checked the experiments and corresponding analysis in the paper. Specifically, the experimental results in Section 5.1 are consistent with the conclusions in Corollary 4.13. Meanwhile, the experimental results in Section 5.2 indicate that by incorporating class related knowledge, the proposed method can improve the generalization performance of the model, and the higher the quality of the knowledge, the greater the performance improvement.", "relation_to_broader_scientific_literature": "The paper develops the fuzzy similarity relation-based quotient space theory and establishes the connection between fuzzy similarity relation-based quotient space theory and fuzzy equivalence relation-based quotient space theory through Theorem 4.5, which enriches the theory of fuzzy quotient space.\nAt the same time, the paper proposes a universal method for integrating human knowledge into the machine learning model, providing new and promising idea for developing human cognition-inspired machine learning methods.", "essential_references_not_discussed": "To the best of my knowledge, no important reference is missing.", "other_strengths_and_weaknesses": "The strengths of the paper are as follows.\n\n1.\tIn classification problems, each class corresponds to a concept. Unlike most existing classifiers, this paper views the classification problems as the concept cognition problems and seeks universal principles from human cognition to improve classification performance.\n\n2.\tThe paper develops the fuzzy similarity relation-based quotient space theory and analyses the working mechanism of the proposed method based on this theory. These theories guarantee the interpretability and effectiveness of the proposed method.\n\n3.\tThe experiments in the paper also verify the interpretability and effectiveness of the proposed method.\n\nThe weaknesses of the paper are as follows.\n\n1.\tThe paper does not discuss the difference between the proposed method and CLIP, which is the typical method to align image and text.\n\n2.\tThe paper does not provide a detailed explanation on how to coarsen the low-quality knowledge. Specifically, what is the definitions of f_{coa} in formula (25)?", "other_comments_or_suggestions": "1. The fonts in Figure 1, Table 1, and Figure 2 should be enlarged.", "questions_for_authors": "Currently, multi-modal pre-trained models, e.g., CLIP, have been proposed, which aims to align image and textual information. What is the difference between these methods and the proposed methods?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper received four effective reviews, and all of them are positive. Overall, the paper is of good quality and should be accepted.
Decision: Accept (poster)