Enhance Multi-View Classification Through Multi-Scale Alignment and Expanded Boundary
Authors: Yuena Lin
Yiyuan Wang
Gengyu Lyu
Yongjian Deng
Haichun Cai
Huibin Lin
Haobo Wang
Zhen Yang
Venue: ICLR 2025 Poster (2025)
Keywords: Multi-View Classification
Multi-Scale Alignment
Fuzzy Logic
Decision Boundary
Primary area: unsupervised, self-supervised, semi-supervised, and supervised representation learning

Abstract: Multi-view classification aims at unifying the data from multiple views to complementarily enhance the classification performance. Unfortunately, two major problems in multi-view data are damaging model performance. The first is feature heterogeneity, which makes it hard to fuse features from different views. Considering this, we introduce a multi-scale alignment module, including an instance-scale alignment module and a prototype-scale alignment module to mine the commonality from an inter-view perspective and an inter-class perspective respectively, jointly alleviating feature heterogeneity. The second is information redundancy which easily incurs ambiguous data to blur class boundaries and impair model generalization. Therefore, we propose a novel expanded boundary by extending the original class boundary with fuzzy set theory, which adaptively adjusts the boundary to fit ambiguous data. By integrating the expanded boundary into the prototype-scale alignment module, our model further tightens the produced representations and reduces boundary ambiguity. Additionally, compared with the original class boundary, the expanded boundary preserves more margins for classifying unseen data, which guarantees the model generalization. Extensive experiment results across various real-world datasets demonstrate the superiority of the proposed model against existing state-of-the-art methods.

Published conference paper at ICLR 2025
ENHANCE MULTI-VIEW CLASSIFICATION THROUGH MULTI-SCALE ALIGNMENT AND EXPANDED BOUNDARY
Yuena Lin 1t, Yiyuan Wang 122, Gengyu Lyu Yongjian Deng'
Haichun Cai Huibin Lin Haobo Wang & Zhen Yang 1,+
College of Computer Science, Beijing University of Technoloog,, Beijing
2 Idealism Beijing Technology Co., Ltd., Beijing
College Computer and Data Science, Fuzhou University, Fuzhou
School of Software Technology, Zhejiang University, Hangzhou
yuena in@ 126 26. com, yJyuanwangeemaile Lyugengyuegmail com, f fjsmchce163. com, 221010005 edu en wanghaoboezju. edu. cn, (yjdeng, yangzhen
bjut. edu. cn
@annghenn@bjuu.... @bjuut edu.cn,
ABSTRACT
Multi-view classification aims at unifying the data from multiple views to com- plementarily enhance the classification performance. Unfortunately, two ma- jor problems in multi-view data are damaging model performance. The first is feature heterogeneity, which makes it hard to fuse features from different views. Considering this, we introduce multi-scale alignment module, includ- ing an instance-scale alignment module and prototype-scale alignment mod- ule to mine the commonality from inter-view perspective and inter-class perspective respectively, jointly alleviating feature heterogeneity. The second is information redundancy which easily incurs ambiguous data to blur class boundaries and impair model generalization. Therefore, we propose a novel ex- panded boundary by extending the original class boundary with fuzzy set the- ory, which adaptively adjusts the boundary to fit ambiguous data. By integrating the expanded boundary into the prototype-scale alignment module, further tightens the produced representations and reduces boundary ambiguity. Additionally, compared with the original class boundary, the expanded boundary preserves more margins for classifying unseen data, which guarantees the model generalization. Extensive experiment results across various datasets demonstrate the superiority of the proposed model against existing oate-oofthheart methods.
model
INTRODUCTION
Multi-view data are widely applied in real-world applications by combining diverse heterogeneous features to describe the same object (Wang et al. (2023a) Zhang et al. (0024b). Unluckily, feature heterogeneity and information redundancy concealed the mulli-view data have become the obstacles in multi-view learning (MVL) (Tan et al. (2024)) The feature heterogeneity means that the multi-view data are collected from different data distributions without unified data format, causing fusion difficulty among the view features (Wang et al. (2024a)). The information redun- dancy signifies the latency of unnecessary features or ambiguous data, which hinders the ability to delimit the decision boundaries appropriately and naturally engenders classification ambiguity.
Existing MVL models mainly delve into alleviating the heterogeneity by finding informative com- mon subspaces (Zhang et al. (2023a)) or ideal view weights (Hu et al. (2022)). The former explores the feature consistency and complementarity among the multi-view data (Lyu et al. (2024b); Wang et al. (2024c [Liag al. (2024)). which neglects significant class differences for separating reliable decision boundaries. The latter presumes that the qualities or importances of views are stable for samples (Kumar Maji (2023)), and aims at allocating appropriate weights to different views (Ho- ufar et al. (2023); Liu et al (2023). However, the varied view qualities fail to match diverse samples practically (Han et al. (2022b)), which also leads unreliable decision boundaries. For instance,
"Corresponding author, + equal contributions.

---

Published as conference paper at ICLR 2025
(a) Raw data
(b) ETMC
(c) PDMF
(d) RCML
(e) Ours
Figure t-SNE on Scene15 dataset. The boundaries of the green class and the brown class are delin-
eated. Compared with existing advanced models, our model shows much more distinet boundaries.
tigers and zebras have similar stripes but different colors, which makes the model give the color view a higher weight. Nevertheless, zebras and pandas have the same colors but disparate stripes, which induces the model to weigh more for the stripe view. Such scenarios easily result in unstable qualities or significances of different views to construct unreliable decision boundaries eventually.
Recent MVL works are dedicated to realizing trusted decisions by estimating the inherent un- certainties among multi-view data (Xu et al. (002b)) 2024)), and integrating multiple views at an evidence level. Actually, though they modify the logits to realize better classification decisions (Sensoy et al.] (2018), they still fail to deal with the aforementioned problems. To illustrate the issue and our motivation intuitively, we provide the t-SNE in Figure [ which includes our model MAMC (enhance multi-view classification through multi-scale alignment and expanded bound- ary) and three advanced MVL models ETMC (Han et all (2022b)) PDMF (Xu et al. (2023b)), and RCML (uu et al. (2024) Scene15 dataset. The t-SNE reflects two issues of current MVL mod- els: 1) They cannot well mine the commonality to tighten the representations; 2) They ignore to exploit the class difference for clearing the ambiguous decision boundaries. These observations imply that they are unable to reduce the feature heterogeneity and the information redun- dancy satisfactorily. Motivated by these observations, our model focuses on mining common- alities among view representations and learning clear decision boundaries in the representation space simultaneously to tackle the feature heterogeneity and the information redundancy. On the one hand, MAMC introduces a multi-scale alignment module to learn expressive view representa- tions by mining the commonalities among views and class differences among prototypes; on the other hand, it expands the original class boundaries to the wider decision boundaries in the rep- resentation space, fitting ambiguous samples to clear the blurry areas between different classes adaptively. Specifically, the multi-scale alignment module includes two complementary contents: 1) An instance-scan scale alignment module mines the inter-view commonality by aligning the view representations of each instance. 2) A prototype-scale alignment module exploits the inter-class difference from the instances with the same labels to find appropriate decision boundaries jointly.
The design of expanded boundaries is based on fuzzy set theory (Zhang et al. (023bb) ). and its expansion process involves fuzzy representations and crisp represenlations, where the fuzzy rep- resentations are defined in the fuzzy representation space while the crisp representations are in the normal representation space. The core of boundary expansion is based on the assumption that crisp representations are a special type of fuzzy representations. positives of the crisp representations in the fuzzy representation space, which are then employed to construct elements of the expanded boundary. Finally, the expanded decision boundary inte- grated into the prototype-scale alignment process. By narrowing the expanded boundary and its corresponding class protolype while distancing other class prototypes, the inter-class difference is sufficiently exploited. Additionally, the expanded boundary reserves extra space for instances that belong to the class latently, which is beneficial for improving the model generalization. Our contributions are summarized as follows:
enables us to find learnable
We propose MAMC to address the problems caused by the feature heterogeneity and the information redundancy. Through introducing the multi-scale alignment module, MAMC sufficiently mines the inter-view commonality to tighten instances and inter-class differ- ence to delimit the decision boundaries.
We propose a novel adap-ive expanded boundary to tackle the ambiguous decision boundaries. By integrating the expanded boundary into the prototype-scal alignment module, the inter-class difference is reinforced to clear the ambiguous decision bound- aries, while the representations inside the boundary are tightened.
We explain the rationality of the model design theoretically, and extensive experimen- tal results across diverse public datasets and comprehensive experimental analysis have
2

---

Published as conference paper at ICLR 2025
verified that the proposed model shows significant superiority against the existing state- of of-the-art models.
PRELIMINARIES
2.1 FUZZY SET THEORY
Fuzzy set theory builds on classical (crisp) set theory by introducing the concept of membership degrees, which allows for fine fine-grained definitions of whether an element belongs to a set (Zadeh (1955). A fuzzy set A in the universe U is characterized by a membership function fa (x) which allocates each element in U with real number in [0, 11, where U is defined as a set that contains all the elements under consideration for a particular discussion or problem, and fA (Ir) represents the membership degree of x in A (Zhelezniak et al. (2019). Fuzzy set theory provides pow- erful framework for reasoning about sets with uncertainty, but the specification of membership functions depends greatly on domain knowledge. Specifically, we have the following definitions:
Definition: A function fa: U + L C R is called a membership function.
Definition: A tuple A (U, (a) including a universe U and membership function LA is called fuzzy set.
Following the definitions, when L = [0, 1]. is a fuzzy set. When the membership function takes on only two values 0 and I, namely L {0, 1} according to whether æ belongs to A. In this case, A reduces to a classical set.
2.2 MULTI-VIEW LEARNING
Joint learning on multi-view data has been proven effective on various tasks, such as clustering (Zhuge et al. 20000); Shen al. (2024a)), and classification (fiang et al (2024): Lyu et al (2024a)) Current works are roughly divided into the subspace-base (Shen et al. 2024b) and view-weighted
approaches (Zhang al. (2024a)) The first category aims finding common subspace where the consensus representations are generated for all the views Wang et al. (202b)) Canonical correla- tion analysis (CCA) based approaches are one type of representative methods that project different views into a subspace by maximally correlating them. (Kumar & Maji (2023) ) proposed D2CCA that integrates the CCA theory with the objective function to find the joint representations. (Yuan) projected into a certain space by an arbitrary nonlinear mapping. Meanwhile, many works are dedicated to searching optimal weights for different views. (Xu et al. (2020)) proposed to make deep interac- tive information using view-specific information in an adaptive weighted manner, and seamlessly integrated the view-specific information with multi-view loss fusion strategy to achieve joint decisions. (Houfar et al. (2023) ) attended the distinctions between different views because of the sample importances, and proposed a dynamic learning strategy for automatically weighting views and samples. However, the subspace-based approaches mainly focus on mining feature commonal- ity to handle feature heterogeneity but ignore the inter-class difference for delimiting the decision boundaries. And the view-weighted approaches have trouble producing generalized weights for views, which also makes it hard to find an effective boundary. In summary, these approaches fail to tackle the problem caused by information redundancy, especially when ambiguous samples distributed around the decision boundaries. Motivated by this, we focus on mining commonalities among view representations and learning clear decision boundaries in the representation space simultaneously to alleviate the feature heterogeneity and the information redundancy.
et al. (2022) proposed canonical F-correlation framework where each feature
2.3 MULTI-VIEW CLASSIFICATION
Existing multi-view classification methods can generally be divided into two categories: feature dusion-based and decision- methods. Feature fusion-based methods focus on effectively com- bining features from all views (Meng et al. 2. MV-HFMD (Black & Souvenir (2024) explores feature fusion by introducing a novel fusion scheme and mutual distillation to adapt neural net- works for multi-view classification. Mmdynamics (Han et al. (2022a)) dynamically assesses the informativeness of both feature-level and view-level data across different samples, ensuring trust- worthy integration of multiple views. However, these methods often neglect feature heterogene-
3

---

Published conference paper at ICLR 2025
ity, which undermines the effectiveness of feature fusion. For the decision-based methods, they are dedicated to making more reasonable decisions. ETMC (Han et al. (022b)) introduces a new paradigm for multi-view learning by dynamically integrating difterent views at the evidence level, thereby enhancing classification reliability. RCML (Xu et al. (2024) attends to that conflictive in- formation in different views, and proposes conflictive opinion aggregation strategy that exactly models the relation of multi-view common and view-specific reliabilities. MVAKKN (Fan et al. (2023) utilizes the Dempster-Shafee -Saffer theory to integrate information from each view, successtully extending adaptive KNN to multi-view setting. IPMVSC (Hu et al. (203)) constructs multi-view fuzzy classification model inherited from the natural interpretability of fuzzy rule-based systems to realize interpretable classification. Many methods of this kind interact with the multiple views only in the decision phase, which cannot well integrate the information from all the views. con- trast, our model aligns view features during the representation learning phase and combines them in the decision phase, allowing for more effective and interactive information integration across all views.
3 PROPOSED METHOD
The architecture of our model is shown in Figure including view-specific representation ex- tractors, multi-scale alignment module, and a joint classifier. To detail the model architecture conveniently, we first provide some notations.
Notations. Formally, multi-view dataset D [({= (v) J S i S N} is defined as N i.i.d. instances with V views and corresponding labels. We use a real-value vector xz"") e PR (1 < e < V, == 1, .N) to denote the feature vector i-th instance for the v-th view, where D, is the feature dimension of the v-th view and K is the number of classes. For the labels, natural number 1/1 E N to represent the ground truth of i-th instance and E {0, 1]A to represent the corresponding one-hot vector. To integrate information from the multiple views, model aims at learning desired mapping f: X y from the multi-view dataset D for better predicting the correct label of unseen instances.
3.1 VIEW-SPECIFIC REPRESENTATION LEARNING
Heterogeneous multi-view data conceal many inherent issues, such as inconsistent dimensions of view features. To address the issues, we employ multiple view-specific exclusive view representations in the same dimension (Zhong al. (2024); Jiang al. (2024)) For the v-th view data X we apply a view-specific encoder E(d) to produce the view representation matrix (v) and decoder D(e) to restore the raw view feature, where the encoder and the decoder are constructed by multi-layer perceptions (MLPs). The reconstruction process given by
encoders to generate
Lrec Σ
x - -D()) (E(*) ( (x()))
3.2 CETTACE--SCALA ALIGNMENT
The key to reducing the feature heterogeneity of multi-view data is to mine the inter-view com- monality. Contrastive learning exploits commonality among data by pulling together the positives while whsiiig away the negatives [Wang et al. (2024b)). The instance-scale alignment module com- bines contrastive learning and regards all the view representations of each instance positives
while others as negatives. Concretely, the view representations {h(") J abstracted from the view-specific encoders recognized positives and others negatives. Hence, the instance- scale alignment loss is
N
Dis Diss(tf (1(") (((") Tene
Lins NV
log
Dis (1("),,("")) (1()) Tine + Ermay Ejjr Dis R.
ugfu 1=1
Tine
where Tims is temperature coefficient, and Dis (., is distance function. By aligning the positive view representations, the critical common semantics to recognize similar samples exploited,

---

Published conference paper at ICLR 2025
Training Instances
View Representations
Instance-Stae Aligumant
View
XP
View
View
Toyather Plsh Avoyy
DCD
wojueedeg flrpprrg
8
Boundary
Expiraled Boundary
D
Representations
Proestypas
Undeon Samples
Cross Entropy Eate
8
Joint classiieer
g00
Mean Pooling
1
Prediction
Ground Truch
Figure 2: Three main components of the proposed model: (1) View-specific representation tractors. Multiple view-specific auto-encoders abstract critical view representations (H())) J== from multi-view data (X (X()) 1e=1- (2) Multi-scale alignment module. The instance-scal. alignment module aligns view representations {h J of each instance I;, where the icons with various shapes correspond to different instances and those with different colors correspond to representa- tions from different views. The prototype-scale alignment module integrates the adaptive expanded boundaries, and then aligns the view representations with the same labels. (3) Joint classifier. All the view representations are sent into the joint classifier to obtain fused predictions.
while the feature heterogeneity is mitigated. By differing the negatives, the discrepancies between different samples gradually emerge, thereby making the learned representations more suitable delimiting the decision boundaries.
3.3 PROTOTYPE-SCATE ALIGNMENT
In contrast to the instance-scale alignment module, the goal of the prototype-scale alignment mod- ule to exploit the inter-class difference for delimiting clear decision boundaries. The core of achieving this goal is to construct the expanded boundary.
3.3.1 EXPANDED BOUNDARY
Since the information redundancy in multi-view data easily blurs the decision boundaries, general prototype learning ignoring the inter-class difference has limited ability work effectively. In- stead, propose adaptive expanded boundary to assist the prototype-scale alignment module, expecting to clear the ambiguous boundaries.
To sufficiently gather the class information for exploiting the inter-class difference, we collect all the view representations with the same labels to construct the class prototypes and use them to centralize the view representations, For the training data of the k-th class, the centralizing process defined as:
C₁ V V
ΣΣ
h
u=l
y
(v)
h (v)
2
Ck, 1,...

---

Published as conference paper at ICLR 2025
where = is the centralized representation vector of the i-th instance in the v-th view, Ys is the set including the training data whose labels are ki and |-] is used for counting the cardinality of a set. C₁ the prototype vector of the k-th class, which filters the figh-freeuency noise by averaging.
A key step of boundary expansion is to construct the fuzzy representation space with membership functions, and the Gaussian membership function is the most universal one (Li et (2023)). The appropriate number of membership functions imparts the expanded boundary with more adapt- ability. To this end, we adopt Gaussian membership functions. For the I-th membership function, the corresponding membership degree is defined by
(0) [1L
-mj)
(0) C. = exp
- 1,2,2 L, j= 1,,,, ,d,
(5)
26,,
where is the dimension of the view representation, (1 the j-th element of i-th instance in the v-th centralized view representation vector, m, and رJ.1 are the trainable mean and standard deviation of the I-th Gaussian membership function respectively. These membership degrees are fused to obtain the final one for producing fuzzy representations, and the process is defined as
2
(())
Comb r (v) (.5.
112,,,2 L,
(6)
2 R I(")
(7)
where + is the element of the membership degree matrix R and @ is Hadamard product to transform the crisp view representation 2(v) into the fuzzy representation Z(w), Comb is the combination operator to fuse membership degrees and how to determine will be explained later.
According to properties of mem- bership functions, each value of R(e) limited in [0, 1], which implies that the crisp representa- tions can be viewed as one spe- cial of fuzzy ones. Based on this, we regard crisp represen- tations and fuzzy representations as positives. Besides, we assume that their discrepancies reflected by the coordinate differences are tolerable since the discrepancies between positives do not signifi- cantly violate the commonality se- mantics in general. Inspired by this, we implement the tolerable discrepancy to expand the origi- nal class boundary, and the bound- ary expansion process is depicted in Figure Mathematically. the el- ements of the expanded boundary are transformed by the coordinates of the fuzzy representations, which are formalized as
Z2 Centralized Representation Space
((2
Prototype
View Representaion
(1 - 11)zz
(1 7 22)
Fuzzy Representaion
Boundary Element
(7,,72)
Mapping to Fuzzy Representation
①
Mapping to Boundary Element
(f,21.P222)
Z
Figure 3: The boundary expansion process 2-dimension space. One centralized view representation is firstly utilized calculate the fuzzy representation and then mapped as the boundary element
Z(1)_
())) + 2 2 (1-1100 x)))) c Z = 12-Ri (2- 2 Z(0),
(8)
where (()) composed of boundary elements, and are two real value matrices whose each el- ement is 1 and 2 respectively. Though simple, the expanded boundary inherits two desired proper- ties from the fuzzy representations: 1) Semantics semantics-positive. The boundary elements are supposed to be the positives of crisp representations, which benefit recognizing the ambiguous samples far away from the class centers. 2) adaative The expanded boundaries are based on the learn- able membership functions and could adjust adaptively to involve the ambiguous samples in the training process. These two properties aim to capture nearly all the samples inside each boundary for fully exploiting the inter-class difference.

---

Published conference paper at ICLR 2025
3.3.2 Loss FUNCTION
After expanding the original boundaries, we restore the coordinates of the centralized fuzzy repre- sentation matrix Z by adding their corresponding class prototype C' the restored fuzzy repre- sentation matrix H, since it is tough to separate the classes in centralized representation space whose class centers are all origin points. The expansion of the original class boundaries leads to the new-born prototype of boundary elements C, which causes a trade-off between C' and C. We mix up both to harvest the fused prototype C by linear combination as
C C
11) C,
where each prototype vector of C' is calculated by Eq. 13 Ö is calculated by averaging the bound- ary elements of each class, and is a balance coefficient to reconcile C and Ć.
To generalize for unseen data as much as possible, we expect maximize the spectrum of each class, which is equal to maximizing the variance within each class. Meanwhile, we expect to achieve better separateness among different classes and compactness within cach class, which intends to maximize the distances between different classes while minimizing the variance within each class. Considering these factors, the loss function is formalized as a minimax game by
Lyro min max (P V),
(10)
P+V V
where P is defined to distance different classes and to tighten samples within each class while generalizing for unseen data. They could be defined
p = NKV E
1=
K
n=1
.
hn
T2
(11)
V= NV Σ
(0)
2
==
=k
Ck
For convenience, we let Q == P + V, and propose an upper bound of Q in our model, expecting to further optimize Q by optimizing its upper bound. We define the upper bound Opro as
V
K
Tprn
Spro = NV
ΣΣΣ log
(12)
t= k= gazk
E,
Taro
2
where Cn is the fused prototype vector of the k-th class, h the restored boundary elements of the i-th instance the w-th view, and Tpro is the temperature coefficient to control the compactness of classes. Generally, higher Tpro makes the classes more compact. We provide the theoretical proof that Spro is an upper bound of in the Appendix.
In the prototype-scale alignment loss, we integrate all the view representations to serve as the class samples since partial view information easily releases misleading information or loses crit- ical information, which may lead to ambiguous decision boundaries. In this way, all the view information is gathered to collaboratively form the inter-class difference for delimiting the deci- sion boundaries. Additionally, according to (Oord et al. (2018)), contrastive learning evaluates the mutual information between the raw view data and the restored boundary elements, namely
Iz.
Z log (N) Qpros
(13)
1=1
where / (+) used calculate the mutual information, N is the sample amount participated in the loss function. This inequation reveals that more negatives raise the lower bound and preserve more complete information from the raw data for better alignment performance. Typically, integrating all the view representations is flexible way to multiply the amount of negatives.
3.3.3 COMBINATION OPERATOR DETERMINATION
The combination operator Comb in Eq. (6) is crucial to the expanded boundaries. On the one hand, the Comb operator cannot damage the self-adaptive ability of the expanded boundaries.
7

---

Published as conference paper at ICLR 2025
On the other hand, the Comb operator controls the class variances to generalize for unseen data. The former is achieved by the learnable membership functions, and the latter affected by the determination of the Comb.
Theorem When employing Min Pooling as the Comb operator, the overall variance of each class is maximized. The proof is provided in the Appendix.
Therefore, when the ideal Comb operator is defined as the minimal of the membership functions, the spectrum of each class is maximized. It means that the expanded boundary could general- ize more latent (unseen) samples belonging to corresponding classes. Finally, the prolotype-scale alignment loss Lpro in Eq. (10 is reduced as
N
Tprn
0)
E
Ppo min
log
(14)
NV
Tyro
h
3.4 CLASSIFICATION PREDICTION
During the classification process, we employ a joint classifier that adopts the parameter sharing strategy, and it has several merits: 1) Shared parameters receive information from all the views simultancously, which is conducive to constructing a classifier specific for the multi-view data. 2) The multi-scale alignment module has mined the inter-view commonality and the inter-class difference to construct clear decision boundaries, reducing the demand on the complex classifier with a large number of parameters. The model prediction produced by averaging the outputs of the joint classifier by
V Σ
V
D.
P
Σ V
1
(N!
(15)
where (-) is logistic classifier, p. is the output of g (-) in the e-th view, and p. is the final prediction. During training, we use the cross cross-entropy to calculate the classification loss as
N
K
Vi log Pij.
(16)
N
where p; is the final prediction from Eq. 15 and p.j is its j-th element. ÿ; refers to the one-hot vector label of the i-th instance, and U. is the corresponding element this vector. The overall loss Cau comprises four parts, namely reconstruction loss, classification loss, instance-seale alignment loss, and prototype-se scalle alignment loss, which given by
Call Crect + aLins + 3C
(17)
where a and B3 are penalty coefficients. Call focuses on mining the commonality among different views and clearing ambiguity areas around the decision boundaries, which successfully alleviates the damage of the feature heterogeneity and the information redundancy.
EXPERIMENTS
In our experiment, we evaluate model on eight public multi-view datasets including HandWrit- ten, Scence15, PIE, CCV, Animal, 100Leaves, Hdigit, and YoutubeFace. To show the effectiveness of the proposed model, seven Sate-of-theeart methods are adopted to compare with, including mm- dynamics (Han et al. (2022a)), ETMC (Han et all (202bb)). UMDL (Xu et al. (2023a)). PDMF (x al. (2023b)), IPMVSC (Hu al. (2023)), MV-HFMD (Black & Souvenir (2024)). and RCML Xu el al. (2024). For each dataset, we split 80% instances for training and the remainder for testing. To obtain reliable results, implement the same dataset split ten times for all the models. More
details of the datasets are provided in the Appendix.
4.1 EXPERIMENTAL RESULTS
Table presents the classification results between our model and other baseline models, and adopt Accuracy (Acc), Purity, Recall, and Macro-F1 as the metrics. From the statistical compar- isons, some important observations are revealed:

---

Published as conference paper at ICLR 2025
Table Classification results (%) with baseline models. The best results are highlighted in bold.
"OOM" indicates that the models raise the out-of-memory failure.
Datasets
Metrics
mmdynamics
ETMC
UMDL
PDMF
IPMVSC
MV-HFMD
RCML
Ours
Acc
98.5=0.5
96.8+0.7
97.7=0.5 98.2=0.7 98.7=0.4
98.2+0.6
97.1=0.9 98. 9:0.4
Precision
98.510.5
96.310.7
97.810.5
98.310.7 98.710.4
98.210.5
97.110.9 98.9=0.4
HandWritten
Recall
98,5=0.5
96.9+0.7
97.7+0.5
98.3=0.7 98.7+0.4
98.2+0.6
97.2=0,9
98.9:0.4
Macro F1
98.5=0.5
%6.11.77
97.7+0.5
98.2+0.7
98.740.4
98.110.5
97.110.9
98.9=0.4
Acc
62.0=2.2
66.51188
63.3104
67.811.1
71.81144
80.711.2
70.011.0
81.5+0.7
Precision
57.3+2.8
66.6+2.5
61.2+0.7
65.8+2.8
73.8+0.7
80.6+1.6
70.6+1.3
81.0+0.6
1cene15
Recall
60.1=2.2
65.311.6
62.7+0.5
65.9#1.1
70.5#0.8
80.3+1.3
68. 1..9 80.8=0.7
Macro
57.6=2.5
62.8+1.8
61.1+0.7
63.0=1.4
69.1+0.9
80.0#1.4
67.1+1.1
80.5+0.7
Acc 72.5=3.7
90.5+2.3
71,1=3.3
88.7=3.0
91.9#2.0
86.8+2.4
91.822,9 93.1
Precision
70.6:4.0
88,013,8
74.613.9
87.613.8 81.742.1
87.712.7
91.143.4
92.8+2.1
PIE
Recall
70.6=3.7
89.6+3.6
71.1=33
87.3+4.9
82.1+188
88,9+2,4
91.0=3,5
93.6+1.8
Macro
67.413.8
87,413.8
69.313.5
85.714.5
80.5+2.5
83.816.1
89.813.9
92.1+2.0
29.6#1.1
42.611.4
36.211.5
50.5+144
42.611.8
53.410.9
42.4+1.7
54.0:11
Precision
22.8+2.8
41.4+2.6
35.5+1.3
48.5+1.3
48.3+2.0
51.1+111
42,3+2,2 53.0=1.
CCV
Recall
23.2+1.0
37.1+1.4
33.7+14
45.4+13 35.5+1.3
50.5+0.8
37.0+14
50.6#10
Macro F1
20.1=1.1
36.021.6
33.1=1.4
45.1#1.5
34.3#1.5
50.420.9
35.9+1.5
51.020.9
Acc
56.7=1.4
56.620.8
34.0=1.1
57.0=0.7 39.5#1.9
59.620.7
56.8+1.2
60.2+1.1
Precision
52.4+2.2
53.313.2
33.740,9
49.510.6 37.313.9
56.510.6
54.5+2.5
56.7+1.
Animal
Recall
49.9=1.3
49.7+1.0
30.4=0.8
47.2+0.7 33.3+2.5
53.7#0.4
1.. 53.8+0.8
Macro
50.4:1.4
49.711.1
30.410.7
46.710.6 33.312.7
54.010.5
49.611.3
54.1+0.8
Acc
93.5
90.842.1
98.410.8
97.710.7 66.114.1
98.310.2
88.611.5
98.5+1.1
Precision
93.6=1.2
90.5+2.5
98.7+0.8
97.7+0.9 70.2+3.4
98.4+0.0
88.7+2.0
98.5+1.0
100Leaves
Recall
93.9=1.1
91.3+2.4
98.4+0.9
98,0+1.0 69.0+2.0
98.9+0.2
89.4+1.4
99.0=0.7
Macro F1
92.7=1.3
89.3+2.7
98.3=0.9
97.5=1.0 64.0+2.7
98.420.1
87.0=1.7
98.5+1.0
Acc
99.620.1
90.8+2.1
98.0=0.2
99.420.2 97.8+0.3
84.1+2.0
98.320.3
9
Precision
99.6#0.1
98.4+0.2
98.0+0.2
99.4+0.2 97.8+0.3
88.9+1.7
98.3+0.3
99.8=0.1
Hdigit
Recall
99.6=0.1
98.4+0.2
98.0=0.2
99.4=0.2 97.9+0.3
87.5+2.0
98.3+0.3
99.8=0.1
Macro F1
99.610.1
98.410.2
98.010.2
99.410.2 97.810.3
86.5+2.3
98.310.3
99.8+0.1
Acc 56.310.3
71.942.1
OOM
85.620.3 28.210.3
83,410.8
52.611.0
87.1+0.2
YoutubeFace
Precision
74.8=0.6
83.2+0.8
OOM
89.2=0.4 29.9+4.0
85.6+1.0
83.1+1.0
89.5+0.4
Recall 46.0=0.3
67.7+3.1
OOM
84.3+0.4 5.4+0.3
82.8+0.8
38.7+1.3
86.2:0.3
Macro 53.5=0.5
73.042.8
OOM
86.620.3 5.220.4
83.920.8
46.621.4 87.7.0.1
1) The proposed model shows extraordinary superiority among the comparing methods across all the datasets, especially the Scene15 dataset. Compared with existing methods, our model is ideal to align all the view information while learning satisfactory decision boundaries, which are critical for the classification task.
2) For datasets that all the models receive wonderful results such as HandWritten, 100Leaves, and Hdigit datasets, the satisfactory decision boundaries are easily delimited. However, the proposed model could further improve the performance, which implies the rooted ambiguity near the deci- sion boundaries is further alleviated.
3) MV-HFMD shows very outstanding performance, which implies the significance of effective feature fusion multi-view learning. However, it may neglect to address the feature heterogeneity that plays an important role in facilitating the feature fusion, the model performance is some- what limited when compared with our model.
4) Compared with fuzzy multi-view learning model IPMVSC that focuses on using fuzzy rules to realize an interpretable classifier, our model attends to the feature-level issues and applies the fuzzy set theory in the representation space. Besides, from the results, find that IPMVSC does not work well on large datasets, but our model works well on both small and large datasets.
In summary, the proposed model successfully mines the commonality to tackle the feature het- erogeneity, learns the adaptive boundaries to mitigate the harm from the information redundancy, and preserves margins for unseen data to enhance the model generalization.
4.2
ABLATION STUDY
To evaluate the effectiveness of the multi-scale alignment module and the rationality of the Comb operator determination, conduct the ablation study whose results are provided in Table and

---

Published conference paper at ICLR 2025
Table Results of ablation study on classification (%), which are concerned with the functions of the multi-scale alignment module and the selection of Comb operator.
Lile
Lree
Lina
Lpro
Comb
HandWritten
Animal
PIE
CCV
96.08=0.47
55.7820.04
84.9320.36
42.48=0,58
97.44=0.54
56.95=0.42
89.71=1.20
49.34=3.33
96.7610.68
56.9510.65
90.8110.82
51.0142.39
9..4
57.8010.115
91.5411.82
51.6942.18
Min
98.1510.41
59.20:10.75
92.40=0.92
52.3410.34
Min
98.85+0.42
60.16+1.03
93.11+1.72
53.97+1.14
Max
97.67+0.42
58.26+0.31
91.92=0.60
52.18+2.68
Mean
98.69+0.30
59.74+0.81
92.46+1.09
53.11+0.40
57
Sp
(a) HandWritter
(b) Hdigit
(c) HandWritten
(d) Hdigit
Figure 4: Hyperparameter analysis on the HandWritten and Hdigit datasets. (a) and (b) delve into the functions of penalty coefficients a and B, (c) and (d) delve into the functions of temperature coefficients Tims and prro
have the following observations:
1) According the results, the instance- scale alignment and the prototype-scale alignment work effectively to improve the inter-view commonalities and inter-class differences, which are essential for addressing the problems caused by feature heterogeneity and information redundancy.
2) In contrast to the instance-scale alignment, the prototype-scale alignment shows greater im- provements, which attributed to exploiting the helpful inter-class differences for delimiting the clear boundaries.
3) Compared with different Comb operators, the Min Pooling exhibits the best performance, which demonstrates the superiority of preserving more margins for classifying unseen data.
4.3 HYPERPARAMETER ANALYSIS
We conduct the hyperparameter analysis on two penalty coefficients a and 8, and two tempera-
ture coefficients Tins and Tpro. For and 88, vary their values from {0.1, 0.25, 0.5, 0.75, 1). and for Tins and Tpro, we vary their values from (0.5, 0.75, 1.0, 1.5, 2.0) and {0.5, 0.75, 1.0, 1.2, 1.5) respectively. The results HandWritten dataset and Hdigit dataset are presented in Figure 4 Compared with the temperature coefficients, the penalty coefficients affect the results more signif- icantly, which illustrates the effectiveness of the multi-scale alignment module. Besides, the best results are prone to larger temperature coefficients that weigh more on similar samples.
CONCLUSION
Feature heterogeneity and information redundancy among the multi-view data are likely to cause the difficulty of feature fusion and the ambiguity around the decision boundaries, which moti- vates us to propose novel MVL model MAMC for addressing the problems. MAMC employs the view-specific auto-encoders to abstract exclusive view representations and introduces multi- scale alignment module to reduce the feature heterogeneity by mining the feature commonality. Besides, we propose novel expanded boundary exploit the class difference, which benefits to clear the ambiguity from the information redundancy. Extensive experiments demonstrate the superior performance and rational design of the proposed model.
10

---

Published as conference paper at ICLR 2025
ACKNOWLEDGMENTS
This work was supported by the National Natural Science Foundation of China (No. 62306020), the Young Elite Scientist Sponsorship Program by BAST (BYESS2024199), the National Key Re- search and Development Program of China (No. 2023YFB3107100), and the Beijing Natural Science Foundation (L244009). The authors gratefully acknowledge the support of K.C.WONG Education Foundation.
REFERENCES
Samuel Black and Richard Souvenir. Multi-view classification using hybrid fusion and mutual distillation. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 270-280, 2024.
Zizhu Fan, Yijing Huang, Chao Xi, and Qiang Liu. Multi-view adaptive k-nearest neighbor classi- fication. IEEE Transactions on Artificial Intelligence, 5(3):1221-1234, 2023.
Zongbo Han, Fan Yang, Junzhou Huang, Changqing Zhang, and Jianhua Yao. Multimodal dynam- ics: Dynamical fusion for trustworthy multimodal classification. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 20707-20717, 2022a.
Zongbo Han, Changqing Zhang. Huazhu Fu, and Joey Tianyi Zhou. Trusted multi-view clas- sification with dynamic evidential fusion. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(2):2551-2566 2022b.
Khamis Houfar, Djamel Samai, Fadi Dornaika, Azeddine Benlamoudi, Khaled Bensid, and Abdel- malik Taleb-Ahmed, Automatically weighted binary multi-view clustering via deep initialization (aw-bmve). Pattern Recognition, 137:109281, 2023.
Shizhe Hu, Ruilin Geng, Zhaoxu Cheng, Chaoyang Zhang, Guoliang Zou, Zhengzheng Lou, and Yangdong Ye. parameter-free multi-view information bottleneck clustering method by cross- view weighting. In Proceedings of the 30th ACM International Conference on Multimedia, pp-
3792-3800, 2022.
Xingchen Hu, Xinwang Liu, Witold Pedrycz, Qing Liao, Yinghua Shen, Yan Li, and Siwei Wang. Multi-view fuzzy classification with subspace clustering and information granules. IEEE Trans- actions on Knowledge and Data Engineering, 35(11):11642111 11655, 2023.
Zhangqi Jiang, Tingjin Luo, and Xinyan Liang, Deep incomplete multi-view learning network with insufficient label information. In Proceedings of the AAAI Conference on Artificial Intelligence, pp- 12919-12927, 2024.
Debamita Kumar and Pradipta Maji. Discriminative deep canonical correlation analysis for multi- view data. IEFE Transactions Neural Networks and Learning Systems, pp. 1-13, 2023.
Andong Li, Zhaohong Deng, Wei Zhang, Zhiyong Xiao, Kup-Sze Choi, Yueying Liu, Shudong Hu, and Shitong Wang. Multiview transfer representation learning with tsk fuzzy system for eeg epilepsy detection. IEEE Transactions on Fuzzy Systems, 32(1):38-52 52, 2023.
Xinyan Liang. Pinhan Fu, Qian Guo, Keyin Zheng, and Yuhua Qian. Dc-nas: Divide-and and---uu-- neural architecture search for multi-modal classification. In Proceedings of the AAAI Conference on Artificial Intelligence, pp. 13754-13762, 2024.
Chengliang Liu, Jie Wen, Xiaoling Luo, and Yong Xu. Incomplete multi-view multi-label learning via label-guided masked view-and category-aware transformers. In Proceedings of the AAAI Conference on Artificial Intelligence, pp. 8816-8824, 2023.
Gengyu Lyu, Weiqi Kang, Haobo Wang, Zheng Li, Zhen Yang, and Songhe Feng. Common- individual semantic fusion for multi-view multi-label learning. In International Joint Conference Artificial Intelligence, pp. 4715-4723, 2024a.
11

---

Published as conference paper at ICLR 2025
Gengyu Lyu, Zhen Yang, Xiang Deng, and Songhe Feng. L-vsm: Label-driven view-specific fu- sion for multiview multilabel classification. IEEE Transactions on Neural Networks and Learning Systems, pp. 1-15, 2024b.
Hongdao Meng, Yongjian Deng, Qiyu Zhong, Yipeng Wang, Zhen Yang. and Gengyu Lyu. Feder- ated multi-view multi-label classification. IEEE Transactions Big Data, pp. 1-13, 2024.
Aaron van den Oord, Yazhe Li, and Oriol Vinyals. Representation learning with contrastive pre- dictive coding. arXiv preprint arXiv:1807.03748 pp. 1-13, 2018.
Murat Sensoy, Lance Kaplan, and Melih Kandemir. Evidential deep learning to quantify classifica- tion uncertainty. Advances in neural information processing systems, pp. 1 1-11, 2018.
Qiangqiang Shen, Yongyong Chen, Changqing Zhang, Yonghong Tian, and Yongsheng Liang. Pick- and-place transform learning for fast multi-view clustering, IEEE Transactions on Image Process- ing, 33:1272-122 1284, 2024a.
Qiangqiang Shen, Tingting Xu, Yongsheng Liang. Yongyong Chen, and Zhenyu He. Robust tensor recovery for incomplete multi-view clustering. IEEE Transactions on Multimedia, 26:3856-3870, 2024b.
Xin Tan, Ce Zhao, Chengliang Liu, Jie Wen, and Zhanyan Tang. A two-stage information extrac- tion network for incomplete multi-view multi-label classification. In Proceedings of the AAAI Conference on Artificial Intelligence, pp. 55249-15257 2024.
Bing Wang, Ximing Li, Changehun Li, Shengsheng Wang, and Wanfu Gao. Escaping the neutral- ization effect of modality features fusion in multimodal fake news detection. Information Fusion, pp. 102500, 2024a.
Jiaan Wang. Jianfeng Qu, Kexin Wang, Zhixu Li, Wen Hua, Ximing Li, and An Liu. Improving the dialogue via contrastive learning. In Proceedings of the AAAI
robustness of Conference on Artificial Intelligence, pp. 19135-19143, 2024b.
Jing Wang. Songhe Feng, Gengyu Lyu, and Zhibin Gu. Triple-grann granularity contrastive learning for deep multi-view subspace clustering. In Proceedings of the 31st ACM International Conference on Multimedia, pp. 2994-3002, 2023a.
Jing Wang, Songhe Feng, Gengyu Lyu, and Jiazheng Yuan. Surer: Structure- unified graph neural network for multi-view clustering. In Proceedings of the AAAI Conference Artificial Intelligence, pp. 15520-15555 15527, 2024c.
Shiye Wang, Changsheng Li, Yanming Li, Ye Yuan, and Guoren Wang. Sulf-super information bottleneck for deep multi-view subspace clustering. IEEE Transactions on Image Processing, 32: 1555-1567 1567, 2023b.
Cai Xu, Zehui Li. Ziyu Guan, Wei Zhao, Xiangyu Song, Yue Wu, and Jianxin Li. Unbalanced multi- view deep learning. In Proceedings of the 31st ACM International Conference on Multimedia, pp. 3051-3059, 2023a.
Cai Xu, Wei Zhao, Jinglong Zhao, Ziyu Guan, Yaming Yang, Long Chen, and Xiangyu Song. Pro- gressive deep multi-view comprehensive representation learning. In Proceedings of the AAAI Conference on Artifiicall Intelligence, pp. 10557-10565, 2023b.
Cai Xu, Jiajun Si, Ziyu Guan, Wei Zhao, Yue Wu, and Xiyue Gao. Reliable conflictive multi-view learning. In Proceedings of the AAAI Conference on Artificial Intelligence, pp. 16129-16137, 2024.
Jinglin Xu, Wenbin Li, Xinwang Liu, Dingwen Zhang, Ji Liu, and Junwei Han. Deep embedded complementary and interactive information for multi-view classification. In Proceedings of the AAAI Conference on Artificial Intelligence, pp. 6494-6501, 2020.
Yun-Hao Yuan, Jin Li, Yun Li, Jipeng Qiang. Yi Zhu, Xiaobo Shen, and Jianping Gou. Learning canonical fcorrelation projection for compact multiview representation. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 19260-19269, 2022.
12

---

Published conference paper at ICLR 2025
Lotfi A Zadeh. Fuzzy sets. Information and Control, 8(3):338-352) 1965.
Chaoyang Zhang, Zhengzheng Lou, Qinglei Zhou, and Shizhe Hu. Multi-view clustering via triplex information maximization. IEEE Transactions on Image Processing, 1001299-4313, 2023a.
Chenglong Zhang, Yang Fang, Xinyan Liang, Xingyu Wu, Bingbing Jiang, et al. Efficient multi-view unsupervised feature selection with adaptive structure learning and inference. In Proceedings of the Thirty-Thire Third International joint Conference on Artificial Intelligence, pp. 5443-5452, 2024a.
Chenglong Zhang, Xinyan Liang, Peng Zhou, Zhaolong Ling, Yingwei Zhang. Xingyu Wu, Weiguo Sheng, and Bingbing Jiang. Scalable multi-view unsupervised feature selection with structure learning and fusion. In Proceedings of the 32nd ACM International Conference Multimedia, pp-
1-9, 2024b.
Wei Zhang, Zhaohong Deng, Te Zhang, Kup-Sze Choi, and Shitong Wang. One-step multiview fuzzy clustering with collaborative learning between common and specific hidden space infor- mation. IEEE Transactions on Neural Networks and Learning Systems, pp. 1-14, 2023b.
Vitalii Zhelezniak, Aleksandar Savkov, April Shen, Francesco Moramarco, Jack Flann, and Nils Y Hammerla. Don't settle for average, go for the max: Fuzzy sets and max-pooled word vectors. In International Conference on Learning Representations, pp. 1-17, 2019.
Qiyu Zhong, Gengyu Lyu, and Zhen Yang. Align while fusion: generalized nonaligned multiview multilabel classification method. IEEE Transactions on Neural Networks and Learning Systems, pp. 1010, 2024.
Wenzhang Zhuge, Hong Tao, Tingjin Luo, Ling-Li Li Zeng. Chenping Hou, and Dongyun Yi. Joint representation learning and clustering: A framework for grouping partial multiview data, IEEE Transactions on Knowledge and Data Engineering, 34(8):3826-358) 3840, 2020.
13

---

Published conference paper at ICLR 2025
A ADDITIONAL PRooFs
A.1 PROOF OF PROPOSITION
Proposition 2pro is an upper bound of Q.
Proof: Without losing generalization, we let Tpro = 1, and we have
V
Tprn
(c)
2
Spro
log
NV
22
v=l
V
K
log
NV
1= b
NV ΣΣΣ
log
log
(18)
ΣΣ
log Σ
(0) 1/0-_z) -2
h
(1)
1
NV
v= k= wi
ΣΣΣ
log KE
1 -2:
h2
(0) 2 1/2
k= w
NV ΣΣΣ
log E
22
+
h
(v)
C2
2 + log K
k=
K is constant to denote the number of classes, and we ignore it to obtain new loss Opros namely
V
e.
11
(1)
Ca 112)
log E
e
+
(19)
NV
vx=k
According to the Jensen's 's inequality, we could get
(2) (v) 22
exp
E
h
<
E
exp
(20)
Considering that the logarithmic function is monotonically increasing function, we have
V
K
2pro
E
+ To (v) Tx
Q.
(21)
NV
V=1
Am
m
A.2 PROOF OF THEOREM
Theorem When employing Min Pooling as the Comb operator, the overall variance of each class maximized.
14

---

Published as conference paper at ICLR 2025
Proof: Without losing generalization, we let Tpro
1, and we have
v
K
( Ex
2
Spro
log
NV
A -c,
V=1
| (0) -
ΣEΣ
log
h1
Cx
NV
(22)
NV ΣΣΣ
log
A1
( -z, 1|
aall2)
NV
log Σe
-c
+
Var(cu);
NV
==
where Var (Cx) stands for the variance of the k-th class cluster. Actually, it could be observed that (0)
NY EV Σ En.
h
きx = Ek= Var(Tx). In practice, we weigh the crisp
prototype more and set r) to approximate 1, then we have
K
K Var (Tx) = - Var (cx).
lim V m-1
lim
(23)
When applying the centralized representations for constructing the expanded boundary, the max- imal overall variance within each class cluster is formalized
K
K
2
max lim V max
Var(c)) max
Ck
9-+1
1=1
k
r!") 2 (n) 0
max
max
(24)
k=
N (1)
(2- ΣΣ
11)
max
where K is the number of classes. Obviously, for each representation, when each element of x(0) becomes smaller, V becomes greater, which is in line with
((2
(1(10)
( 2 [ (2-(1((0) other
1 2/111
(25)
min
min' and (,.(0)] other of Min Pooling, and other simple Comb operators such Mean Pooling, or Mar Pooling. Finally, we have
where {x((0] )
the membership degree vectors, which are the outputs
N ΣΣ D=
V
V
N
(1!00
N
ΣΣ
2 - { (((()
other о 2 10)
(26)
This means that among the simple Comb operators, Min Pooling as the Comb could maximize the overall variance of each class.
15

---

Published conference paper at ICLR 2025
B EXPERIMENT SETTINGS
B.1 DATASETS
We provide the complete statistics of the used datasets in Table 3 HandWritten and Hdigit datasets are composed of handwritten digit images, PIE 2 and YoutubeFace datasets collect the data on facial recognition and expression recognition, Scene15 dataset contains images from 15 different scene categories, CCV 0 dataset comprises videos under complex conditions, Animal dataset contains images of different animals, 100Leaves dataset is composed of images from 100 different leaves.
Table Statistics of datasets, which include the number of instances, classes, and the view dimen- sions.
Dataset HandWritten
Instances
Classes
Views
View dimensions
2,000
10
240/76/216//7//6/6
1cene15
4,485
15
20/59/40
PIE
680
68
484/256/279
CCV
6,773
20
20/20/20
Animal
11,673
20
06689/20000/2,001/////
100Leaves
1,600
100
60/60/60
Hdigit 10,000
10
784/256
YoutubeFace
101,499
64/64/512/6447888
B.2 IMPLEMENTATION DETAILS
The proposed model implemented in Pytorch and trained with an SGD optimizer. The dimen- sion settings of the encoder and decoder are formed by {D., 1.4 512, 1.2 x 512, 512) and (512, 0.6 x De, 0.8 x D.. D&} respectively. The number membership functions is set as 'Tee learning rate chosen from [1e-3, ,3e -e-e-, 5e 1e-22, and the coefficients a and B are chosen from {0.1, 0.25, 0.5, 0.75, 1}. The temperature coefficients Tins and Tpro are chosen from (0.5, 0.75, 1.0, 1.5, 2.0) and (0.5. 0.75,.7.7.. 1.0, 1.2, 1.5) respectively. All experiments are conducted
on server with NVIDIA GeForce 3090 (24 GB memory each).
B.3 TRAINING ALGORITHM
The proposed MAMC includes three main components of the proposed model, namely view- specific representation extractors, a multi-scale alignment module, and joint classifier. Among the three components, the view-specific auto-encoders abstract critical view representations from multi-view data, and the multi-scale alignment module mines the inter-view commonality and inter-class difference. The joint classifier calculates the fused predictions. Algorithm presents the detailed training process of the proposed MAMC.
SUPPLEMENTARY EXPERIMENTS
C.1 LEARNING CLEAR DECISION BOUNDARIES
Information redundancy in the multi-view data would lead to ambiguous boundaries. To further verify that MAMC has the ability learn clear decision boundaries, we implement our model on
HHandWritte:: http://archive.it www.eeei....ccuc/ini..... http://www...ccceeeee////tttttta /ttdd/atttttt//iii /ttdatasettss/liillleeeaaaee
"Hdigit: htaps://archivee www.eiic............ www.ee/////////// cdw/mm/index.,pipe phh...
SPIE: https:/www.cs wwt.:/es.c.......... www.//af/////ppoeeeeccccc
"YoutubeFace: https://www.cc http://wwwwws....... h///olff yffaces..
1Scene15: http://www-crt evrai www.eedd//onnecce weimmc/ggp/aatt/sceeeeeeeee http///aa///eeee/////
"CCV: http://www.ee.commlll
Animal: https://www.cs.uuk ucf.euu xmzhang/ datg//taatsss....
8 Leveevss https///rcciive ics ues.
16

---

Published conference paper at ICLR 2025
Algorithm Training process of MAMC
Input: Multi-view dataset: D = [({=; (1) 1 10=1. v:)l S v N}; Hyperparameters Times Tpros a, and [3 Training epochs T.
Output: Model parameters.
Initialization: Initialize the parameters of the neural network.
Process:
1: for epoch = 1 to T do
2: Calculate the view representation matrices (H())) V
with view-specific auto-encoders.
3: \\ Instance-scale alignment
4: Realize the instance-scale alignment by Eq. (2).
\\ Prototype-scale alignment
6: Centralize the view representations by Eq. (3)-(4).
Calculate the boundary elements by Eq. (8)-
8: Realize the prototype-scale alignment by Eq. (14).
\\ Classification prediction
10:
Calculate the classification loss by Eq. (15)-(16).
11: end for
100
60
100
ETMC
ETMC
50
PDNT
mmeynamics
80-
PDNT
mmaynanica
70
40
RCHL
70-
RCMIL
50
5 30
50
40
40-
20
mmndnamicss
30-
20
10
20.
10
0 Ours
10-
10
10
10
10"
10a
10c
6
10
104
12
10
10a
10
10a
10a
10a
10
10
10
Standare deviadion lut
Standard ceviation (u)
Standard deviation
(a) HandWritten
(b) CCV
(c) PIE
70
90
100
EINC
mmdynaaics
ETMC
mmddyaancs
90-
EIMC
middnnaics
PDMP
70
80:
PDMT
50-
Durs
40
40
20
20
10.
0
16t
10a
169
10F
16a
10
102
10a
10
16
10
16
102
10a
10a
16
Standard deviation (o)
Standard deviation (ol
Standard deviation (o)
(d) Animal
(e) Scene15
(f) Hdigit
Figure 5: Performance comparison multi-view datasets with different levels of noise.
manufactured ambiguous data. In our experiments, we add Gaussian noise with different levels of deviations a to half of the views, and provide the comparison results in Figure We discover that when the noise increases, the performance of all the models drops quickly since larger noise results in more ambiguous boundaries. Even so, the performance of our model descends more slowly than other models, especially when the noise is small. Actually, the data noise in the real multi-view data is not large, which demonstrates that MAMC exploits the desired diffe-eiffeeence from all the view information clear the ambiguous decision boundaries.
17

---

Published as conference paper at ICLR 2025
Talein Urin
3.5
535
Exch
(a) HandWritten
(b) Scene15
(c) PIE
(d) Animal
878
32
Lon
3.5
10
(e) CCV
(f) Hdigit
(g)
100Leaves
(h) YoutubeFace
Figure 6: The convergence analysis of MAMC across all the datasets.
C.2 Loss ANALYSIS
We conduct the convergence analysis of the proposed method across all the datasets, where ex- perimental results of loss values are illustrated in Figure 6 It is observed that the losses decrease steadily and converge between 50 and 800 epochs across all datasets. Such a phenomenon empiri- cally confirms the convergence of MAMC.
C.3 VISUALIZATION
In this experiment, we provide the t-SNE the comparative MVL models and raw data in Figure) 7 In practice, one instance is encoded several view representations, which are then concatenated as final representation vector. The final representation vector input into the t-SNE algorithm to obtain the visualization. From the results, we could find that though the data ambiguous, most models disentangle different classes to some degree. However, our model shows much better results since it mines the feature commonalities to tighten samples inside each class and exploits inter-class differences for separating different classes.
C.4 COMPLEXITY COMPARISON
We present the model complexity comparison in Table Table Table lists the complexity of all the models, and we can find that our model has the same level complexity as some compara- tive models. Table 5 provides detailed computational overhead analysis including MACs, concrete running time, and Parameters. For fair comparison, we set the same batch size and training epochs for all the models. From all the statisties, we could find that our model is not cumbersome, but is competitive with some advanced models.
Table 4: Complexity comparison between the comparative models and the proposed model.
Model
Complexity
mmdynamics
((N)
ETMC
O(N)
UMDL
(
PDMF
(
RCML
O(N)
Ours
(
18

---

Published as conference paper at ICLR 2025
(a) Raw
(b) mmdynamics
(c) ETMC
(d) PDMF
(e) RCML
(f) Ours
Figure 7: t-SNE of the comparative models and the proposed model on Scene15 dataset.
Table Complexity comparison among the comparative models and the proposed model.
Datasets
Metrics
mmdynamics
ETMC
UMDL
PDMF
RCML
Ours
Running Time
21.911s
35.739s
10057.537s
21.972s
17.064s
23.302s
HandWritten
MACs
267.871M
3.323M
1273.548M
189.989M
1.661M
1373.655M
Parameters
1.0543M
0.0066M
5.2389M
1.722M
0.0066M
5.353M
Running Time
28.99 18,996s
42.596s
31938.726s
45.678s
22.922s
41.651s
Scene15
MACs
66.604M
0.457M
132.890M
68.387M
0.457M
617.869M
Parameters
0.264M
0.002M
20.632M
1.098M
0.002M
3.197M
Running Time
11.525s
11.233s
2721.368s
9.214s
7.775s
17.499s
PIE
MACs
536.394M
17.739M
248.090M
188.088M
2.217M
996.040M
Parameters
2.099M
0.070M
1.429M
1.540M
0.070M
4.622M
Running Time
40.678s
61.986s
29783.373s
69.529s
37.905s
65.490s
CCV
MACs
56.521M
0.307M
125.338M
9.978M
0,307M
263.633M
Parameters
0.225M
0.001M
46.361M
1.331M
0.001M
3.323M
Running Time
650.847s
181.911s
52103.475
221.021s
119.074s
177,9465
Animal
MACs
7643.710M
44.493M
1269.197M
576.355M
44.493M
9360.186M
Parameters
29.873M
0.174M
141.214M
10.937M
0.174M
37.353M
Running Time
12.697s
16.173s
7386.190s
19.825s
10.078s
44.393s
100Leaves
MACs
247.370M
4.915M
142.234M
80.740M
4.915M
672.518M
Parameters
0.971M
0.020M
3.113M
0.814M
0.020M
3.323M
Running Time
57.370s
58.236s
44589.8325
85.163s
41.918s
59.256s
Hdigit
MACs
506.520M
0.666M
211.558M
42.877M
2.662M
883.593M
Parameters
1.982M
0.010M
100.825M
2.614M
0.010M
4.245M
Running Time
1660.614s
598.449s
513.350s
523.801s
702.112s
YoutubeFace
MACs
4393.923M
67.456M
1466.827M
67.456M
7988.087M
Parameters
4.328.
0.066M
15.840M
0.066M
8.544M
19


Reviews:
[{"review_id": "fUniE1UNGI", "reviewer": "Reviewer_p7J9", "summary": "The authors propose an expanded boundary using fuzzy set theory to adaptively adjust class boundaries for ambiguous data, enhancing representation tightness and reducing boundary ambiguity. This expanded boundary also preserves more margin for unseen data, improving model generalization. Experiments across diverse datasets show the model’s superiority over state-of-the-art methods.", "questions": "Please address the question above.", "limitations": "", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The technique appears sound.\n\n2. The presentation is clear and easy to follow.\n\n3. The experimental results are sufficient to demonstrate the effectiveness of the proposed method.", "weaknesses": "1. The model includes too many loss terms, which may limit its practicality for real-world applications, since it is often hard to set the hyperparameter to balance the contributions among them in practice. \n\n2. Figure 4 is unclear; I strongly suggest the authors present it in 2D for better readability.  Since there are too many dimensions making it hard to conduct the parameter sensitiveness analysis.\n\n3. Some relevant references are missing [1][2][3], since [1] and [3] also study multi-view classification tasks and [2] focus on multi-view learning from a fuzzy view. The authors should discuss and compare with them. \n[1] Multi-view Classification Using Hybrid Fusion and Mutual Distillation, WACV, 2024.\n[2] Self-Supervised Information Bottleneck for Deep Multi-View Subspace Clustering, TIP, 2023.\n[3] Multi-View Fuzzy Classification With Subspace Clustering and Information Granules, TKDE, 2023.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "iIY2BHVBYK", "reviewer": "Reviewer_1xbb", "summary": "This paper proposes a multi-scale alignment module and an expanded boundary based on fuzzy set theory to address feature heterogeneity and information redundancy in multi-view classification. The alignment module enhances feature fusion, while the expanded boundary adapts to ambiguous data, reducing boundary ambiguity and improving generalization.", "questions": "Please see weakness.", "limitations": "", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "1. The proposed MAMC model effectively addresses feature heterogeneity by employing view-specific auto-encoders and a multi-scale alignment module, which extracts common features across views, enhancing feature fusion and improving classification performance.\n2. The model introduces an expanded boundary to manage information redundancy and class ambiguity, leveraging class differences to create clearer, more adaptive decision boundaries, which aids in distinguishing classes more effectively.", "weaknesses": "1. The details of the paper need further refinement. For example, in subfigures (c) and (d) of Figure 4, the axis labels should use \"temperature coefficients\" instead of \"penalty coefficients.\"\n2. Equation 17 represents the overall optimization objective, involving four different loss functions. Why do the penalty coefficients apply to only two of these loss functions?\n3. Figure 5 in the appendix should include variance, as it is a key aspect of stability analysis.\n4. There are many minor issues that need refinement. For example, the first column in Table 1 should be labeled \"datasets\" instead of \"methods.\"\n5. The experiments were conducted using eight 24GB 3090 GPUs, but there are still out-of-memory issues reported in Table 1. This clearly indicates that the model's complexity and computational overhead require analysis and comparison, including metrics such as runtime, MACs, FLOPs, and parameters.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "K39L13RO6k", "reviewer": "Reviewer_9Q5P", "summary": "This papers addresses two issues in multi-view data: feature heterogeneity and information redundancy, by proposing a multi-view representation learning approach that enhances multi-view classification through multi-scale alignment and boundary expansion. Specifically, to address feature heterogeneity, this paper proposes a multi-scale alignment module that aligns from both instance and prototype scales, to explore commonalities from both inter-view and inter-class perspectives, thereby mitigating the issue of heterogeneity. To address information redundancy, this paper proposes a new expanded boundary method that adaptively adjusts boundaries using fuzzy set theory to handle fuzzy data. By integrating this expanded boundary with the prototype scale alignment module, the representation can be tightened and boundary ambiguity reduced, thereby enhancing the model's generalization capabilities. Experiments show that this model outperforms existing methods on multiple datasets.", "questions": "Please refer to Weaknesses.", "limitations": "", "rating": 5, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "This article utilizes fuzzy sets to propose a novel self-adaptive expanded boundary method to enhance the separability of inter-class representations and the compactness of intra-class representations in multi-view data.", "weaknesses": "1. The multi-scale alignment module proposed in this paper is an incremental approach. It is a commonly used paradigm in existing multi-view representation learning methods to enhance the discriminability and semantic consistency of representations.\n2. The comparative methods in the experimental section are not comprehensive or representative enough. A thorough investigation of related work in the past three years should be conducted to demonstrate the effectiveness of this method.\n3.One of the main problems addressed in this paper is feature heterogeneity, which is explained as the fusion difficulty caused by heterogeneity among multiple different data distributions. However, the proposed multi-scale alignment module does not provide an in-depth analysis of this issue or explain how it solves the problem.\n4. The explanation of symbols in the paper is not clear enough, such as in Eq.(16). \n5. The experimental section lacks an analysis of model convergence.\n6.The datasets have not been cited.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "I9OQAQPPNL", "reviewer": "Reviewer_8qZm", "summary": "This paper delves into the problems in multi-view data, namely fusion difficulty caused by feature heterogeneity and boundary ambiguity caused by information redundancy. To tackle these problems, this paper proposes a novel multi-view learning model MAMC, which reduces the fusion difficulty by introducing a multi-scale alignment strategy to mine the inter-view commonality and inter-class difference, while alleviating the boundary ambiguity by proposing an expanded boundary to learn a clear decision boundary. Overall, this model is interesting and works well to address the problems.", "questions": "- Though the visualization is quite good, I am confused about how they obtained the results. They should explain how to combine the multi-view features to realize t-SNE visualization, such as concatenation or other ways.\n\n- I find the comparative models in the paper mostly deal with uncertainty. Thus, I want to know whether the proposed model can be regarded as one robust model to handle uncertainty. And what are the differences between those models and MAMC?\n\n- Whether the authors give the wrong symbols in Figure 4? As the caption described, the symbols in (c) and (d) should be two temperature coefficients, and the authors need to check it up.", "limitations": "", "rating": 8, "confidence": 5, "soundness": 4, "presentation": 3, "contribution": 4, "strengths": "- The paper structure is logical and the motivation is clear and easy to understand.\n\n- The design of the expanded boundary is detailed, actually, it is interesting and seems simple-yet-effective.\n\n- The authors provide the theoretical proof to explain the construction of the expanded boundary.\n\n- The MAMC has very good performance in the classification results and visualization.", "weaknesses": "- It would be better for the authors to provide the additional results with the general prototype in the ablation study, which makes it easier to understand the advantage of the expanded boundary. \n\n- The model is simple-yet-effective, however, I don’t know whether it is efficient. So the authors could provide a complexity analysis to understand the model efficiency better.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)