CyIN: Cyclic Informative Latent Space for Bridging Complete and Incomplete Multimodal Learning
Authors: Ronghao Lin
Qiaolin He
Sijie Mai
Ying Zeng
Aolin Xiong
Li Huang
Yap-Peng Tan
Haifeng Hu
Venue: NeurIPS 2025 poster ()
Keywords: Multimodal representation learning
Incomplete multimodal learning
Missing modality issue
Information bottleneck
Cross-modal transaltion
Primary area: deep_learning

Abstract: Multimodal machine learning, mimicking the human brain’s ability to integrate various modalities has seen rapid growth. Most previous multimodal models are trained on perfectly paired multimodal input to reach optimal performance. In real‑world deployments, however, the presence of modality is highly variable and unpredictable, causing the pre-trained models in suffering significant performance drops and fail to remain robust with dynamic missing modalities circumstances. In this paper, we present a novel Cyclic INformative Learning framework (CyIN) to bridge the gap between complete and incomplete multimodal learning. Specifically, we firstly build an informative latent space by adopting token- and label-level Information Bottleneck (IB) cyclically among various modalities. Capturing task-related features with variational approximation, the informative bottleneck latents are purified for more efficient cross-modal interaction and multimodal fusion. Moreover, to supplement the missing information caused by incomplete multimodal input, we propose cross-modal cyclic translation by reconstruct the missing modalities with the remained ones through forward and reverse propagation process. With the help of the extracted and reconstructed informative latents, CyIN succeeds in jointly optimizing complete and incomplete multimodal learning in one unified model. Extensive experiments on 4 multimodal datasets demonstrate the superior performance of our method in both complete and diverse incomplete scenarios.

CyIN: Cyclic Informative Latent Space for Bridging Complete and Incomplete Multimodal Learning
Ronghao Lin 1:2 Qiaolin He', Sijie Mai Ying Zeng', Aolin Xiong', Li Huang',4 Yap-peng Tan", Haifeng Hu
School of Electronics and Information Technology. Sun Yat-Sen University, China
School of Electrical and Electronic Engineering. Nanyang Technological University, Singapore 3 School of Computer Science, South China Normal University, China Desay SV Automotive Co., Ltd, China
Pazhou Laboratory, China
[1inrh7,
(ronggallin@maailll sysu edu.
si sij jimaai @m. scnu. edu. en, cn,Li. Ln, Huang&ddessy com eyptan@ntu.. eeu.uueddu.gg........ sg, huhaif@mail ttt.ffmaall.yyu.... sysu
edu.co cn
Abstract
Multimodal machine learning, mimicking the human brain's 's ability to integrate various modalities has seen rapid growth. Most previous multimodal models are trained on perfectly paired multimodal input to reach optimal performance. real-world deployments, however, the presence of modality is highly variable and unpredictable, causing pre-trained models in suffering significant performance drops and fail to remain robust with dynamic missing modalities cireumstances. In this paper, we present a novel Cyclic INformative Learning framework (CyIN) to bridge the gap between complete and incomplete multimodal learning. Specifically, we firstly build an informative latent space by adopting token- and label-level Infor- mation Bottleneck (IB) eyclically among various modalities. Capturing task-related features with variational approximation, the informative bottleneck latents are puri- fied for more efficient cross cross-modal interaction and multimodal fusion. Moreover, to supplement the missing information caused by incomplete multimodal input, we propose cross-modal cyclic translation by reconstruct the missing modalities with the remained ones through forward and reverse propagation process. With the help of the extracted and reconstructed informative latents, CyIN succeeds in jointly optimizing complete and incomplete multimodal learning in one unified model. Extensive experiments on multimodal datasets demonstrate the superior performance of our method in both complete and diverse incomplete scenarios.
1 Introduction
To obtain the optimal multimodal performance, previous methods implicitly assume that every modality present at training will also be available at inference time. However, in real-world scenarios, multimodal data may be missing due numerous factors such as fail sensors, causing uncertainty of the presence of input modalities [1]. The multimodal model exhibit pronounced sensitivity to such incomplete multimodal input, resulting in severe performance degradation when deploying the pre-trained multimodal model in downstream inference, especially for Transformer-basee models [2]. This issue is summarized as missing modality issue [3-5], and the methods devoted to addressing are called incomplete multimodal learning methods [6-8].
Corresponding author.
1 Code is released at https://githut: com/RH- Lin CN.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Current methods focus on enhancing the robustness of multimodal models by designing various delicate modules in multimodal learning to deal with diverse missing circumstances, mainly divided into alignment and generation methods. The former leverage technologies such as contrastive learning [4, 9, 10], canonical correlation analysis [11-13] and data augmentation like mixup or noisy input [5, 7, 14] to align the representations with complete and incomplete multimodal inputs, while the latter introduce generative models such as autoencoders [15-17], variational autoencoders [18-21], graph-based networks [22, 23], and diffusion models [24] to reconstruct the missing information.
Although these methods have alleviated the missing modality issue by bridging the information gap in varying degrees, they suffer from insufficient exploitation in missing information and fask-unrelated noise interruption no matter alignment or generation [6, 24]. Moreover, previous methods typically require training separate models tailored to each possible combination of missing modalities [16, 22]. Consequently, their capacity in generalization and robustness maybe largely limited due to unknown and dynamic missing circumstances in real-world scenarios. Besides, most of previous methods inevitably sacrifice the complete multimodal performance when addressing incomplete input, failing in jointly combining complete and incomplete multimodal learning in a single model [7].
In this paper, we propose CyIN, novel Cyclic INformative Learning framework that unifies com- plete and incomplete multimodal learning within unified model. Firstly, we constructs an effective informative latent space via token- and label-level Information Bottlenecks (IB) to encourage the infor- mation flow in multimodal interaction. The former builds information bridge on token embeddings at low-level perception and adopt cyclic interaction among various unimodal representations, while the latter utilize the ground truth labels as guidance to introduce high-level semantics to the informative space. Combining these two IB objectives, we efficiently capture task-relevant features and filter out the redundant noise by sampling the bottleneck latents with variational approximation. Besides, to address missing modality issue, we introduce cross-modal cyclic translation to perform forward and reverse propagation between remained and missing modalities, thereby reconstructing the missing information in the informative space. By jointly optimizing both cyclic IB and translation process, CyIN seamlessly bridges complete and incomplete multimodal learning in a unified informative latent space. The key contributions of our paper can be summarized as:
Informative Latent Bottleneck Space. Built by token- and label-level IB, the proposed informative latent space efficiently bridge complete and incomplete learning in one unified framework, where multimodal fusion and missing information reconstruction can both benefit from the purified bottleneck latents.
Cyelic Interaction and Translation. The presented cyclic information processing progress greatly boost the performance of cross-modal interaction complete multimodal learning and enhance the translation quality in incomplete multimodal learning.
Extensive experiments on 4 datasets validate that CyIN achieves state-of- offte--f-hh-arr performance in multimodal learning and maintain robustness across various missing modality scenarios.
2 Preliminary
2.1 Complete and Incomplete Multimodal Representation Learning
Considering multimodal inputs with multiple unimodal data source X, E XI..,, J. multimodal learning aims at integrating complementary information from paired modalities u (Mo, "i,..., uy } to learn multimodal representations that can drive inference on a variety of down- total number of modalities. The performance hinges on the strategy of multimodal fusion to effectively Iverages informaiotn from each modality [27,28] which in turn requires the completeness of modalities.
stream tasks [25, 26], where lu| U denotes the
As shown in Figure 1, each modality raw input is firstly processed by modality-specifie encoders F., : Xx, F, produce unimodal representations F.,. and then merged by multimodal fusion decoders Dar {F.) + FM to generate the multimodal representations FM- With multimodal input denoted as X,, Xcompietes the process of complete multimodal learning can be formulated as:
FM- (Fo. Fi,... Fu), F, = Ex,,.)., where u E {uo, U1, .... uy}
When there are missing modalities in the multimodal inputs, denoted X. Xincomplete (X premain Xmiee ), the learning process of multimodal models is turned into incomplete multimodal

---

learning, where X Xrrmman denotes the original unimodal data imput while X denotes the zero vector without corresponding unimodal information. Thus, the incomplete multimodal representations FM in Equ. should be denoted as:
FMr DM (Fremainn pmmiss), premoin E. (Xuu, pimss 0, where u E {uo, "i, uy} (2)
The final prediction 3m obtained by MLP head on the multimodal representations ÿM M MLP(Frr). Then, the optimization objective can be computed according to the specific regression (Mean Absolute Error) or classification (Cross Entropy) tasks, denoted as:
E Ugt ÚMl+ for regression
Crask
E lygt log Darl for classification
Due to the fact that the missing information will interrupt the original multimodal fusion space [4, 22] and semantic ambiguity issue may raised by overfitting on the remained modalities [2. 14]. the performance of incomplete multimodal learning decreases significantly compared with complete multimodal learning. Thus, the demand of jointly improving complete multimodal fusion and enhancing the reconstruction performance for incomplete input with missing modalities in one unified model remains challenging for the general multimodal systems [7].
2.2 Information Bottleneck
The approach of Information Bottleneck (IB) is proposed to compress the source state S into a compact bottleneck latent B, which contains the least necessary features from S while preserves the most relevant information about the target state I' [29-31]. Using mutual information to provide the the relevance between states, such trade-off can be written the following minimization problem:
min ((S; B) 3 I(B:T)
P(B[S)
where T(-1:) denotes the mutual information between two states and the ppprr-paramtter 3 controls the trade-off degree. By introducing parameterized networks, the former mutual information term I (B; S) can be modeled by IB encoder Es + B to extract information from source state, while the second term [(T; B) can be conducted by IB decoder Dr B + T referring the target state.
For the purpose of optimizing, VIB [32] utilize variational approximation to replace the intractable mutual information terms, yielding the following upper bound:
T(S: -3 ((B:T) (Ssp((()
(BSS) ((B)))) (BBp((B) E [log qs (T]B) (5)
where (B|S) (N o²) and (BB (0, satisfy the Gaussian Distribution. Denoting loss objective Lis = (B; - 3 I(T; B), optimizing Ca, is equivalent to minimizing the upper bound in Equ. 5. Details derivations are presented in Appendix B.
Utilizing reparameterization trick [32, 33] to encourage gradient propagation, the bottleneck latents B can be sampled from pa ()) denoted as:
p O z, where 10,
Serving bridge carrying compactly relevant information between S and T, the bottleneck latent B can effectively construct an informative space control the information flow from the source state S to the target state T. More related works and background can be found in Appendix A
3 Methodology
3.1 Multimodal Informative Latent Space
As shown in Figure 1, both complete and incomplete multimodal learning require task-relevant informative representations, regardless of modeling intra- or inter-mode modal relationships. Therefore, constructing unified informative latent space can not only be beneficial in bridging the modality gap the fusion of multimodal representations, but strengthening the reconstruction of the most meaningful features among various modalities. To enhance the efficiency of informative space, we introduce two complementary IB mechanisms to encourage the compactness the bottleneck latents at both perceptual low-level and semantics high-level, referring to token- and label-level IB.
3

---

Fu
Cyelic Informative Space
Multimodal Fusion and Predict
Raw Multimodal Input
E (x)
Approcmatio
Prodecs H (.6181))
Fs
Facader by ArBIFF
Desoder de RoRajo
Audio
C
Xevepsore
Fis
Duf(F)
y
Bur the way patled off like "Wort That acak
Fy
Racoddr Pul
Decseer B. (U/FARL
(x0)
8, by 7
Language
G-
Xacoopiice
B,1B,
2" Trining Stace
B;
Coeguaa
F Fr mixe
Transhane Co pr
(x(
B.
8:
Modiag Midility
Exo
Transhaw P,
Complie Leaning
Vision
S/TE (M o. Myt R C. (1 V....
Inorggiie Lounnig
X
Figure I: Framework overview. The proposed CyIN build cyclic informative space to jointly train the complete and incomplete multimodal ealrning.
Token-level Information Bottleneck. Without losing generality, for arbitrary two modalities in multimodal inputs X,, E (Xo.... Xu ). we denote one modality input Xş as the source state while another one XT the target state in Equ. 4. Let Xs/Xi f2 ..., ........... ...) denote the sequence of
token-level inputs for modality u where S/T E {ug. "1 uy }. where denotes the sequence length. Then, producing by the corresponding modality-speciff representations can be obtained as Fs/ Fr = [fi]],= ||= R L.C. where fu denotes token embedding and C denotes the feature dimension of specific modality.
encoder Eg X, Fu, the unimodal
With IB encoder Ess Fs + Rs, the token-level bottleneck latents Bs (b/)) can be attained by applying information bottleneck all token embeddings, denoted as:
I Σ S-- 2 I
(B (KT(po(bssSt) ((b)))] - Eng (ppsp((b)))) Pr Dog go (F+())))
Since unimodal representations Fs/ Fr are concatenated by token embeddings with continuous value, formulate posterior probability (br(bb))) with IB decoder Dr Bş r Fr project bottleneck of the source representation into the target one. Besides, given po (bss N(un (o)) and gbb')) ~ N/ I). then Equ. 7 can be derived
SS ~s - (KLN(NE ()) + Ebg (Il Jr Dr(bs) [13)}
(8)
Cyclic Interaction. With unimodal representations F, ..., Fir from various modalities, the source and target modalities in token-level IB can be cyclically chosen in an iterative way to filter the redundant noise contained in each unimodal representation and enhance cross-modal interaction.
Considering the source state and target state as the same modality when Fs = Fr,, the optimization of token-level IB focuses learning mopelity-specific features and capturing intra-modal dynamics, denoted as CS-$ The two terms in Equ. 8 focuses on compressing sufficient information from each token embedding and re-projecting the original sequence, respectively.
On the other hand, when the source state and target state comes from diverse modalities with Fs * Fr, the token-level IB aims at modeling invariant information across different modalities and seizing the inter-modal dynamics. Then Equ. aims at integrating modality-shared features for the source modality embeddings to match the target one. Besides, interchanging the role of modalities as source and target states Equ. 8, referring as L and -T the modality-shared features can be further explored by such directional information flow.

---

Combing the intra- and inter-modal settings, the final loss of token-level IB can be denoted as:
1 + ( + +[bb+5)] where S/T (uo.-.,, ut:
Lib Esuu (Cib
Label-level Information Bottleneck. Except for token-level IB focusing on low-level perception, introduce label-level IB inject the supervision of high-level semantics in the information flow. For unimodal representation Fs from each modality Xs as the source state, we utilize the ground truth label Mgt (regression scores or recognition classes) as the supervised target. Given N batched samples (X'H for each modality, with IB encoder Es Fs Bş and predictor Ps Bg + Ús. the label-level bottleneck latents can be constrained with the following loss:
Cis Σ (FFg-ning
KT(po (Bs(FF) Il q (Bs))) 3 (efg)((((F))))gg
flog (v.. (U.++B)))] Bb)]
(10)
lterating computing Li through all input modalities S E {wo, "1, Y we can derive the overall label-level IB loss. In practice, for regression task, the posterior probability 4s (Mgr) (B) is formulated as qo (Vge (Bs) = l1 1a--0sl Asll = lla.e - PP then we have:
N
Lis = Est
1 N
Σ
[KL(N(Hh (o+))) (1 + EB Bgt Ps(Bs) (11) III
While for classification task with classes, the posterior probability (o is computed as go go(uge Bs) = 11 0'5 = =I1' (Bs) (Bs)) then we have:
N
Σ [KL(N(MB (ob)) (0. - EBs Σ Goe log (s( Cuo = Est N
(12)
Optimized by token- and label-level IB loss, build an informative space which controls the information flow inside and across modalities in the guidance of task-related semantics.
3.2 Cross-modal Cyclie Translation
Since the bottleneck latents learned in the informative space, task-irrelevant features and unimodal inherent noise are sufficiently filtered out. Hence, with incomplete multimodal inputs, reconstructing missing information in the built informative latent space becomes significantly easier than attempting reconstruction in the original space. Here we present the cross-modal cyclic translation with forward and reverse propagation enhance model's robustness under incomplete multimodal scenarios.
Forward Propagation. In order to reconstruct the missing information, we leverage Cascaded Residual Autoencoder (CRA) [34] as the translator I's Bs + BT across various modalities as the machine translation task [35], CRA has been demonstrated sufficiently in mitigating the modality gap and translating information from the source modality S to the target one T, denoted as:
B = I T's-T (Bs) = pa (BT Bs)
(13)
where translator I' consists of a series of stacked Residual Autoencoders (.A(()) denoted as:
RAA i i= n- RAS + 1"''RASA RAS i>
(14)
where rr denotes the output of each RA block and the last output of R.A, is the overall translated information denoted as B5 For the translation process 5 + T, we align the translated information with the original one by reconstruction loss computed as:
SS-T =1 BT B
(15)
Reverse Propagation. To improve translation performance and encourage sufficient exploration of inter-modal dynamics, further apply cyclic consistent learning [17, 36] to reverse the translated
5

---

direction of information flow as back-translan translation trick [37]. Since forward propagation denotes the reconstruction process of information flow Bs - B5-T from source target modality, we adopt reverse propagation to translate the reconstructed information back as B + Bg denoted as:
T - (B pa BB B3
(16)
where translator Ts shares the model weights with the one used in forward propagation. Thus, the reconstruction loss for the reverse propagation can be denoted as:
Bs D 11
(17)
Generalize to Multiple Remained Modalities. For multimodal learning with more than two modalities as input, the missing circumstance of modalities is highly uncertain when encountering incomplete multimodal input. When the number of remained modalities are more than one, how to effectively integrating features from these modalities to reconstruct the missing one is yet to address due to the diverse incomplete input circumstances. Thus, we present a generalizable solution to employ the paired cross-modal translator to generate missing information regardless of the number of remained modalities.
Considering data X iincoppete = {X [Xremain [ with remained {uf...., 4n and missing us modality, we aims at translating the informative bottleneck latents Bremain = (B..... B&} of the remained modalities Xremain into the latents Bmiss of missing modality X denoted as:
B = B T (B
pv (B Bj. B3)
(18)
Instead of training more translators to fit in various circumstances the remained modalities input, we leverage the additive characteristic of Gaussian Distribution to integrate the translated missing information. Since the original bottleneck latents (Bremain, Brasss B are all sampled from the informative space, we directly combine the outputs of each cross-modal translator according to the source modality and denote the translated latents of missing modalities as:
[M|
B
BTS
Σ jØ
(
= II (B.B,) j
(19)
jfL
where Brce can be denoted as the translation from a special form of Gaussian Mixture Model Irn-((Mi of). Regardless of the specific settings of remained modalities, we utilize 8 as the final translated informative latents to supplement the information of missing modality Biiss
Combining forward and reverse propagation, the objective for cross-modal cyclic translation is:
Ltron
CS-T CT- where S/T E (uu Uj,,.., ug} and S *
(20)
3.3 Complete and Incomplete Multimodal Fusion
We jointly conduct complete and incomplete multimodal learning in one unified multimodal fusion network, boosting the robustness with incomplete input and maintaining the performance with complete ones. For simplicity, we employ Multimodal Transformer [38] to present multimodal fusion decoder Da (B, Bi.... .... Bu} + Fys for bottleneck latents output by the cyclic information space. Note that the fusion module can be replaced by any networks.
Considering bottleneck latents of arbitrary two modalities { (Bj, Bx), the multimodal fusion ecoder Das consists of a series of head cross-modal attention layers, composed of:
3 CM Attention, B&) Softmax [B,W, - B&
y" = _C..catgg.....
(21)
Z, Y,H Looer where r {l, R}
M, = FeedForvard Layer
+ LayerNNorm(Z))
where H denotes the number of heads, C denotes the dimension of bottleneck latents, Wg. Wi, W., W. denotes the weight matrices and R denotes the number of mross-modal attention
layers. The output of last layer A used the bimodal fusion latent of modalities {uj, ux

---

Iteratively extracting 2 with paired modalities from u E {uo...., 40 J for both complete and incomplete multimodal learning, the final multimodal representation can be denoted as:
Dar B1,..., Bu), Das (Bremannn Bmise)) } Concat([M 1 M 1
FM
(22)
Optimization Objective To sum up, our framework involves four learning objectives, including task prediction loss Ltask in Equ. 3, token-level information bottleneck loss Lis in Equ. 9, label-level information bottleneck loss Lus in Equ. 11 or Equ. 12, and cross-modal cyclic translation loss Liran in Equ. 20. The total loss can be written as:
1 (Cris + L(sb) + +errnn
Ltotal = Lrask +
(23)
where 3 denotes the balancing trade-off weight of mutual information among bottleneck latents and representations and 7 denotes the weight of translation training for incomplete multimodal learning.
Multi-stage Training Since both complete and incomplete multimodal learning requires an effective informative bottleneck space, we divide the training process into two stages. The first stage set y to make the multimodal learning focus on the construction and stabilization of informative space under complete multimodal learning, while the second stage set y to gradually introduce the training of cross-modal translator to enhance the incomplete multimodal learning.
4 Experiments
In this section, we conduct comprehensive experiments for complete and incomplete multimodal learning on 4 datasets to evaluate the performance of the proposed framework.
Tasks and Datasets. Multimodal Regression task: MOSI and MOSEI are multimodal sentiment analysis datasets contain 2,199 and 22,856 YouTube opinion video clips, respectively, where each clip annotated with continuous sentiment score ranging from -3 (strongly negatíve) +3 (strongly positive) as Likert scale. Multimodal Classification lask: IEMOCAP dataset provides 7,369 video- recoded conversation, annotated with six emotion categories: (happy, sad, neutral, angry, excited, and frustrated]. MELD dataset includes 13,391 utterances drawn from multi-party conversations of television series Friend, labeled with seven emotions: {neutral, surprise, fear, sadness, joy, disgust, and anger). These two datasets are provided for multimodal emotion recognition.
Implementation Details. All experiments are performed on H800 GPU with Pytorch 2.4.1 CUDA 12.4. For audio and vision modality, we leverage ImageBind [39] as a feature extractor for better alignment performance. For language modality, while sBERT [41] on IEMOCAP and MELD for fair comparison with baselines. Detailed about hyper-parameters in each dataset is presented in Appendix D.
use pre-trained BERT [40] on MOSI and MOSEI
Evaluation Protocols. Following [8. 17, 22, 24], we evaluate the performance of different methods by (1) complete multimodal input (u E (l,a, v)) and under (2) fixed missing proto- ([/ v)) and (3) random missing protocol with missing rates MR E 0.2 0.3, 0.4, 0.5, 0.6, 0.7). Here E {7, a, v) denote language, audio and vision modal- ities. Note that diverse with previous methods, we leverage one unified model for evaluation on various input cireumstances with 10 runs. The practical details of missing protocols can be found in Appendix E. Details about baselines and evaluation metrics can be found in Appendix F-G.
col (u E (B/(v))((a/(LLa)))]
Quantitative Comparisons. The average results under both complete input various incomplete multimodal input circumstances are reported in Table 1. Compared with previous basclines, CyIN reaches superior performance most metrics in scenarios with both complete and incomplete multimodal input. Specifically, with the purify capability of informative space, CyIN sufficiently explore the mopelity-specifie and -shared features and conduct efficient multimodal fusion based on the bottleneck latents when all multimodal input are present.
Under fixed missing protocols, CyIN consistently outperforms baselines across fixed one or two missing modality settings, demonstrating strong generalization even without specialized tuning for any specific modalities. The ability to integrate informatio from both dominant and inferior modalities highlights the flexibility of the informative bottleneck space. Besides under random missing protocols, CyIN remarkably enhances the model's robustness to various random modality missing scenarios,
7

---

Table 1: Performance comparison between the proposed CyIN and baselines with the average results in complete modality setting E (1, a, 2} and incomplete modality settings, with fixed missing protocols including modality settings u E (T//ft//aa (L,a/aaaa/LLaaa//a f/a, v) and random missing protocols including missing rates MR E [0.1, 0.3, 0.4, 0.5, 0.7.
Setting
Models
MOSI
MOSEI
IEMOCAP
MELD
Acc77
Fif
MAE]
Cert
Acc?f
Fl1t
MAE]
Corr
Acet
wF11
Accf
wF11
CCA
27.7
74.9
1.106
0.541
16.1
82.9
0.654
0.666
61.7
61.5
51.2
46.6
DCCA
25.1
73.6
1.220
0.422
39.3
73.3
0.788
0.425
56.8
55.5
-7..7
37.0
DCCAE
19.7
69.5
1.642
0.357
38.9
73.5
0.782
0.437
57.4
56.4
48.0
36,9
CPM-Net
0.348
75.4
55.7
Complete
CRA
34.8
83.2
0.916
0.741
51.4
85.5
0.553
0.765
63.4
62.2
57.6
54.8
(1,a,))
MCTN
43.0
84.6
0.752
0.783
47.9
84.2
0.592
0.721
58.4
57.8
36.3
52.4
MMIN
GCNe
43.6
85.8
0.752
0.792
52.6
85.9
0.531
0.778
63.0
63.0
60.4
58.5
IMDe
13.8
85.7
0.724
0.796
53.8
85.1
0.532
0.756
61.4
64.8
61.1
59.7
LNLN
44.0
84.3
0.762
0.766
52.6
85.1
0.542
0.772
62.9
62.5
57.1
CytN
00.011
86.1
66.1
CCA
21.8
$7.7
L.264
0.339
43.1
69.7
0.744
0.446
45.9
41.0
49.1
39.4
DCCA
19.7
58.8
L.418
0.261
41.3
70.0
0.799
0.392
11.3
38.9
34.1
DCCAE
21.6
67.6
CPM-Net
13.1
60.1
1,383
0.352
38.6
73.8
1.095
0.139
41.3
39.8
33.8
92.8
Fised
CRA
27.3
67.8
1.158
0.404
45.4
78.5
0.672
0.593
47.9
45.9
50.7
45.4
Missing
MCTN
0.392
40.6
MMIN
31.3
68.4
L.093
0.133
46.2
77.7
0,661
0.588
50.8
50.2
53.9
43.4
GCNet
29.5
69.5
L.065
0.538
45.5
73.6
0.697
0.551
52.8
$1.9
50.4
46.7
IMDer
31.4
70.6
L.043
0.533
47.1
76.6
0.680
0,583
54.7
54.4
53.1
49.8
29.7
64.8
1.102
77.6
0.581
53.6
CyIN
32.8
72.2
1,037
0,599
47.6
78.6
0.656
0,594
57.4
56.6
54.4
49.4
CCA
23.1
66.3
L.220
0.420
44.1
74.8
0.725
0.526
49.9
49.3
48.8
39.8
DCCA
0.323
DCCAE
19.7
62.7
1.566
0.236
39.0
65.4
0.810
0.345
43.2
41.7
47.1
34.3
CPM-Net
16.9
63.9
1.340
0.325
34.3
74.0
1.497
0.078
53.1
53.7
41.6
33.6
CRA
28.6
0.558
Missing
MCTN
30.4
67.2
L.052
0.573
45.4
73.2
0.692
0.550
35.3
37.5
52.4
44.1
MMIN
33.3
70.9
L.014
0.584
47.5
79.3
0.644
0.635
49.7
49.7
53.9
45.8
GCNet
33.8
73.8
0.623
46.6
79.2
0.680
0.630
55.3
58.3
47.7
34.6
74.9
0.644
78.9
55.8
56.1
49.5
LNLN
34.2
72.8
0.978
0.627
47.9
79.2
0.639
0.642
55.5
55.8
51.7
49.0
CyIN
35.0
75.7
0.943
0.650
48.3
79.9
0,633
0.650
57.5
57.5
54.8
50.5
with minimal performance loss even at severe missing rate. The result illustrates broader application of CyIN for downstream multimodal tasks as in real-world where the presence of modalities highly dynamic and unpredictable. More results and analysis are reported in Appendix H
Qualitative Comparisons. We project the features from test set of MOSI dataset into 3D t-SNE space [42]. As shown in Figure 2(a), we firstly visualize the distribution of translated B and original unimodal latents B., to show the reconstruction quality. The cross-modal translated latents closely clustered to the original one, illustrating the effectiveness of CyIN in transferring information across various modalities despite the huge modality gap.
Besides, we present multimodal latents Fas with and without reconstructed bottleneck gree with random missing MR = 0.7. We can observe that inferring with missing modalities leads to serious interference to the multimodal representations, while the reconstructed latents can competently supplement the missing information during multimodal fusion, yielding better clustering result.
Case Study. As shown in Figure 2(b), we compare model predictions with and without CyIN on some examples from test set of MOSI and IEMOCAP datasets under fixed missing modality protocol. In samples and #3, removing the semantic utterance causes the pretrained model to make incorrect, even opposite, predictions, which implicitly showing the dominant role of language modality in multimodal affective computing tasks [8, 16, 43, 44]. On the other hand, the presence of other modalitics also have delicate contribution in the accuracy of prediction, referring to the subjective biased prediction when missing audio or vision modality in sample #2 and #4. contrast, by reconstructing the missing information with the proposed CyIN, predictions become more stable and precise, demonstrating the superiority and robustness of the informative bottleneck space.
Ablation Study. We conduct ablation study with the proposed modules on MOSI and IEMOCAP datasets with complete and the most severe random missing protocol MR 0.7, as shown in Table 2. Token- and label-level IB are both crucial for constructing the informative latent space, as removing either results performance degradation. The cross-modal cyclic interaction and translation mainly effect in capturing modality- shared features in multimodal fusion and enhancing
8

---

lapui Hlanguage cision, sudie))
Medal
Ground Trucs
Predict CytN
Predict weo
"And Ie. Edkes' know, be meybee ao mai when buis denn't Linow"
Spoculative Wenndiing Time
"So liink
simply you hoppen
Angry
Frestrate
Accustoryy and Agliated Tose
"You wed have inualee You was't lave troble."
Happy
Happy
Neuual
13
Caln
(a) Feature Distribution
(b) Case Study MOSI (#1-2) 2) and IEMOCAP (#3-4)
Figure 2: (a) Feature distribution of translated unimodal latents and multimodal latents and (b) Exam- ples on the test set of MOSI and IEMOCAP datasets when inferring with and without reconstructed information from CyIN. The / and X in modal status denotes the remained and missing modalities.
incomplete reconstruction, respectively. Moreover, we demonstrate the efficiency of informative bottleneck space in both complete and incomplete multimodal learning. Without the constrain of bottleneck, the task-irrelevant redundancy and heterogeneous noise in the original feature space raises diffieulty in integrating information from various modalities and reconstructing missing modalities. Lastly, reconstructed from the remained modalities, the translated latents can productively increase the robustness missing modality issue especially in severe incomplete input circumstance.
Table 2: Ablation study of the proposed CyIN on MOSI and IEMOCAP dataset with complete multimodal settings { a, o and random missing protocols with missing rates MR 0.7.
Setting
Model Variants
MOSI
IEMOCAP
Acc7t
FI1
MAE]
Corri
Acct
wF1t
CyIN w/o Cui
48.0
86.3
0.712
0.801
66.0
43.9
84.3
0.737
0.789
65.2
64.8
Complete (1,,,0)
w/o Cisb
47.3
85.4
0.693
0.800
6336
63.9
w/o Cyclic Interaction
43.4
85.9
0.742
0.795
64.3
63.6
w/o Cyclic Translation
43.3
85.7
0.743
0.797
65.2
65.0
Informative Space
42.0
83.1
CyIN
28.0
65.9
1.117
0.530
48.6
49.0
w/o Lub
27.3
63.5
1.223
0.509
46.3
46.2
Random Missing 0.7
w/o Cus
25.1
63.7
43.2
42,9
w/o Cyclic Interaction
26.5
67.8
1.134
0.473
47.8
48.6
Cyclic
24.5
wio Informative Space
23.7
62.9
1.240
0.430
44.1
43.6
w/o Translated Latents
23.4
56.7
1.299
0.441
41.2
39.2
Hyper-parameter Sensitivity. empirically evaluate effect various hyper-parameter settings of [3 and in loss function of Equ. 23 under MR = 0.7. The results in Table 3 shows that setting 3 3= --32 32 gives better performance as a balanced trade-off for mututal information between [(S: B) and IB; where improper bottleneck strength harms performance. Besides, setting 10 yields the best overall performance across metrics, indicating that moderate reconstruction enforces better consistency across modalities. We observe that small 7 results in weak regularization, making hard to align modalities under severe missing conditions. contrast, a large hurts performance by focusing too much consistency and ignoring the need to effectively build informative space.
Moreover, to ensure generality and stability under diverse missing modality scenarios, CyIN assumes each modality have equal contribution in multimodal learning. However, in practice, different modalities may contribute unequally depending on the task [45]. For instance, language plays dominant role in multimodal sentiment analysis, as highlighted by prior works [8, 46, 47]. To

---

sensitivity on 3 and y of CyIN on random missing protocols with missing rate MR = 0.7.
Table 3:
Table 4: Hyper-parameter senttttt on differ- ent ratios of CRA layer for translation among language, vision, audio modalities of CylN on MOSI dataset on random missing proto- cols with missing rate MR 0.7.
MOSI
Model Variants
Acc71
FIt
MAEL
Currt
6=2
24.6
59.4
1.248
0.461
8=4
27.5
61.3
1.226
0.478
# Layer (la:vaa))
MOSI
8-8
30.0
63.6
1.199
0.519
1=10
8=16
28.0
65,9
1.117
0,530
Acc71
FIf
MAEL
Corrt
8=32
26.4
66.0
1.118
0,506
1:1:1
28.0
65.9
1.117
0.530
8=64
23.3
64.1
1.206
0.463
2:2:1
28.5
67.2
1.122
0.521
1y=
25.8
64.1
1.197
0,437
4:4:1
28.7
67.5
1.114
0,532
27.5
63.1
1.135
0.488
1:1:2
27.5
66.3
1.119
0.531
8=16
7=10
28.0
65.9
1.117
0,530
1:1:4
27.4
65.6
1.116
0.534
1=
26.8
62.9
1.272
0.519
2=
25.5
60.4
1.311
0.490
partially address this, we conduct experiments by varying the number of CRA layers in the cross- modal translation modules in Table 4. Specifically, allocated more layers reconstruct the language modality comparing with audio and vision modalities, reflecting its higher importance and allowing the model learn representation with more semanties. This design show the flexibility of CyIN architecture without hardcoding any modality-specific priors in it.
Computational Efficiency. Compared with the ofate-of-the-arr methods, the proposed CyIN achieves the lowest total parameter count and significantly lower FL.OPs as shown in Table 5. The inference time of CyIN 3x faster than GCNet and 5x faster than IMDer. Due the cyclic translation process during training, the training time of CylN is slightly slower than GCNet which constructs graph neural networks for modality reconstruction, but still faster then IMDer which utilizes score- based diffusion model for cross-modal generation.
Table 5: Comparion of training and inference computation efficiency on MOSI dataset.
Model
Total Param (M)
Total Training Time
Inference Time (fiteration)
FLOPs (T)
GCNet
144,34
1.47h
70.65 103.75s
3.747
IMDer
168.31 M
1.89h
5.466
CyIN (ours)
123.49
1.61h
22.41s
1.594
5
Conclusion
In this paper, we present novel framework named CyIN, which constructs an informative latent space to jointly conduct complete and incomplete multimodal learning. Guided by token- and label- level Information Bottlenecks, CyIN succeeds in learning a compact yet semantically rich bottleneck latent which purifies task-related features and improves robust multimodal fusion. The proposed cyclic interaction and translation mechanism further encourage sufficient exploration in inter-modal dynamics and enhances the reconstruction quality in missing modalities. Comprehensive experiments on 4 multimodal datasets demonstrate that CyIN not only surpass previous methods in complete multi- modal learning but also retains superior performance and stability under various incomplete scenarios, highlighting its effectiveness and generalization capacity for real-world multimodal applications.
Limitations
Our framework still has several limitations. 1) It treats all modalities equally to improve the gen- eralization, while ignoring possible imbalance contribution from various modalities. We will try to adaptively allocate weights according to the information abundance of each modality. 2) The performance current cross-modal translators can be replaced by more recent generative approaches such as diffusion models or flow-based models to attain optimal reconstruction performance. 3) We suppose broader training on diverse multimodal understanding datasets or integration with Large Language Models could strengthen the generalization and scalability of the proposed framework.
10

---

Acknowledgments
This work was supported by the National Natural Science Foundation of China (62076262, 61673402, 61273270, 60802069) and by the International Program Fund for Young Talent Scientific Research People, Sun Yat-Sen University.
References
[1] Renjie Wu, Hu Wang, Hsiang-Ting Ting Chen, and Gustavo Cameiro. Deep multimodal learning with missing modality: A survey. arXiv preprint arXiv:2409.07825, 2024. 1,
[2] Mengmeng Ma, Jian Ren, Long Zhao, Davide Testuggine, and Xi Peng. Are multimodal transformers robust to missing modality? In 2022 IEEE/CVE Conference Computer Vision doi: 10.1109/CVPR5268..022
and Pattern Recognition (CVPR), pages 18156- 18165, 2022.
01764. 1, 18
[3] Mengmeng Ma, Jian Ren, Long Zhao, Sergey Tulyakov, Cathy Wu, and Xi Peng. Smil: Multimodal learning with severely missing modality. Proceedings of the AAAI Conference on Artificial Intelligence, 35(3):2302-2310 May 2021. doi: 10. (609/aaai.v3Sin 16330. URL bttps://ojst aaai ww.ooo////ioo. index. php/ /AAT/article/viee//aaaaaaaaaaaaa
[4] Ronghao Lin and Haifeng Hu. MissModal: Increasing Robustness to Missing Modality in Multimodal Sentiment Analysis. Transactions of the Association for Computational Linguistics, 11:1686-1702 2023. ISSN 2307-387X. doi: 10. 10.1112/tacl. 00628. URI. htRps:::::/ //doi.
00628. 2, 18
[5] Ziqi Yuan, Yihe Liu, Hua Xu, and Kai Gao. Noise imitation based adversarial training for robust multimodal sentiment analysis. IEEE Transactions on Multimedia, 26:529-539, 20244 doi: 10.11090.M..2223.200717
1,2
[6] Mingeheng Li, Dingkang Yang, Yang Liu, Shunli Wang, Jiawei Chen, Shuaibing Wang, Jinjie Wei, Yue Jiang, Qingyao Xu, Xiaolu Hou, Mingyang Sun, Ziyun Qian, Dongliang Kou, and Lihua Zhang. Toward robust incomplete multimodal sentiment analysis via hierarchical repre- sentation learning. The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024. URL https: http:///pppereiieeeee net.e wttp://wwweeeooounniiiooooo forum?1d==gTTTH996 1,2
[7] Ronghao Lin and Haifeng Hu. Adapt and explore: Multimodal mixup for representation learning. Information Fusion, 105:102216, 2024. ISSN 1566-2535. doi: https://doi.org/10
202. URL https: /twpp::///www.. scieceedreeec................ www.s//.//./////eecceceoooccceee wttpenccaeaaaaae/ecceeeeceeteeel p1i/S15662535230652888 2,
[8] Haoyu Zhang, Wenbin Wang, and Tianshu Yu. Towards robust multimodal sentiment analysis with incomplete data. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024. URL httt:::///ppereeeee 29
nwt/forum? httt:/forumidddmE...... iddiiddemmgaiii c7GRA. 1, 8, 9, 24, 25,
[9] Petra Poklukar, Miguel Vasco, Hang Yin, Francisco S. Melo, Ana Paiva, and Danica Kragic. Geometric multimodal contrastive representation learning. Kamalika Chaudhuri, Ste- fanie Jegelka, Le Song, Csaba Szepesvari, Gang Niu, and Sivan Sabato, editors, Proceed- ings of the 39th International Conference on Machine Learning, volume 162 Proceed- ings of Machine Learning Research, pages 17782-17800, PMLR, 17-23 Jul 2022. URL http::s//proceedings mlr. https//vv12pppoooooeemm
html. 2, 18
[10] Rui Liu, Haolin Zuo, Zheng Lian, Björn W. Schuller, and Haizhou Li. Contrastive learning based modality-invarian feature acquisition for robust multimodal emotion recognition with missing modalities. IEEE Transactions Affective Computing, 15(4):1856-187) 2024. doi: 10.109/AAFF222043378777,
[11] Harold Hotelling. Relations Between Two Sets of Variates, pages 162-190. Springer New York, New York, NY, 1992. ISBN 978- -4612-4380-, doi: 10. 10.1007/978- https://dols otg/10..//// 1007 1-11-1-4111--------1-1- 9_100-9911... 25
1-4612-4380-- 14. URL

---

[12] Galen Andrew, Raman Arora, Jeff Bilmes, and Karen Livescu. Deep canonical correlation analysis. In Proceedings of the 30th International Conference on International Conference on Machine Learning- Volume 28, ICML'13, page 111-1247-111-255. JMI..orrg 2013. 25
[13] Weiran Wang, Raman Arora, Karen Liveseu, and Jeff Bilmes. On deep multi- representation learning. In Proceedings of the 32nd International Conference on International Conference on Machine Learning- Volume 37, ICML' 15, page 1083-1092. JML..org. 2015. 2,25
[14] Devamanyu Hazarika, Yingting Li, Bo Cheng. Shuai Zhao, Roger Zimmermann, and Soujanya Poria. Analyzing modality robustness in multimodal sentiment analysis. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguisties: Human Language Technologies, pages 685-695-66969 Seattle, United States, July 2022. Association for Computational Linguistics. doi: 10.18653/v (22022.naac)) main.50. URL attps://aclanthollo org/ 2222/202..22 maacl-main 50. 2, 18
[15] Yoshua Bengio, Pascal Lamblin. Dan Popovici, and Hugo Larochelle. Greedy layer- wise training of deep networks. In B. Schölkopf, J. Platt, and T. Hoffman, ed- itors, Advances Neural Information Processing Systems, volume 19. MIT Press, 2006. URL https://proceeddinngg neurips cc/paper.........eeoeeeeeeee...e files/ patpe///pppere////11eee 00071aa99000771612222e2223633330
pdf. 2, 25
[16] Hai Pham, Paul Pu Liang, Thomas Manzini, Louis-Philippe Morency, and Barnabás Póczos. Found in translation: Learning robust joint representations by cyclic translations between modalities. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pages 6892-6899, 2019. 2, 8, 25,
[17] Jinming Zhao, Ruichen Li, and Qin Jin. Missing modality imagination network for emotion recognition with uncertain missing modalities. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 1 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 2608-2618, Online, August 2021. Association for Computational Linguistics. doi: 10.18653/v1 1/2021 loll- URL org/ 10021.acll long. 203. 5, 7, 18, 24, 25
[18] Mike Wu and Noah Goodman. Multimodal generative models for scalable weakly-supervisee learning. In Proceedings of the 32nd International Conference on Neural Information Processing Systems, NIPS' 18. page 5580-5500 5590, Red Hook, NY. USA, 2018. Curran Associates Inc 2
[19] Yuge Shi, N. Siddharth, Brooks Paige, and Philip H. S. Torr. Variational mixture-of offere--o--ppperr autoencoders for multi-moda deep generative models. Curran Associates Inc., Red Hook, NY, USA, 2019.
[20] Miguel Vasco, Hang Yin, Francisco S. Melo, and Ana Paiva. Leveraging hierarchy in multimodal generative models for effective cross-modality inference. Neural Networks, 146:238-255, 2022. ISSN 0893-6080. doi: https://doi.org// 100 www.///..enee..221....... //www. www.:scieeeediiriee cww/ http//cicncearaiclleeeeo/// http://ca..aatce/////////////000
URL https:
[21] Miguel Vasco, Hang Yin, Francisco S. Melo, and Ana Paiva. How to sense the world: Leveraging hierarchy in multimodal perception for robust reinforcement learning agents. In Proceedings of the s International Conference on Autonomous Agents and Multiagent Systems, AAMAS
'22, page 1301-1309, Richland, SC, 2022. International Foundation for Autonomous Agents and Multiagent Systems. ISBN 978 1553922136... 2
[22] Zheng Lian, Lan Chen, Licai Sun, Bin Liu, and Jianhua Tao. Genet: Graph completion network for incomplete multimodal learning in conversation. JEEE Transactions on Pattern Analysis and Machine Intelligence, 45(7):8419-8433 2023. doi: 10. 10.1109/TAAM.22333225555 2, 18, 24, 25
[23] Yuntao Shou, Tao Meng. Wei Ai, Fuchen Zhang, Nan Yin, and Keqin Li. Adversarial alignment and graph fusion via information bottleneck for multimodal emotion recogni- tion in conversations. Information Fusion, 112:102590, 2024. ISSN 1566-2535. doi: http:///aiiooogll.....ddoaaiaaia science/attcce///ss//16655535333 2
URL www.hhttsp///e http://wwwwwwwwwweeeeeeeeeee/eee
com/
12

---

[24] Yuanzhi Wang. Yong Li, and Zhen Cui. Incomplete multimodality emotion recognition.
In Teirth- Conference on Neural Information Processing Systems, 2023. URL https:
/ //openreview Sesefformm2d-BBdPPUSSRR
2, 18, 24, 25
[25] Xiao Wang, Guangyao Chen, Guangwu Qian, Pengcheng Gao, Xiao- Yong Wei, Yaowei Wang, Yonghong Tian, and Wen Gao. Large-scale multi-modal pre-trained models: A comprehensive survey. Machine Intelligence Research, 20(4):447-4822 2023. ISSN 2731-538X, doi: 10. s11633-022- 1410-8. URL https::////// $11633-022- 1410-8. 2, 18
mi resear net/ http////aaracee/o/////o...
[26] Duzhen Zhang, Yahan Yu, Jiahua Dong, Chenxing Li, Dan Su, Chenhui Chu, and Dong Yu. MM- LLMs:
Recent advances in MultíModal large language models. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar, editors, Findings of the Association for Computational Linguistics: ACL 2024, pages 12401-12430, Bangkok, Thailand, August 2024. Association for Computational Linguistics. doi: 10. 101118351///222/ findiing.innnss---- ac1. acn.ings-acl.738. 2024. www.inggn-n-iii acl. 738/. 2
URL https: https://aclant:e http://laaaantoogy....ooo org/
[27] Dimitris Gkoumas, Qiuchi Li, Christina Lioma, Yijun Yu, and Dawei Song. What makes the difference? an empirical comparison of fusion strategies for multimodal language analy- sis. Information Fusion, 66:184-199 2021. ISSN 1566-2535. doi: https://di..rr//I 1016/
Linffus.2020.002 005. URL https: /www//wwwo..oo////oo///. wwwte//wwwcccccceeeeettcccc www.ee/eeeeeee..ooo/ccaeeaaeaeee $1566253520303675. 2, 18,25
(28) Muhammad Arslan Manzoor, Sarah Albarri, Ziting Xian, Zaiqiao Meng, Preslav Nakov, and Shangsong Liang. Multimodality representation learning: A survey on evolution, pretraining and its applications. ACM Trans. Multimedia Comput. Commun. Appl., 20(3), October 2023. ISSN 1551-6857
doi: 10.1145/3617833 URL https:t/////... //doi.///oiio/ /00/10. 1145/3617833 2
[29] Naftali Tishby, Fernando C. Pereira, and William Bialek. The information bottleneck method. In Proc. of the 37-th Annual Allerton Conference on Communication, Control and Computing. pages 368-377, 1999. URL https: /ttp///axvvvvvv.e///i/eeeeea.ava org/ abs/ hbs//pbysicss0004057. 3,
[30] Naftali Tishby and Noga Zaslavsky. Deep learning and the information bottleneck principle. In 2015 IEEE Information Theory Workshop (ITW), pages 1-5, 2015. doi: 10. 109/TT.2005 7133169. 18
[31] Andrew Michael Saxe, Yamini Bansal, Joel Dapello, Madhu Advani, Artemy Kolchinsky, Brendan Daniel Tracey, and David Daniel Cox. On the information bottleneck theory of deep learning. In International Conference on Learning Representations, 2018. URL https: //openreview.o
-eaidd-yyyy_GG---p 3, 18
[32] Alexander A. Alemi, Ian Fischer, Joshua V. Dillon, and Kevin Murphy. Deep variational information bottleneck. In International Conference on Learning Representations, 2017. URL https://openreviee
hiw.mmmddddxxxxxxeepppe 3, 18
[33] Max Welling Diederik Kingma. Auto-encoding variational bayes. 2nd International Conference Learning Representations, ICLR 2014, 2014.
[34] Luan Tran, Xiaoming Liu, Jiayu Zhou, and Rong Jin. Missing modalities imputation via cascaded residual autoencoder. In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition (CVPR), July 2017. 5,
[35] Dzmitry Bahdanau, Kyung Hyun Cho, and Yoshua Bengio. Neural machine translation by jointly learning to align and translate. January 2015. 3rd International Conference on Learning Representations, ICLR 2015: Conference date: 07-05-2015 Through 09-05-2015. 5
[36] Zilong Wang, Zhaohong Wan, and Xiaojun Wan. Transmodality: An end2end fusion method with transformer for multimodal sentiment analysis. Proceedings of The Web Conference 2020, WWW '20, page 2514-2520, New York, NY, USA, 2020. Association for Computing Machinery. ISBN 9781 4503702333 doi: 10. 0144533366423338000 URL pesttps://doo org/ 10. 11445/33666423 3380000. 5
13

---

[37] Cong Duy Vu Hoang, Philipp Koehn, Gholamreza Haffari, and Trevor Cohn. Iterative fack-translation for neural machine translation. In Proceedings of the 2nd Workshop on Neural Machine Translation and Generation, pages 18-24. Association for Computational Linguisties, July 2018. URL hattss:/ssitet google. com/site/ petsstteeettttiieeeeeeeeeeeete w/ntt 1 18/mom/ https: //sites. google- com/ wttpe///////./////meeeeeemee/eee 2nd Workshop on Neural Machine Translation and Generation, WNMT 2018; ; Conference date: 15-07-2018 Through 20-07-2018.
[38] Yao-Hung Hubert Tsai, Shaojie Bai, Paul Pu Liang, J. Zico Kolter, Louis- Chiii--hiliippe Morency, and Ruslan Salakhutdinov. Multimodal transformer for unaligned multimodal language sequences. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 6558-6569, Florence, Italy, July 2019. Association for Computational Linguistics. doi: 1118655311/1199/ 1656. URL https: /aa/lalaanttee //t///aaantooogoooooroooooooaaee
1656. 6
[39] Rohit Girdhar, Alaaeldin El-Nouby, Zhuang Liu, Mannat Singh, Kalyan Vasudev Alwala, Armand Joulin, and Ishan Misra. Imagebind: One embedding space to bind them all. In Proceedings of the IEEF/CVF Conference on Computer Vision and Puttern Recognition (CVPR), pages 15180-15 1590,, June 2023.
[40] Jacob Devlin, Ming- Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: Pre-training of deep bidirectional transformers for language understanding. Proceedings of the 2019 Confer- ence of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume (Long and Short Papers), pages 4171-4186, Minncapolis, Minnesota, June 2019, Association for Computational Linguistics. doi: 10. 11166//11/11114222 URL https: / http//ccaannnhhooo org/ N19 1423. 30
[41] Nils Reimers and Iryna Gurevych. Mentence-BERT: networks. In Kentaro Inui, Jing Jiang, Vincent Ng. and Xiaojun Wan, editors, Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 3982-3992, Hong Kong, China, November 2019. Association for Computational Linguistics. doi: 10.18653/v1/D199 1410019-.1..... URL https:://alllntttoo////// www.eeaooooooooooooooooe---. org/D19-1410
Sentence embeddings using Siamese BERT-
[42] Laurens Van der Maaten and Geoffrey Hinton. Visualizing data using t-sne. Journal of Machine Learning Research, 9(86):557--260)) 2008. URL http: //jmrr..///r/// org htggpapers/v9/
vandermaaten08a. html. 8
[43] Devamanyu Hazarika, Roger Zimmermann, and Soujanya Poria. Misa: Modality-ingg inntarattnn and-specific representations for multimodal sentiment analysis. In Proceedings of the 28th ACM International Conference Multimedia, pages 1 1122- 1131, 2020. URL https://d/doi org/ N0.1145/339411 3413677......... 8, 18,
[44] Ronghao Lin and Haifeng Hu. Dynamically shifting multimodal representations via hybrid- modal attention for multimodal sentiment analysis. IEEE Transactions on Multimedia, 26: 2740-2755, 2023. doi: 0011099TM..22223.3300
8, 29
[45] Xiaokang Peng, Yake Wei, Andong Deng, Dong Wang, and Di Hu. Balanced multimodal learning via on-the-fly gradient modulation. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, 2022. 9
[46] Ronghao Lin and Haifeng Hu. Multi-task momentum distillation for multimodal sentiment analysis. IEEE Transactions on Affective Computing, pages 1-18, 2023. doi: 10.1109/TAFFC. 2023.3282410. 9, 18
[47] Zheng Lian, Licai Sun, Yong Ren. Hao Gu, Haiyang Sun, Lan Chen, Bin Liu, and Jianhua Tao. Merbench: A unified evaluation benchmark for multimodal emotion recognition. arXiv preprint arXiv:2401.03429 2024. 9, 18
[48] Wei Han, Hui Chen, and Soujanya Poria. Improving multimodal fusion with hierarchical mutual information maximization for multimodal sentiment analysis. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 9180-9192, Online and Punta Cana, Dominican Republic, November 2021. Association for Computational Linguisties. doi: 10. 00.1863/vv/2021 10155//11/221...n--maaiiii memnlp-mann 723. URI https:://///t.. hattps://aclanthhtt awlaaaaaannoogyyyooaa org/ 2021. emnlp-man main. 723. 18
14

---

[49] Haoyu Zhang. Yu Wang, Guanghao Yin, Kejun Liu, Yuanyuan Liu, and Tianshu Yu. Learning language-guun guigdadggggdddddddee
guided adaptive hyper-modality representation for multimodal sentiment analysis. In Proceedings of the 2023 Conference on Empirical Methods in Natural Longuage Processing, pages 756-767, Singapore, December 2023. Association for Computational Linguisties.
[50] Xiongye Xiao, Gengshuo Liu, Gaurav Gupta, Defu Cao, Shixuan Li, Yaxing Li, Tianqing Fang. Mingxi Cheng, and Paul Bogdan. Neuro-inspired information-theoree hierarchical perception for multimodal learning. In The Twelfth International Conference on Learning Representations, 2024. URL https: http//pppeeevieieeee wwweenrvviw..eeee
18
[51] Guimin Hu, Ting-En Lin, Yi Zhao, Guangming Lu, Yuchuan Wu, and Yongbin Li. UniMSE: Towards unified multimodal sentiment analysis and emotion recognition. In Yoav Goldberg. Zornitsa ozareva, and Yue Zhang, editors, Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 7837-7851, Abu Dhabi, United Arab Emi- rates, December 2022. Association for Computational Linguisties. doi: 10. 11.1886//11/222. emnlp-main.534 URL https::/acllanthology/ org/2022. emnlp-main. 534. 18
[52] Dongyuan Li, Yusong Wang, Kotaro Fumakoshi, and Manabu Okumura. Joyful: Joint modality fusion and graph contrastive learning for multimoda emotion recognition. In Houda Bouamor, Juan Pino, and Kalika Bali, editors, Proceedings of the 2023 Conference Empirical Methods in Natural Language Processing, pages 16051-16069 Singapore, December 2023. Association for Computational Linguistics. doi: acl aclanthology wtgg/222.emmll-maiii. www.pmmain..96..
1ttp:///22222eeee w/t.23..mmmmmmmaii..... 996. URL https://
[53] Ronghao Lin, Ying Zeng, Sijie Mai, and Haifeng Hu. End-to-end semantic-centric video-based multimodal affective computing. arXiv preprint arXiv:2408 076694, 2024. 18
[54] Zebang Cheng, Zhi-Qi Cheng, Yan He, Kai Wang. Yuxiang Lin, Zheng Lian, Xiaojiang Peng, and Alexander G Hauptmann, Emotion-LLLMMA: Multimodal emotion recognition and reasoning with instruction tuning. In The Thirtt-eighth Annual Conference on Neural Informa- tion Processing Systems, 2024. URL https: /ttp::::///ppeeeeeeeeeee www.vviiee.nneeeeeeeeeeee 18
[55] Jin Li, Shoujin Wang, Qi Zhang, Shui Yu, and Fang Chen. Generating with fairness: A modality- diffused counterfactual framework for incomplete multimodal recommendations. In Proceedings of the ACM on Web Conference 2025, WWW '25, page 2787-2798, New York, NY, USA, 2025. Association for Computing Machinery. ISBN 9798400712746, doi: 10.1145/3694410.774444 URL /eattps://ool org/ 10. /144///////4111./ .0116411...1466000 3714606. 18, 30
[56] Vittorio Pipoli, Alessia Saporita, Kevin Marchesini, Costantino Grana, Elisa Ficarra, and Federico Bolelli. Im-fuse: A mamba-basee based fusion block for brain tumor segmentation with incomplete modalities. In James C. Gee, Daniel C. Alexander, Jaesung Hong, Juan Eugenio Iglesias, Carole H. Sudre, Archana Venkataraman, Polina Golland, Jong Hyo Kim, and Jinah Park, editors, Medical Image Computing and Computer Assisted Intervention MICCAI 2025, pages 225-235, Cham, 2025. Springer Nature Switzerland. ISBN 978-3-033-009884-1 18, 30, 31
[57] Paul Liang, Amir Zadeh, and Louis-Philippe Morency. Foundations & trends in multimodal machine learning: Principles, challenges, and open questions. ACM Comput. Surv. apr 2024. ISSN 0360-0300. doi: 10. 145/3656588. URL https: //tp:://doioooooo org./i...oooooooooooooooooo wrg/o0..//10..... 1145/3656586 Just Accepted. 18
[58] Fei Zhao, Chengeui Zhang, and Baocheng Geng. Deep multimodal data fusion. ACM Comput. Surv, 56(9), April 2024. ISSN 0360-0300. doi: 10. 10.1144 145/344444 URL https::///d/oo org/ 10. 10.1145/3649447. 18
[59] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, & ukasz Kaiser, and Illia Polosukhin. Attention is all you need. In I. Guyon, U. Von Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Gar- nett, editors, Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc., 2017. URL. http.httts:/:///otottt ptt.pp:e//pooceeingge www.eeiiiinnnnniinnnneeeeeee wwwurips..c/pppp//20000
Paper, pdf. 18
15

---

[60] Peng Xu, Xiatian Zhu, and David A. Clifton. Multimodal learning with transformers: A survey, IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(10):12113-12122 2023. doi: 10.1 101100//M..2223..222
18
[61] Ruohong Huan, Guowei Zhong, Peng Chen, and Ronghua Liang. Unimf: A unified multimodal framework for multimodal sentiment analysis in missing modalities and unaligned multimodal sequences. IEEE Transactions on Multimedia, 16:5753-5766 2024. doi: 10.1100TMMM.200 3338769. 18
[62] Jingjing Tang, Qingqing Yi, Saiji Fu, and Yingjie Tian. Incomplete multi-view learning: Review, analysis, and prospects. Applied Soft Computing, 153:111278, 2024. ISSN 1568-4946. doi: https://bbi.og///10.01116....... sci cciecce//rriicee/p///16666666660
URL http:///wwwwicccceeeeeettttt com/ 18
[63] Changqing Zhang, Yajie Cui, Zongbo Han, Joey Tianyi Zhou, Huazhu Fu, and Qinghua Hu. Deep partial multi-view learning. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(5):2402-2415, 2022. doi: 10.1 1.111//TPAMI..2000007777
18, 25
[64] Ravid Shwartz-Zi Ziv and Naftali Tishby. Opening the Black Box of Deep Neural Networks via Information. 2017. 18
[65] Kenji Kawaguchi, Zhun Deng, Xu Ji, and Jiaoyang Huang. How does information bottleneck help deep learning? In International Conference Machine Learning (ICML), 2023. 18
[66] Qi Wang, Claire Boudreau, Qixing Luo, Pang-Ning Tan, and Jiayu Zhou. Deep Multi-view Information Bottleneck, pages 37-45. doi: 10. 11117//1977111111777777777 epubs.siane siam. wwwp:oigg//oooeeeeoooooooooeoeoe aatoii/abs//0// 10. 1137/1 97711117777111177777... 5. 18
URL https://
[67] Marco Federici, Anjan Dutta, Patrick Forré, Nate Kushman, and Zeynep Akata. Learning robust representations via multi-view information bottleneck. In International Conference on Learning Representations, 2020. URL https:///peereeeeeeeee
[68] Zhibin Wan, Changqing Zhang, Pengfei Zhu, and Qinghua Hu. Multi-view information- bottleneck representation learning. Proceedings of the AAAI Conference on Artificial Intel- ligence, 35(11):10085-10092, May 2021. doi: 10.1609/aaai, wtt.:/9/aaii.v3511.... v35111.17210. URL https:
//ojs. aaai. orgg/nndxx.. ponndexx.pp//ee hwt./aaticeeeviee////aaaaaaaaaaa 112100. 18
[69] Diederik P. Kingma and Max Welling. Auto-Encoding Variational Bayes. In 2nd International Conference on Learning Representations, ICLR 2014, Banff, AB, Canada, April 14-16, 2014, Conference Track Proceedings, 2014. 18
[70] Changhee Lee and Mihaela van der Schaar. A variational information bottleneck approach to multi-omics data integration. In Arindam Banerjee and Kenji Fukumizu, editors, Proceedings of The 24th International Conference on Artificial Intelligence and Statistics, volume 130 of Proceedings of Machine Learning Research, pages 5513-1521. PMLR, 13-15 Apr 2021. URL https:/ http:////ooceeennsseeeee mlr press/ crress/v130/1ees//
html. 18
[71] Paul Pu Liang. Zihao Deng, Martin Q Ma, James Y Zou, Louis-Philippe Morency, and Ruslan Salakhutdinov. Factorized contrastive learning: Going beyond multi-view redundancy. In A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine, editors, Advances in Neural Information Processing Systems, volume 36, pages 32971-32998, Curran Associates, Inc., 2023. URL /tttps://procceedingg neurips. cc/ wwappaarilllieee www.s/eaaeeeee/eeeeeeeeeaaeeeeee htpppaap/////// paper/2023/file. 6f 30t.:066f00.33000000007777706600 Pateeee0----aaee--eeeeee Conf wwwnnerrenneepe.. pdf. 19
[72] Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. In International Conference on Learning Representations, 2019. URL https:// wwwp:/wwwwwww......eonoeeea net/ferun Ld- 24
[73] Soujanya Poria, Erik Cambria, Rajiv Bajpai, and Amir Hussain. A review of affective computing: From unimodal analysis to multimodal fusion. Information Fusion, 37:98- 125, 2017. ISSN 1566-2535. doi: https//aao.or///00001f.f..nnffi http://wwwwaeeeeeeooooo/e/etaaaa
URI
25
16

---

[74] Geetha A. V., Mala T.. Priyanka D., and Uma E. Multimodal emotion recognition with deep learning: Advancements, challenges, and future directions. Information Fusion, 105:102218. 2024. ISSN 1566-2535. doi: http:///oooogg///000000ff..mtutt
102218. URI. https: 26
//ww.scieeneddrect http://www......a.......o.o.a//a
[75] Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. Ro(bert]a: A robustly optimized [bert] pretraining approach, 2020. URL https: /ppprr/eiee./eeeee/////m. wttt///rrum/dddSys50020
30
[76] Pengcheng He, Jianfeng Gao, and Weizhu Chen. DeBERTav3: Improving deBERTa using ELECTRA-style pre-training with gradient-disentaaglee embedding sharing. In The Eleventh International Conference on Learning Representations, 2023. URL https:://// pestps:/oopeee //penr/veeeeeeeer... tttt/forumiidssse 1d==E7dXhLxxitt 30
[77] Shicai Wei, Yang Luo, Yuji Wang, and Chunbo Luo. Robust multimodal learning via representa- tion decoupling. In Aleš Leonardis, Elisa Ricci, Stefan Roth, Olga Russakovsky, Torsten Sattler, and Güi Varol, editors, Computer Vision - ECCV 2024, pages 38-54, Cham, 2025. Springer Nature Switzerland. ISBN 978-3-031--2244611 30, 31
17

---

A Related Works
A.1 Multimodal Representation Learning
Multimodal representation learning aims at constructing multimodal system conduct multimodal understanding tasks including multimodal sentiment analysis [43, 46, 48-50] or emotion recognition [5I-54], multimodal recommendation [55] and multimodal segmentation [56] and so on. The common framework of multimoda learning can be divided into two main modules: modality-specific representation learning and multimodal fusion, according the purpose of reflecting heterogeneity and interconnections between modality elements [57].
On the one hand, effective extraction of task-relevant features within each individual modality forms the foundation of multimodal representation learning [47, 53]. Across various modalities, these modality-speciiffe encoders emphasize critical features and suppress unrelated noise, thereby shortening the path from raw data to semantically meaningful unimodal representations. On the other hand, multimodal fusion aims bridge modality gaps and exploit inter-modal complementarity [27, 57]. Early fusion schemes simply concatenate unimodal representations while late-fusion approaches combine unimodal predictions via weighted voting or gating. Hybrid fusion combine early and late fusion to conduct hierarchical and more delicated cross-modal interaction [58].
With the emerge of attention mechanism [59]. multimodal model with Transformer-base based architecture efficiently capture intra- and inter-modal dynamics by interleaving unimodal representation with self- and cross-modal attention layers [25, 60]. However, most of previous multimodal methods assume paired, fully observed modalities at both training and inference stages to achieve the state-of-the-an performance of multimodal understanding.
A.2 Missing Modality Issue
The missing modality issue occurs when pre-trained multimodal models suffer from severe perfor- mance degradation at downstream inference due to the absent input of modality data [4, 61]. In real world, the presence of multimodal input could not be guaranteed due to numerous reasons, such as sensor failure. hardware malfunctions, privacy limitations, environmental disruptions, and data transmission problems [1]. Empirical analysis have shown that the efficacy of multimodal methods critically depends on complete presence modalities [14], and that Transformer-basee encoders are especially sensitive to missing inputs [2]. Thus, addressing missing modality issue is essential for robust performance in real-world applications, which is named as incomplete multimodal learning.
Taking alternative perspective of incomplete multimodal learning, the missing modality issue can be regarded as missing specifie-view information issue in the multi-view learning [62]. Previous methods mainly focus on aligning complete and incomplete multimodal representations [9, 63] or generating the missing information with incomplete multimodal input [17, 22, 24J. While alignment- based models avoid the complexity of explicit generation, they often yield suboptimal inference with uncertain and heterogeneous information asymmetry. Diversely, generative based models remain susceptible to the task-unrelated noise and massive redundancy in the unimodal features, which can casily destabilize the information reconstruction or generation process.
Moreover, previous methods require adjusting according to specific missing scenarios to achieve optimal performance, restricting their applications in the uncertain cireumstance of the real world. Beside, most of them sacrifice performance of complete multimodal learning to increase the robustness to the missing modality issue. Therefore, achieving a single unified framework that jointly optimize both complete and incomplete multimodal learning remains open challenge.
A.3 Deep Learning with Information Bottleneck
Original introduced by Tishby et al. [29], Information Bottleneck (IB) formulates representation learning as an Eeormation-thhoreete approach that defines task-relevant representation as trade- off between feature compression and task prediction [31]. Due to the excellent interpretability
of representation learning and generalization estimation of the deep neural networks [30, 64, 65], IB has gained continuous interests in deep learning, especially in multi-view problem [66-68]. VIB [32] introduce variational approximation to generate the bottleneck latents in the information flow, sharing similar objective form with VAE [69] in generation. DeepIMV [70] utilize product-
18

---

of-experts to integrate the marginal specific-view representation into joint latent representation. FactorCL [71] factorizes task-relevant information into shared and unique information with multi- view redundancy defined by mutual information. Nevertheless, they can neither be employed to varying missing circumstance nor succeed achieving optimal intra- and inter-modal information extraction performance.
In this paper, we adopt the principles of IB construct effective informative latent space by designing information flow to features among modalities and guidance of semantics, paving the way for enhanced performance in both complete and incomplete multimodal learning.
B Derivations of the Variational Information Bottleneck
In this section, we derive the formula of the original variational information bottleneck and the proposed token- and label-level information bottleneck in detail, as shown in Equ. 5, Equ. 8 and Equ. 11-12. Besides, we provide the differentiable sampling process of the informative latents presented in Equ. 6.
Equ. represents the information bottleneck loss Lub for the information flow as (S - B + T), where S denotes the source state, B denotes the bottleneck latents and T denotes the target state. Recall that the loss is firstly given by the constrained mutual information, formulated as:
min (S,
min T(S; B) - T(B:T)
(24)
p(B[8)
where 3 > 0 is a trade-off parameters to balance the two mutual information terms. Directly obtaining the mutual information among S/B/T is unachievable. Therefore, we tend to obtain the upper bound of (S, T' to transfer the objective minimization problem to an evidence upper bound optimization problem.
Starting with the first term [(S; B). writing is out in full form
/ dB dS p(S, log B) f dB as p(S, B) log p(B)
p(B[S)
I):
p(S)p(B)
(25)
dB as p(S, log - / dB dS p(B) f
Considering that the computation of marginal distribution of the bottleneck latents p(B) = [ dSp( (BS))p( might be difficult, let q( () be variational approximation to this marginal distri- bution. With the non non-negative Kullback Leibler (KL) divergence K (L(((B) || (B) 2 0, we have dB p(B) log pBB) 2 J dB p(B) log (B), the following upper bound can be derived:
I)S / dBdS p (S, B) log ((BS) f
|aB p(B) log (BB)
(26)
dBdS (()) S) log Po (B|5)
(BB)
where pa( (B(S) can be learned by IB encoder Es - B.
Then for the second term (B; consider it in the same full form
I(B:T)= f
PJT,B) -f dT dB p(T, B) B(T, log - aT dB PTT, B) p(T)p(B)
P(T B) p(T)
(27)
Since B) is intractable, introducing IB decoder Dr B + T, let qo B) be variational approximation to B). Similarly, with KI. divergence KT(ppTT B) qs (TBB) 2 O, we
have dT p(T B) log (og 2 dT p(T) B) log 4o (T) B). Hence T(B; T) has lower bound as follows:
dT dB (T, B) log T (T
T(B:T) >
p(T)
dT dB PT, , B) log go (T B) -f P(T) logr(T)
(28)
{ (aT dB PT, B) log g (T (T + H(T)
19

---

where (T) is the entropy of target state T, determined only by the distribution of T' itself, no matter as the unimodal token embeddings or ground truth labels. Thus, (T') can be dropped in the loss function. When the source state S is diverse from the target state T, we introduce the relationship that independent of given S:
P(T, B(T,B) B) / ds p(S.T f (())(((()))(BI))
(29)
Note that when S = T, have BT, where above expression still stand. Besides, (S) denotes the known distribution for the joint and paired data samples. Utilizing data samples within each batch as the empirical data distribution, we have ((S.T) ((S))(TTS) Em 6s. (5) (T). Hence, [(B; T) can be generally denoted as:
I(B:T) / |ss dT dB (S)) go)
(30)
/ dB pl (BS)) log 1o (TIB)
2 Esspls)
Combining the bound of T(S; B) and [B; T), we can derive the following upper bound for Lib as Equ. 5:
B) (I(:ITT)
$ f dBds p ((5)((B[S) log (B(S))
(B) 33 Esss(s) / dB p(B)S) log go (T
(31)
s~p(()) [pT (B|S) |l q [B)] B BBD((B)s) Es Dog qs (T)B)
Finally, the optimization objeectee of variational information bottleneck can be rewritten min Ci min Es-((s) (()()(())) ((B))] - -3 max BB~ SSS=p(s)) (T]BB)) (32)
Now, we derive the practical expression of two minimization and maximization terms of the above upper bound.
First for minimizing K (B(sS) || (B)). in practice we utilize standard Gaussian Distribution ((0, 1) as the prior distribution of q( ()) and model the approximate posterior distribution pa (|S)) as a multivariate Gaussian distribution N M(H,e). Here μ and E denotes the mean and variance vectors of the latent Gaussian Distribution. Since each dimension of q (B) and po (B))) are independent, can derive the situation with one-dimensional Gaussian Distribution of K L(po (B ()) || q( (B)) as:
K (B)))(B)S) ll q(B) KL(N o (0.1))
/ N(Hn, 0z) log I(.
N(UB,OT)
+))
log
db
(b-jm)
| N()B,02) V2nog exp
20
db
b2
exp
2
2
/ OZ) (-log Vo
- (B) 2
12
+
db
20
2
/ (N(Hi [ogg0% + (b - pn) - 82
db
2
D
1 logoos / o()
1
((UB,oZ)) (b db / N(Hu, 22) db
db
o
(33)
Since any probability density function integrates to I over its domain, we have [ o db 1. According to the definition of the variance and second raw moment of Gaussian Distribution, we have N (uB, (b
2 db- of and f [N(,,, ob)6 62 db 2/ + oz. Thus, the analytical solution
20

---

KL (O) is computed as:
KL(N((B, ()) = log :: 1+
o - ( + o
2
(34)
1
(log o +
n
When each dimension d E {O...., C} is independent in multivariate Gaussian distribution ((,,)) 8). we have:
(L(po(BIS) (B)) / po(b) log (b(b1]5) po (ba)
db
g(b1) ... ((ba)
= f (o(()))
log pa (bdls)
Σ
log (bb) db
d=1
=
Σ
pa (b(s) [log (bbl) - log ((bu) db
(35)
Σ
po (bbbss) (logpobbls) log q(ba) dba
KL (bbals (I Σ
=
Then have:
min KL(N(UB, I1 IO) = min 1 Ec (log o + - -1 - -o
(36)
where C' is the feature dimension of the bottleneck latents B.
While for maximizing En~ BIS) Es- [log 9- (T] we leverage diverse expression to conduct computation based on the property of specific target state T, divided into token- and label-level IB in this paper.
B.1 Derivation of Token-level Information Bottleneck
For unimodal representation F, = {fa)lf E L target state T, we utilize IB decoder Dr + to enable information flow across the source and target representations Fs/ Fr with bottleneck bridge B. Since each dimension is independent for token embeddings of Fs/ Fr, have:
L C go (T = NNr,, oz) II
1.
(ff u )2 2(04)² )
II
N(44 (04)
II
cxp
2noy
A=1 d=1
(37)
Then we have:
C log 2T
1
1
( - +4)²
log go (TIB) =
log ( )2
(38)
2
2 d=1
2 d=1
(a4)2
For simplify, assuming that variance (a of each dimension d E (0. .., C) is consistent and fixed as a constant, IB decoder only needs to output the mean μr¹ = (bs) of the projected token embedding. Then the following equation holds:
max En qo ( B)] min
Σ
(ff
14)²
2
d=l
(39)
1.
min
Esg 1 - Dr(bs) 1|2
i=1
21

---

Combining Equ. 36 and Equ. 39, we can derive Equ. 8 as the objective of S - T token-level information bottleneck:
(Fs - ~ EPprup(Fs) (Bs [Bsss) 3 (ese- go (Fr(BBs))
E (KL(N(UE (ob)) V(0 D)) + 3 Ere III JT Dr(bs) [1
T.
L Σ
Be (log B + +1-11110 pg oBb)
+ 3 Ebsll fr Dr(bs) 11
(40)
B.2 Derivation of Label-level Information Bottleneck
For ground truth label Vgt E RR/RR as target state T, we utilize two forms of the posterior distribution BT) according the continuous (1 as the regression value) discrete (V the one-hot recognition classes) forms of the supervision. We leverage unimodal predictor Ps Bs + Ús for each source modality to model the posterior distribution B)) and output the prediction ýs based on information of single modality.
With continuous labels Yyt E IR' in regression task, consider 96 (T as one-dimensional Gaussian Distribution, computed as:
4 (T) B) = c- --11 10la--all Bell = = luue (Bs()]l
(41)
log 4o ( (T =I1 Vgt Ps(Bs)
Then we turn the maximization of posterior distribution go (T into:
max Bes as min Σ Eng Ugt Ps(Bs)
(42)
Combining Equ. 36 and Equ. 42, we can derive Equ. 11 the objective label-level information bottleneck:
1
1 Be (log oi 1/ oBB) + EB Ill Ygt Ps(Bs) II]
Lub
Σ N
(43)
2
With one-hot discrete labels yot E RV with V classes in classification task, we consider qs (T as multi-dimensional Bernoulli Distribution, computed as:
L (T|BB) = II D (Bs))
(44)
log go (T)B)
Σ Ugt log Ps(Bs)
Then turn the maximization of posterior distribution go (B) into:
max ()) [log go (T] = min EBs Σ yot log (Bs)
(45)
Combining Equ. 36 and Equ. 45, we can derive Equ. 12 as the objective label-level information bottleneck:
N
Lib R
Ee (log oz + 11---00 2/z - 03)) + E
Σ Met log (s(Bs))
(46)
22

---

B.3
Reparameterization Trick for Informative Bottleneck Latent
Since the informative bottleneck latents sampled from pe (B()) (H, o²) where the sampling
process is not differentiable. For the purpose of optimizing the IB encoder Es S + B through gradient decent algorithm, we need to identify the gradient of B the weight parameters 0 of the encoder, denoted as follows according to the Chain Role:
dB OB 0u
²B do²
+
(47)
00
0p 00
do 00
where j and a are directly output by the encoder so that 8u and Oo² easily tractable. ae
ae
To compute aB and 0B from the sampled bottleneck latent B, we firstly sample a random vector z 0y Oo from nonparametric distribution (0, I) and utilize transformation function gz. 11, o²) to attain the bottleneck B, shown as:
B = p
+ Z, where (0,
(48)
Since the process of sampling z N/, I) is independent to parameters μ and the derivation of and d are transformed into the derivation of dg(-] and 0g(+) By
Dj dg which is differentiable.
C Theoretical Grounding of combining IB with Cyclic Translation
We present further explanation for the theoretical analysis of CyIN in combining Information Bot- tleneck (IB) with cross modal cyclic translation. Except for task prediction term, the optimization objective for IB loss and cyclic translation in Equ. 23 is deduced as follows:
C Leas + Lia + +ftran
(49)
LIB T[Ere Leye)
Considering multimodal learning with two modalities without loss of generality, the theoretical modeling of CyIN can be formulated as follows:
S B = B2 -- S2
4
Ti T2
where S denotes the source state and T' denotes the target state.
The left and right side with S - B1 + Ti and S₂ - B2 + Tr denote the chain of information bottleneck. Note that the source and target states have S/T E (Fs, Fs,) (s,)(F (Fr, Fr,} in the token- level IB or S E (Fs, Fs,). }, T'' in the label-level IB, both of which have been theoretically deduced above, formulated as:
Lub [T(1 B1) - BI(BI: (T(S; B₂) - (I(
(50)
1B for modality
1B for modality
The middle part B₁ = B2 denote the cyclic translation with B1 By B ~1 and B2 BT Beye B1, where - denotes the translation process while ~ denotes alignment with the original bottleneck. Considering be divided into cross-modal reconstruction across diverse unimodal latents B₁ B2 and cyclic translation from the reconstructed latent BT Bges, T1 / BB = T 2 (BTSs)) //23//////1/111 (B formulated as: (BI BrSs)
the translator network, the translation process can
back to the origin
Lree
DKr(Be; BT
Reconstruction from modality
Reconstruction from modality
(51)
Loye
DKL (xr (B
(B;
Translating Back from modality 2 Translating Back from modality 1-
23

---

Minimizing above losses, we can obtain informative unimodal bottleneck latents B, for each modality u, which contains inter-modal features to conduct productive multimodal fusion. Comparing with directly fusing unimodal representations Fu. fusion process implemented on the bottleneck latents B.. denoted as Fи = Das B1, B2), can benefit from the compression ability of information bottleneck and lead to less reconstruction difficulty in cross-modal translation.
When both modalities 2 and U2 are presented, known as complete multimodal learning, we can regard the cyclic translation B = B2 as sort of cross-modal interaction enhancing the extraction of modality-sharee dynamies. Such cross-modal synergies can benefit the multimodal fusion process FM Dar B2) to attain the final discriminative representation.
When one of the modalities is missing, the framework is required to conduct incomplete multimodal learning. Assuming is missing, we can obtain bottleneck latents B2 from modality u2 to recon- struct the bottleneck latents Biee, denoted Bg Brec, which can be further decoded as the supplementary information for the multimodal fusion process FM Dxr (B7ee, B2).
The aforementioned two-modality scenarios can be generalized to arbitrary modality pairs without constraint, thereby facilitating efficient scaling CyIN for tasks involving multiple modalities.
D Hyper-parameter Setting
Since missing modalities lead to highly performance fluctuations under different missing scenarios, guarantee fair and consistent comparison among various methods, we conduct evaluation of each experiment ten times using fixed 10 random seeds. We utilize AdamW [72] as the training optimizer.
The split of datasets and
of CyIN are reported in Table 6.
Table 6: Splits of datasets and the corrsponding hyper-parameters settings.
Multimodal Task
Regression
Classification
Dataset
MOSI
MOSEI
IEMOCAP
MELD
Train
1,284
16,326
5,228
9,765
Valid
229
1,871
519
1,102
Test
686
4,659
1,622
2,524
Training Epochs
50
30
50
50
Batch Size
128
128
256
128
Learning Rate of Language Model
4e-5
le-5
5e-6
3e-5
Learning Rate of Other Parameters
le-3
Se-4
lc-4
5e-4
Weight Decay
le-3
le-4
Dimension Cy Unimodal Representation
32
Dimension Cy in IB Encoder or Decoder
256
64
256
64
Dimension C'R of Bottleneck Latent B
128
256
128
32
# Layers of R.A(-) in CRA
16
8
4
Dimension of I-A(-) CRA
[64, 32, 16]
[128, 64, 32]
[128, 64, 32]
[128, 64, 32]
# Layers Cross-modal Attention Heads of Attention H 3
4
8
16
32
4
10
10
5
1 2dd Training Stage
3:7
1:4
E Details about Evaluation Protocols
The evaluation under the cireumstance of (1) complete multimodal input is consistent with other state-of-thh--a baselines, referring to u E (I, a, v), where I, a, and v denote language, acoustic, and visual modalities, respectively.
Following [8, 17, 22, 24], to evaluate model robustness under missing modality scenarios, we adopt two missing data protocols: (2) fixed and (3) random missing protocols.

---

(2) In the fixed missing protocol, a consistent subset of modalities is removed across all iterations. Specifically, we discard either one modality (e.g., u E (l, v)/ v})) or two modalities (e.g..
E (1/(a))(w))
(3) In contrast, the random missing protocol simulate the real-world scenarios by randomly selecting missing modality combinations for each sample in every batch, where either one or two modalities may be absent. Following [22, 24], the random missing rate (M It) is defined as:
Σ 11
MR
(52)
X
where [u.] denotes the number of available modalities for the i-th sample, U is the total number of modalities, and N is the total number of samples. Given realistic application scenario, each sample is guaranteed to have at least one available modality, ensuring |Ms] E f1. U} and consequently, MR < 7ي In our multimodal regression and classification tasks with U = 3 modalities, we consider missing rates MR E {0.0, O........... 0.73, tolerable missingness. Notably, MR = 0.0 represents the complete multimodal scenario where all modalities (u a, v) are present.
where MR 0.7 approximates the maximum
Baselines
Baselines are reproduced by the open-source codes. Following [27]. we conduct fifty-times of random grid search for the best hyper- prrpeepprraeters of each model. The descriptions of baselines are presented as follows.
CCA [11] projects paired modalities into a shared low-dimensional space by maximizing their canonical correlations, providing classical linear imputation for incomplete data.
DCCA [12] replaces the linear projections with deep neural encoders, enabling nonlinear correspon- dences between modalities.
DCCAE [13] augments DCCA with autoencoder- reconstruction objectives [15], jointly pre- serving each modality's internal structure while learning maximally canonical correlations.
CRA [34] employs cascade of residual autoencoders that iteratively refine the reconstruction of representations with complete inputs from the ones with partial inputs.
MCTN [16] leverages encoder-decoder recurrent neural networks to translate between source and target modalities in cyclic translation way.
MMIN [17] extends cycle consistency learning with a cascaded residual architecture by imagining missing information from the observed views.
CPM-Net [63] utilize partial multi-view clustering to tackle incomplete multimodal learning issue, embedding all views into structured latent space where missing features can be inferred via data transmission and cluster-aware classification.
GCNet [22] introduces two complementary graph neural networks to model temporal and speaker relationships in conversational data, and then reconstructs missing features from the learned graph representations.
IMDer [24] adopts score-based diffusion models learn distribution of missing modalities in iterative diffusion and denoising process, regularized by the remained modalities.
LNLN [8] employs dominant modality correction module ensure the quality of dominant modality representations and conduct dominant modality based multimodal learning to resist input noise.
G Evaluation Metrics
The formulas and computation for evaluation metrics on multimodal regression and classification tasks are presented as follows.
For multimodal regression tasks on MOSI and MOSEI, we adopt well-representative metries including Acc7, F1, MAE, and Corr, to evaluate models' performance [73]:
25

---

Acc7 (Seven-class Accuracy) measures the proportion of correct predictions across the seven integer labels in 1-3, +3). formulated as:
1
- Σ i Yn = In = k}
Acc7
Σ = N
(53)
{ [m: k}
where is the total number of samples, In is the predicted label, Yn is the ground-trute truth label, L-1 de- notes the round nearest integer and T(-) is the indicator function. With (-3, -2. -1, 0, 1, 2,2,3, {n yn = k} denotes the set of samples with true label k; and {n: yn = k A In = k} is the subset correctly predicted as k.
F1 (Binary Fl-score) assesses positive-negatio discrimination by collapsing regression labels into two classes (negative/possitive) formulated as:
We 2. (Precision, * Recalle) Σ
N
F
(54)
(Precisione + Recalle)
TP
TP
Precision
TP + FP Recall TP 1 FN
(55)
where C' is the number of classes, We the weight of class (typically w. Ne/N), and T.. F.. and F FN denote the numbers of true positives, false positives, and false negatives for class c, respectively.
MAE (Mean Absolute Error) quantifies the average absolute deviation between predicted and ground-truth scores, formulated as:
DW
MAE(Ú.D 9)
(56)
Corr (Pearson Correlation) captures the linear agreement between predictions and labels, indicating any systematic bias or skew in model outputs, formulated as:
Σ (in 0) (Un 7)
Cor( v)
(57)
E (in E (yn D)
where 5 and I are the means of the predicted scores and ground truth labels.
For multimodal classification tasks on IEMOCAP and MELD, we report Ace and wF1 to balance the weight of scores from each class [74].
Acc (Binary Accuracy) describes the unweighted proportion of correctly predicted samples out of the total number of samples, formulated as:
TP + TN
Acc TP+TN + FP+FN
(58)
wF1 (Weighted Fi-score) balances weighted precision and recall based on the true instances for each class, and the formulation of wF1 in classification is the same Equ. 54.
H More Experiment Results
H.1 Detailed Results Under Various Missing Scenarios
The detail experiment results under various missing modality scenarios with fixed and random missing protocols are presented in Table 7 and Table 8, respectively. Note that the reported results are the average values under 10 fixed random seeds.
With fixed missing protocol including incomplete settings u E (0//6o/(00//Laa)//....) CylN demonstrates superior or competitive performance across nearly all settings, significantly outperforming other models in both unimodal and bimodal missing scenarios. From Table 7, we can observe that language modality plays the most essential role in these multimodal datascts due
26

---

Table 7: Performance comparison between the proposed CyIN and baselines with fixed missing protocols including incomplete settings (atloo///aatLLaa//// //fa, v).
Fix
Models
Acc71
Fl
MAEL
Carr
Acc71
F1
MAEL
Corr
Ace!
WFI
Acc1
wFI1
CCA
26.4
74.7
1,098
0.567
44.6
0.677
0.621
49.7
43.1
DCCA
23.3
73.5
1.196
0.520
41.5
68.0
0.810
0.377
51.7
50.7
48.9
35.3
DCCAE
1.278
CPM-Net
17.2
63.9
1.335
0.345
40.3
74.7
0.832
0.230
50.4
50.5
42.0
34.8
CRA
36.3
82.8
0.900
0.749
49.3
84.9
0.576
0.741
53.8
51.3
55.3
51.6
(
MCTN
0.777
49.6
0.594
0.715
58.4
56.1
52.3
MMIN
42.3
85.0
0.743
0,791
52.8
84.6
0.543
0.761
$7.0
$7.5
59.2
54.9
GCNet
84,8
IMDer
43.0
85.1
0.717
0.794
31.8
85.0
0.552
0.756
60.4
60.1
59
58.2
LNLN
43.9
83.5
0.761
0.760
52.5
84.3
0.547
0.765
62.8
62.4
58.3
$7.1
CylN
0.742
0.751
CCA
17.5
46.8
1,422
0.112
41.5
63.3
0.808
0.272
39.1
34,8
47.7
35.7
DCCA
DOCAE
19.7
56.5
1.406
0.123
37
50.8
0.841
0.301
35.6
34.0
47.3
34.4
CPM-Net
17.3
56.6
1,371
0.313
39.4
74.2
1.9
38.1
37.2
26.3
29.7
CRA
19.7
$2.0
1,438
D.102
41.8
71.1
0.781
0.420
40.7
40.1
46.6
37.5
[0]
MCTN
15.5
42.2
1.431
0.018
41.4
62.9
0.842
0.096
22.3
11.3
48.2
31
GCNet
19.8
57.5
1.383
0.387
40.8
56.6
0.817
0.389
44.6
42.2
40.9
34.0
IMDer
19.6
58.3
1,363
0.392
42.1
0.795
0.372
45.6
42.5
LNLN
15.6
47.7
1.442
0.075
41.0
68.9
0.795
0.338
39.2
33.4
39.3
33.7
CyιN
20.7
6A.2
1,385
0.412
42.3
70.8
0,782
0.363
50.3
48.9
48.5
37.7
CCA
15.5
26.1
1.445
0.125
41.4
48.6
0.837
0.239
48.4
31.6
DCCA
17.4
51.7
1.437
D
41.3
72.9
0,800
0.367
26.6
22.4
45.8
32.9
DCCAE
CPM-Net
16.5
56.8
1,368
0.323
38.8
74,2
0.862
0.205
28.0
20.9
31.8
32.1
CRA
16.6
52.1
1,396
0.013
39.0
72.0
0.781
0.426
29.9
26.3
44.2
36.8
(+)
MCTN
42.2
1,431
0.018
41.4
62.9
0.842
0.096
23.5
48.4
31.6
MMIN
19.7
50.5
1.471
0.075
39.5
69.8
0.789
0.395
35.3
34.1
48.4
31.6
GCNet
1,390
IMDer
17.0
52.0
1.426
0.032
41
62.7
0.826
0.386
46.8
45.8
46.5
38.6
LNLN
15.5
42.2
1.447
0.125
41.8
70.9
0,771
0.405
47.9
47.3
48.1
31.3
CyIN
1.362
71.7
48.2
CCA
27.8
74.9
1,106
0.542
44.5
81.8
0.663
0.639
60.1
58.5
$1.1
46.5
1.377
DCCAE
22.2
69.4
1.425
0.414
38.7
68.2
0.802
0.394
$4.0
52.7
47.3
34.5
CPM-Net
17.2
64.0
1,335
0.345
39.9
74.5
.80
145
44.0
35.1
35.8
CRA
83.4
0.885
0.750
50.5
84.6
0.565
0.756
60.7
60.0
56.4
53.3
(.
MCTN
41.5
83.7
0.777
0.765
47.9
84.3
0.592
0.721
$8.4
57.8
56.3
52.4
GCNet
41.5
84.8
0.742
0.796
49.
84.
0.581
0.753
62.9
63.0
51.4
55.8
IMDer
44.3
0.797
$2.8
85.1
0.560
0.775
60.0
58.3
LNLN
44.0
83.3
0.761
0.766
52.7
84.6
0.546
0.768
$7.9
$7.3
58.1
57.1
CylN
45.0
88.1
0.728
0.792
$3.1
85.5
0.541
0.786
68.2
64.7
60.5
58.3
CCA
26.1
75.6
1.093
0.570
44.7
80.7
0.674
0.625
54.5
49.7
43.1
DCCA
24.1
71.2
1.234
0.398
41.2
73.3
0.791
0.420
50.0
49.9
47.2
35.3
DCCAE
1.327
CPM-Net
17.5
62.9
1.3.99
0.333
38.1
73.4
1,582
0.072
42.8
43.5
41.7
35.2
CRA
36.3
81.5
0.917
0.741
51.3
85.1
0.355
0.761
S8.3
55.7
56.4
53.3
(.e)
MCTN
41.5
0.777
47.99
84.3
0.592
0.721
58.3
58.2
52.4
MMIN
42.3
85.0
0.742
0.791
52.8
85.1
0.534
0.768
$9.3
$9.4
60.1
55.5
GCNet
IMDer
45.8
85.
0.701
0.799
$2.9
85.2
0.556
0.761
60.2
60.0
59.8
58.0
LNLN
43.7
83,5
0.760
0.766
52.7
85.0
0.544
0.770
62.1
61.5
58.2
57.1
CyIN
0.727
0.778
CCA
17.4
48.1
1,420
0.117
41.6
63.6
0.806
0.278
39.6
35.1
48.0
36.1
DOCAE
19.4
55.2
L,569
0.046
37.9
72.7
0.796
0.404
35.5
33.2
46.5
35.7
CPM-Net
16.8
56.2
1,367
0.330
35.2
1.395
0.041
44.5
42.4
25.6
28.9
CRA
18.5
55.0
1,410
0.066
40.4
73.2
0.773
0.453
44.1
42.0
45.3
40.1
(6,0)
MCTN
15.5
42.2
1.431
0.118
41.4
62.9
0.842
102
22.7
11.5
48.8
31
GCNet
17.9
57.2
1,363
0.385
41.2
72.2
0.805
0.393
49.0
48.1
44.7
40.9
IMDer
18.4
1.319
0.386
41.5
0.788
0.446
43.0
LNLN
15.6
48.7
1.441
0.076
40.8
71.7
0.775
0.441
51.5
50.4
35.5
31.8
CyIN
21.0
$9.4
1,305
0.391
41.7
72.9
0.773
0.458
$3.0
$2.1
48.4
43.5
27

---

Table 8: Performance comparison between the proposed CyIN and baselines with random missing protocols including missing rates MR E (0.1, 0.3, 0.4, 0.5, 0.6, 0.7].
MR
Models
MOSI
MOSEI
JEMOCAP
MELD
Acc71
Fl
MAE
Corrt
Acc71
F11
MAE
Corrt
Acct
WFIT
AccT
wFI1
CCA
26.9
72.7
1.133
0.512
45.6
81.2
0.671
0.635
58.9
58.7
49.3
41.9
DCCA
70.1
1.219
40.1
73.1
0.789
0.415
52.4
47.5
35.3
DOCAF
20.2
68.4
1.620
0.336
38.7
72.5
0.787
0.418
$1.0
50.0
46.4
35.5
CPM-Nei
16.8
63.9
1.336
0.339
34.5
74.3
1,780
0.060
55.1
55,1
42.5
33.2
CRA
0.1
MCTN
38.9
79.9
0.846
0.720
47.1
81.4
0.620
0.680
42.0
43,3
51.6
46.8
MMIN
0.808
0.741
83.5
0.562
0.738
55.8
49.0
GCNet
41.2
84.5
0.806
0.751
50.3
83.1
0.653
0.696
60.7
60.8
56.3
54.4
IMDer
42.3
0.796
0.753
51.4
83,9
0.600
0.721
60.9
61.3
58.4
56.7
INLN
CylN
42,5
84.6
0.783
0.759
51.9
844.5
0.562
0.748
64.0
63.9
58.6
56.6
DCCA
22.8
68.7
1.242
0.394
40.4
72.3
0.792
0.400
49.4
48.5
47.2
34.6
DOCAE
20.9
67.6
1,584
0.325
38.9
70.8
0.795
0.389
48.6
47.7
46.7
35.2
CPM-Net
16.9
63.9
1,336
0.338
34.3
74,2
1.457
0.064
53.5
54.2
41.9
33.1
CRA
31.9
76.2
1,025
0 666
48.9
82.7
0.598
0.691
56.6
55.7
$4.2
50.9
MCTN
MMIN
38.3
78.6
0.872
0.701
50.2
82.0
0.597
0.693
$2.6
$2.7
$5.2
48.1
GCNet
39.0
81.0
0.832
0.725
48.1
81.9
0.622
0.682
59.2
59.3
$3.2
$1.8
IMDer
39.3
82.8
0.878
0.729
49.7
82.2
0.610
0.694
59.9
60.3
55.9
53.9
LNLN
39.9
79.6
0.837
0.717
50.1
82.3
0.591
0.713
60.5
61.1
54.2
52.9
CylN
CCA
24.2
68.6
1.193
0.454
44.6
77.1
0.707
0.569
53.1
$2.9
48.9
40.6
DCCA
1,251
40.8
0.390
34.2
DOCAE
20.4
66.2
1.545
0.310
39.1
69.2
0.801
0.371
46.3
45.3
46.9
34.8
CPM-Net
16.9
64.0
1,336
0.336
34.1
74.0
1,180
0.111
53.8
40.4
34.6
CRA
72.3
1,077
48.0
0.661
0.3
MCTN
33.5
72.3
0.977
0.634
46.1
76.1
0.668
0.599
37.7
40.0
51.0
45.4
MMIN
74.6
48.7
0.631
0.659
$1.2
$4.8
GCNer
36.3
75.4
0.968
0.652
48.5
80.7
0.673
0.665
$7.5
$7.6
50.9
49.7
IMDer
37.7
76.6
0.923
0.677
48.6
0.643
0.660
$4.2
$1.8
INNN
76.5
50.9
CylN
38.1
78.4
0.886
0.694
49.2
81.5
0.606
0.685
60.3
60.4
55.8
52.7
39.9
DCCA
21.2
64.6
1,233
0.304
41.1
69.9
0.796
0.578
42.4
41.2
47.1
33.7
DOCAE
19.2
62.9
1,574
0.273
39.3
0.808
0.347
43.4
42,2
46.9
34.1
CPM-Net
16.9
63.8
1,336
0.335
34.2
73.9
1,439
0.061
$2.8
$3.6
41.9
33.2
CRA
28.3
68.2
I,160
0.552
46.7
79.9
0.646
0.626
50.0
49.5
$2.1
47.9
MCTN
35.4
MMIN
33.3
71.2
L.01
0.523
47.4
79.4
0.655
0.629
49.9
50.0
$4.3
46.3
GCNet
33,9
74.6
0.986
0.628
47.1
79.2
0.672
0.623
56.1
56.3
47.7
47.0
IMDer
34.1
75.9
0.944
0.655
47.2
78.7
0.664
0.618
56.8
$7.2
$2.0
49.2
LNLN
34.5
73.3
0.952
0.644
47.2
79.2
0.640
0.646
58.7
59.3
$1.5
49.3
CylN
CCA
21.7
63.8
1,255
0.384
43.6
73.0
0.744
0.489
46.9
46.2
48.6
39,
DCCA
1,289
67.4
33.3
DOCAE
19.5
60.9
1,558
0.246
39.3
63.4
0.818
0.317
40.1
38.7
47.4
34.0
CPM-Net
16.9
63.9
1.335
0.334
34.2
74.0
1.626
0.056
$2.7
53.3
41.1
33,1
CRA
1,209
45.4
0.671
46.
50.5
0.5
MCTN
25.7
59.7
1.161
0.503
44.6
70.7
0.717
0.510
33.3
35.7
52.0
43.3
MMIN
0.605
$3.4
45.0
GCNer
30.7
71.0
1.038
0.598
45.0
78.0
0.699
0.618
$4.3
$4.4
45.6
45.5
IMDer
31.8
72.2
0.995
0.605
46.2
76.4
0.687
0.570
54.8
55,3
50.0
46.8
1.024
77.8
46.7
CyiN
32,5
72.6
0.990
0.599
47.1
78.7
0.658
0.611
$5.3
$5.5
53.6
48.2
DCCA
20.6
60.3
1,324
0.252
41.3
64..4
0.803
0.355
37.3
36.3
47.3
32.8
DOCAE
19.0
57.4
1.545
0.231
39.1
59.2
0.827
0.293
37.5
35,2
47.7
33.4
CPM-Net
17.0
64.0
1.341
0.299
34,3
73.9
1.304
0.093
$2.6
53.2
41.5
33.7
CRA
25.1
61.4
1,264
0-454
43.9
77.2
0.696
0.577
42.7
42.6
49.6
44.1
MCTN
MMIN
28.3
62.9
1.158
0.486
44.8
76.4
0.688
0.577
46.3
46.2
$2.1
43.3
GCNet
29.3
65.9
1.127
0.513
43.8
77.2
0.705
0.584
$1.8
51.8
41.2
43.9
IMDer
30,2
67.8
1,090
0.564
44.6
75.3
0.704
0.544
52.2
$1.7
47.6
44.8
LNLN
28.9
64.9
1.150
0.522
45.6
76.1
0.689
0.565
48.1
47.7
49.3
45.0
CyIN
CCA
19.9
$9.7
1.298
0.326
42.7
68.3
0.775
0.409
41
39.8
48.5
37.7
DCCA
32.5
DOCAE
18.6
55.5
1,539
0.209
38.8
55.9
0.833
0.277
35.7
32.7
47.8
33.2
CPM-Net
17.1
63.8
1,361
0.291
34.2
1.692
0.104
52.2
52.5
41.6
34.1
CRA
1,311
43.0
76.5
40.1
48.7
42.7
0.7
MCTN
23.5
56.1
1.223
0.432
43.7
64.9
0.760
0.413
28.8
30.8
51.2
41.2
MMIN
25.6
1.212
0.545
GCNer
26.3
63.9
1.166
0.494
43.3
74.5
0.733
0.542
47.3
47.2
39.9
41.8
IMDer
27.0
65.1
1.126
0.526
43.1
75.1
0.714
0.470
48.2
46.3
43,0
62.7
1,200
0.472
45.0
74,8
0.705
45.9
48.5
43,1
CyIN
28.0
65.9
1.117
0.530
45.1
74.9
0.700
0.553
48.6
49.0
S1.8
44.4
28

---

to the rich semanties and high recognition-red related information in utterance [16, 43, 44]. Despite better performance in this dominant modality, CyIN pays more attention on the inferior modalities including audio and vision modalities. This result illustrates the generalization ability of the proposed framework, since CyIN has not spectacularly designed extra delicate modules to concentrate training on dominant modalities as LNLN [8]. Besides, with one modality missing and two modalities input, CyIN sufficiently integrate the information from bimodal interaction and reach higher performance, validating the flexibility of informative bottleneck latents in various combination of modalities.
With random missing protocol including missing rates M [0.1, 0.3, 0.4, 0.5, 0.6,0.,,,,. 0.7]. CyIN suffers from less performance degradation as the missing rate increases. The random missing case mostly simulating diverse missing situation in the real-world, when the presence of modalities is not fixed or known. As shown in Table 8, compared with baselines, the robustness of CyIN under various input data sparsity highlights the strength of informative bottlenecks extracted by token- and label-level IB, ensuring stable performance even when the presence of multimodal input are severely missing at M 0.7. Moreover, CyIN reaches sfate-of-the-ar performance on 4 datasets regardless of multimodal regression classification tasks. While most baselines reach suboptimal performance on either one of the tasks due to the limitation of generalization.
The experiment results in various missing scenarios clearly demonstrates that CylN is highly effective in both fixed or random missing scenarios. The general framework enables the informative space to maintain powerful performance despite the presence of input modalities, making it a robust and practical solution for real-world multimodal applications.
H.2 Comparison of Performance Stability
Table 9: Performance stability comparison between the proposed CyIN and baselines with the average results on MOSI and IEMOCAP dataset with fixed missing protocols including modality settings E (1/(a, v) and random missing protocols including missing rates MR 0.7.
MOSI
IEMOCAP
Setting
Models
Acc7t
F1
MAE
Corrt
Acct
wFIt
MCTN
41.5 0.71
83.7 0.37
0.777 + 0.005
0.765 0.003
58.4 0.40
57.8
0.41
MMIN
42.3 0.82
0.45
0.743 + 0.009
0.791 0.005
57.0
57.5 1 0.49
Fixed
GCNet
0.53
0.3
0.740 : 0.003
0.795 0.002
58.7 0
58.3
C (1)
IMDer
43.0
8
0.717 + 0.005
0.794 0.002
60.4+0.4
60.1
ININ
43.9 0.33
83.5 + 0.24
0.761 + 0.008
0.760 + 0.006
62.8 0.51
+2.4+00.88
CyIN
44.0
0.25
85.2 + 0.29
0.742 + 0.004
0.795 + 0.002
63.4 =.10
63.0
MCTN
0.75
42,2 0.37
1.431 + 0.005
0.018 + 0.004
22.7 00.00
11.5 0.24
MMIN
2.95
52.3 1.32
1.427 + 0.009
0.081 + 0.005
48.2 0.51
47.4
Fixed
GCNet
0.83
57.2 1.25
1,363 + 0.008
0.385 0.002
09.4
u E (a,0)
IMDer
0.85
58.1 1.58
1.319 0.016
0.386 0.026
50.0 0.30
49,9 0.38
LNLN
15.6 + 0.75
48.7 0.45
1.441 + 0.008
0.076 0.006
51.5 + 0.56
$0.4 + 0.48
CyIN
21.0 0.35
59.4
1,305 + 0.010
0,391 = 0.019
53,0 = 0.40
52.1 = 0.46
MCTN
23.5 1.15
56.1 1.23
1.223 + 0.018
0.432 0.026
28.8 +
30.8 + 0.69
MMIN
25.6 1.04
59.0 1.60
1.212 0.017
0.440 + 0.019
4..9
45.8 = 0.65
Random Missing
GCNet
26.3
6.89
1.166 + 0.038
0.494 + 0.032
47.3 0.79
47.2 + 0.81
IMDer
27.0 1.30
1.50
1.126 + 0.022
0.526 + 0.023
47.7 1.20
48.2 1.12
MR= 0.7
LNLN
26.1 1.07
62.7
1,200 0.019
0.472 0.021
46.9 1.66
CylN
28.0 0.90
1.117 # 0.018
0.530 0.021
48.6 0.71
49.0 0.70
We evaluate the performance stability of the models on MOSI dataset under three representative circumstances, including fixed missing protocols u E (1) (a, f} and random missing protocols MR = 0.7 Based on the overall average performance reported in Table 7 and Table 8, we further computed the standard deviations of the corresponding results on each missing scenarios with 10 runs, to demonstrate the performance variation degree in real-world circumstances. The final stability measure are reported in Table 9.
Under fixed missing modality protocols such as u E (0/fa, v}, the standard deviations of all methods show relatively small, reflecting that when the missing modalities are predeternined, models can learn to compensate in a more determine way. In contrast, random missing protocol introduce much higher variance, especially in severe missing scenario with MR = 0.7. We summarize this into two reasons: First reason is imbalance contribution of modalities (with dominant modality like
29

---

language containing more semantics compared with the inferior ones like audio or vision in specific tasks), and the second one is random missing setting more closed simulate the unpredictable inference circumstances in real-world, exposing each model's 's real robustness.
Across both fixed and random missing scenarios, CylN not only achieves the superior or compet- itive performance on most metrics but also maintains relatively low fluctuations, highlighting the remarkable balance between performance and stability for the constructed informative space.
H.3 Generalization Performance on Different Language Models
To further validate the generalization performance, We train and evaluate CyIN with different sizes of Pretrained language models including BERT [40]. RoBERTa [75], and DeBERTa-V. V3 [76] as shown in 10. The experiment results shows a clear benefit from scaling up the PLM backbone, as larger models like DeBERTa-Y V3 yield better textual representations and thus higher multimodal performance when all modalities are available. The similar trend occurs when evaluating with the most severe missing circumstance M R 0.7, which indicates the effective generalization ability of CyIN.
Table 10: Comparison of the proposed CyIN using different sizes of language models on MOSI dataset, under both complete and randomly incomplete multimodal learning settings.
Model Variants
MOSI
Setting
(PLM)
Acc71
Fif
MAE
Corrt
BERT
48.0
86.3
0.712
0.801
Complete u E (l,a,v}
RoBERTa
49.5
88.2
0.692
0.823
DeBERTa-V3
50.3
90.1
0.671
0.841
BERT
28.0
65.9
1,117
0.530
Random Missing MR = 0.7
RoBERTa
29.3
67.2
0.992
0.556
DeBERTa-V3
32.2
69.5
0.965
0.578
H.4 Various Scales of Other Multimodal Tasks
We have extended experiments of CylN on 6 non-affective datasets with 2-4 modalities and random missing rates in Table 11 14, including the following tasks: Multimodal Recommendation (3 modalities: visual, audio, textual) [55] (Amazon Baby, Tkitok, Allrecipes datasets), Multimodal Face Anti-spoofing (3 modalities:
RGB, Depth, IR) [77] (CASIA-SURF dataset), Multimodal Dense Prediction (2 modalities: RGB, Depth) [77] (NYUv2 dataset), and Multimodal Medical
Segmentation (4 modalities: FLAIR, TI, T1c, T2) [56] (BraTS2023 dataset).
Table 11: Experiment comparison for Multimodal Recommendation task of accuracy and fairness performance (%) three datasets, including Amazon Baby, Tiktok and Allrecipes dataset. The modality setting is set in random missing protocol with 0.4 missing rate as the SOTA (55) paper.
Recall f
Precision K=10
NDCG 1
Ft
Fium
Dataset
Model
K=10
K=20
K=20
K=10
K=20
K=10
K=20
K=10
K=20
SOTA [55]
5.26
8.54
0.56
0.45
2.76
3.60
87.24
90.12
1.11
0.90
Amazon Baby
CyIN(ours)
5.43
8.56
0.57
0.45
2.89
3.68
90.10
92.70
1.14
Tiktok
SOTA [55]
7.39
0.48
0.37
2.95
3.60
88.15
92.27
0.96
CyIN(ours)
4.91
7.57
0.49
0.38
3.15
3.81
89.02
92.31
0.98
0.75
SOTA [55]
2.49
3,36
0.24
0.16
1.33
1.55
96.82
89.52
0.49
0.33
Allrecipes
CyIN(ours)
2.61
3.43
0.26
0.17
1.45
1.66
99.57
91.94
0.52
0.34
The experimental results illustrate that the proposed CyIN reaches comparable performance with the of-te-of-the-ae art (SOTA) methods, which further demonstrates the effectiveness of CylN in generalizing to diverse numbers of modalities or multimodal fusion tasks. Moreover, the scale of datasets varies from 1,449 samples (NYU v2) to 65,671 samples (Tiktok), 87k samples (CASIA-SURF), 139, 10 samples (Amazon Baby), which partially show the scaling performance the proposed CylN. Due to the computation resource limitation, we leave exploration larger scale datasets in the future work.
30

---

Table 12: Experiment comparison for Multimodal Face Anti-spoofing task with Average Classifi- cation Error Rate (ACER) (4) metric on CASIA-SURF dataset. The modalities setting is set in fix missing protocol as the SOTA [77] paper.
Modality Setting Depth
SOTA [77] CyIN(ours)
RGB
7.33
4.48
2.13
2,83
10.41
7.75
1.02
2.26
3.88
3.06
1.38
1.20
0.69
0.66
Average
3.84
3.18
Table 13: Experiment comparison for Mul- timodal Dense Prediction task with mloU (1) metric on NYUv2 dataset. The modalities setting set fix missing protocol as the SOTA [77] paper.
Table 14: Experiment comparison for Multi- modal Medical Segmentation task with Dice similarity coefficient (DSC) % (1) metric BraTS2023 dataset. The average results are reported conducted in fixed modality setting as the original SOTA [56] paper.
Modality Setting
SOTA [77]
CyIN(ours)
RGB
Depth
Class
SOTA
CyIN(ours)
44.06
43.93
41.82
43.84
Enhancing Tumor
74.6
75.1
49.89
50.46
Tumor Core
85.0
85.8
Whole Tumor
90.6
91.2
Average
45.26
46.07
I
Social Impacts
Devoted in bridging complete and incomplete multimodal learning, the proposed CyIN shows strong potential in real-world field, such as healthcare, education, social media analysis, marketing, advertising, and human- computr interaction. By integrating data from multiple sources such as text, audio, image and video, CylN enables multimodal system for a more detailed understanding of human emotions and intentions.
However, the use of rich and multi-source data introduces risks related to privacy violations, unau- thorized surveillance, and potential misuse for manipulation. In scenarios with missing modalities, efforts to reconstruct data may introduce biases, fabricate faking information, particularly affecting underrepresented groups and raising fairness concerns. These concerns highlight the need of careful and responsible deployment for the proposed method in real-world applications.
31

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The main claims presented the abstract and introduction clearly reflect the paper's contributions and scope. They accurately state the design and capabilities of the proposed CyIN framework in jointly handling complete and incomplete multimodal learning.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation as long as is clear that these goals attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification: Limitations have been discussed in Section Limitations.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and prool's
Question: For each theoretical result, does the paper provide the full of assumptions and complete (and correct) proof?
32

---

Answer: [Yes]
Justification: The paper have provided a complete and correct derivation proof for the mentioned equation in Appendix B.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification: The paper provides detailed information on the hyperparameter settings necessary for reproducibility in Appendix D. Moreover, as stated in the abstract, the source code is publicly released.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, No answer this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how replicate the results, access a hosted model (e.g.. in the of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing verifying the results.
33

---

5. Open access to data and code
Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [Yes]
The datasets used in this paper are publicly accessible and available upon request. Moreover, stated in the abstract, the source code is now publicly released, ensuring transparency and reproducibility.
Justification:
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nig) cc/
public/guides/ Sow.leeeeeoooeSSeeiiiiiiiiiii
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (c.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https: //nips "tippscccpublliiiieeee wttppplcc//aacccccccccccccccecct
/ee/eCoeeeummiiaacccceccee ssi aessonPoliice onnolicy) for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Answer: [Yes]
All the training and test details including dataset splits, hyper-paramette parameters, and type of optimizer and are specified in Appendix D. Besides, following previous methods, the best hyper-ypaue parameters are chosen by fifty-times of random grid search for each model.
Justification:
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: As shown in Appendix H.2, we report the performance variation to demonstrate the superiority of the proposed method across various missing scenarios.
Guidelines:
The answer NA means that the paper does not include experiments.
34

---

The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the enror bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall
with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification: We report the required computer resources in Implementation Details of Section 4.
Guidelines:
The answer NA means that the paper does include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't make it into the paper).
9. Code ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics https: http://wwww..ppppetteeeeee/e///e hwwppaii//tcisstiiiiiiiieaaaaaae ccaiiiiiiieieleeeeee
Answer: [Yes]
Justification: This research conforms all the requirements of moral and ethical norms in the NeurlPS Code of Ethics.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. if there is special consid- eration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: We have discussed both positive and negative social impacts in Appendix
35

---

Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considcrations.
The conference expects that many papers will be foundational research and tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning conrectly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper deseribe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: The paper utilizes public used datasets and pre- treitraied models, which poses no such risks.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model. for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification:
The used data have been correctly cited in the paper and the license and terms of use are properly respected.
Guidelines:
The answer NA means that the paper does not existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
36

---

The name of the license (e.g., CC-BY 4.0) should be included for each asset. For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: As shown in Abstract, the source code publicly released.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
37

---

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if' applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification: The core method development in this research does not involve LLMS as any important, original, non-standard components.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to our LLM policy (https: / //neruurpp.... wcc//onfeeencce/2225/11/11 for what should or should not be described.
38


Reviews:
[{"review_id": "0GIxzS2V1x", "reviewer": "Reviewer_bhs8", "summary": "The authors develop  a novel Cyclic INformative Learning framework (CyIN) to bridge the gap between complete and incomplete multimodal learning, which builds an informative latent space by adopting token- and label-level Information Bottleneck (IB) cyclically among various modalities.", "questions": "Multimodal latent representation learning was once a popular research topic in AI, with classical methods such as CCA widely explored. However, recent progress in this area appears limited. The main change lies primarily in the backbone architecture—shifting from traditional deep neural networks to today’s large language models (LLMs).\n\nI did not see significant contributions from this work to the multimodal learning community. Its primary contribution appears to be the straightforward application of existing multimodal learning techniques—specifically those involving the information bottleneck—on modern LLMs.\n\nThe following is a more subjective comment from my personal perspective: VAE and the information bottleneck framework often feel like \"one-size-fits-all\" solutions that can be applied to virtually any representation learning problem. While they are well-suited for academic exploration and publishing, the complex training procedures involved—like those described in this paper—are unlikely to scale effectively for solving real-world AI tasks. For this reason, I personally do not prefer this type of work.", "limitations": "N/A", "rating": 4, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "1. the paper is well written\n2. the technique details are sound\n3. the experimental results are convincing\n4. the novelty is not enough", "was_revised": true, "final_justification": "The authors clarify their contirbution of this work, which provides a new perspective of information bottleneck to the multimodal learning community in bridging complete and incomplete multimodal learning.", "rebuttal": "We would like to express our sincere gratitude for your thoughtful questions and valuable feedback on our strengths, including **\"well written paper\", \"sound technique details\", and \"convincing experimental results\"**. Our responses to your concerns are presented in lists as follows. **We are eager to engage in further discussions with you to address your concerns and enhance the quality of our work.**\n\n# **Q1: About novelty and contribution.**\n\n**A1:** We appreciate the reviewer’s feedback and would like to clarify and reaffirm the key contributions of our work to the multimodal learning community:\n\n- **Addressing the Limitations of Prior Work**: Many existing approaches either neglect the performance degradation caused by real-world challenges such as sensor failure and corrupted inputs, or suffer from task-irrelevant noise when reconstructing them. Moreover, strategies such as data augmentation or imputation often reduce performance on complete multimodal inputs and demand prior knowledge about the missing situation, making it difficult to jointly handle complete and incomplete multimodal scenarios in real-world applications.\n\n- **A Unified Framework for Complete and Incomplete Multimodal Learning**: Our proposed CyIN framework bridges this gap by unifying complete and incomplete multimodal learning within a shared informative latent space, offering a new direction for building robust multimodal systems capable of handling diverse scenarios without requiring separate models or heuristics. The token- and label-level IB with cyclic translation mechanism over informative latents improves cross-modal interaction in multimodal fusion and enables effective reconstruction of missing modalities.\n\n- **Extensive Empirical Experiment Validation**: We have validated CyIN across 6 sub-tasks and 10 diverse multimodal datasets involving 2–4 modalities. The framework consistently demonstrates strong performance under various missing modality settings. \n\n# **Q2: Well-suited for academic exploration but not scalable for solving real-world AI tasks.**\n\n**A2:** We'd like to provide an open-minded discussion of both the academic and application value of our work. Admittedly, VAE and IB are often seen as \"one-size-fits-all\" solutions, as you suggested. However, academic exploration continues to yield valuable insights, and these models still have broad applicability. For example:\n\n- VAEs are widely adopted in image latent encoding for recent AIGC applications such as Stable Diffusion [4].\n- IB is actively being explored for enhancing safety in large language models [5].\n\nThus, we believe deeper theoretical exploration of VAE and IB in the context of multimodal models is equally important as pursuing application-focused research.\n\nMoreover, to further validate the applicability of our proposed method, we conduct extensive experiments on current industrial multimodal applications, including:\n\n- **Sentiment analysis**: MOSI, MOSEI datasets\n- **Emotion recognition**: IEMOCAP, MELD datasets\n- **Multimodal recommendation** [1]: Amazon Baby, Tiktok, Allrecipes datasets\n- **Face anti-spoofing** [2]: CASIA-SURF dataset\n- **Dense prediction** [2]: NYUv2 dataset\n- **Medical segmentation** [3]: BraTS2023 dataset\n\n***\n**Table.** Experiment comparison for **Multimodal Recommendation** task of accuracy and fairness performance (%) on three datasets, including Amazon Baby, Tiktok, and Allrecipes. The modality setting follows the random missing protocol with 0.4 missing rate as in the SOTA¹ [1] paper.\n\n| Dataset       | Model             | Recall@10 ↑ | Recall@20 ↑ | Precision@10 ↑ | Precision@20 ↑ | NDCG@10 ↑ | NDCG@20 ↑ | F@10 ↑  | F@20 ↑  | F_fuse@10 ↑ | F_fuse@20 ↑ |\n|---------------|-------------------|-------------|-------------|----------------|----------------|------------|------------|---------|----------|---------------|---------------|\n| Amazon Baby   | SOTA¹             | 5.26        | 8.54        | 0.56           | 0.45           | 2.76       | 3.60       | 87.24   | 90.12    | 1.11          | 0.90          |\n|               | **CyIN (ours)**   | **5.43**    | **8.56**    | **0.57**       | 0.45           | **2.89**   | **3.68**   | **90.10** | **92.70** | **1.14**      | 0.90          |\n| Tiktok        | SOTA¹             | 4.81        | 7.39        | 0.48           | 0.37           | 2.95       | 3.60       | 88.15   | 92.27    | 0.96          | 0.74          |\n|               | **CyIN (ours)**   | **4.91**    | **7.57**    | **0.49**       | **0.38**       | **3.15**   | **3.81**   | **89.02** | **92.31** | **0.98**      | **0.75**      |\n| Allrecipes    | SOTA¹             | 2.49        | 3.36        | 0.24           | 0.16           | 1.33       | 1.55       | 96.82   | 89.52    | 0.49          | 0.33          |\n|               | **CyIN (ours)**   | **2.61**    | **3.43**    | **0.26**       | **0.17**       | **1.45**   | **1.66**   | **99.57** | **91.94** | **0.52**      | **0.34**      |\n***\n***\n**Table.** Experiment comparison for **Multimodal Face Anti-spoofing** task with Average Classification Error Rate (ACER ↓) on the CASIA-SURF dataset. The modality setting follows the fixed missing protocol as in the original SOTA² [2] paper.\n\n| RGB | Depth | IR | SOTA² (ACER ↓) | **CyIN (ours) (ACER ↓)** |\n|:---:|:-----:|:--:|:----------------:|:-------------------------:|\n| ✓   |       |    | 7.33             | **4.48**                  |\n|     | ✓     |    | **2.13**         | 2.83                      |\n|     |       | ✓  | 10.41            | **7.75**                  |\n| ✓   | ✓     |    | **1.02**         | 2.26                      |\n| ✓   |       | ✓  | 3.88             | **3.06**                  |\n|     | ✓     | ✓  | 1.38             | **1.20**                  |\n| ✓   | ✓     | ✓  | 0.69             | **0.66**                  |\n|     |       |     |                 |                           |\n| **Average** |     |     | 3.84             | **3.18**                  |\n***\n***\n**Table.** Experiment comparison for **Multimodal Dense Prediction** task with mIoU (↑) on the NYUv2 dataset. The modality setting follows the fixed missing protocol as in the original SOTA² [2] paper.\n\n| RGB | Depth | SOTA² (mIoU ↑) | **CyIN (ours) (mIoU ↑)** |\n|:---:|:-----:|:----------------:|:--------------------------:|\n| ✓   |       | **44.06**        | 43.93                     |\n|     | ✓     | 41.82            | **43.84**                 |\n| ✓   | ✓     | 49.89            | **50.46**                 |\n|     |       |                  |                           |\n| **Average** |                  | 45.26            | **46.07**                 |\n***\n***\n**Table.** Experiment comparison for **Multimodal Medical Segmentation** task with Dice Similarity Coefficient (DSC %) (↑) on the BraTS2023 dataset. The results are averaged over four modalities: FLAIR, T1, T1c, and T2, following the fixed modality setting in the original SOTA³ [3] paper.\n| Class            | SOTA³ (DSC ↑) | **CyIN (ours) (DSC ↑)** |\n|------------------|---------------|--------------------------|\n| Enhancing Tumor  | 74.6          | **75.1**     |\n| Tumor Core       | 85.0          | **85.8**       |\n| Whole Tumor      | 90.6          | **91.2**        |\n***\n\nThe experimental results shown above illustrate that **CyIN achieves state-of-the-art performance**. Besides, we'd like to point out that the training procedure is flexible in different training architecture, including: Transformer-based (Ours), Graph-based (SOTA¹ [1]), CNN-based (SOTA² [2]), Mamba-based (SOTA³ [3])  for both modality-specific feature extraction and multimodal fusion.\n\nLastly, we scale up CyIN with different sizes of PLMs including BERT, RoBERTa, and DeBERTa-V3 as follows, indicating efficient scaling ability similar to the scaling law of PLM training. \n***\n**Table.** Comparison of the proposed **CyIN** using different sizes of language models on **MOSI** dataset under both complete and randomly incomplete multimodal settings.\n\n| Setting                 | Model Variant (PLM) | Acc7 ↑ | F1 ↑   | MAE ↓  | Corr ↑ |\n|-------------------------|---------------------|--------|--------|--------|--------|\n| **Complete**            | BERT                | 48.0   | 86.3   | 0.712  | 0.801  |\n|                         | RoBERTa             | 49.5   | 88.2   | 0.692  | 0.823  |\n|                         | **DeBERTa-V3**      | **50.3** | **90.1** | **0.671** | **0.841** |\n|                         |                     |        |        |        |        |\n| **Random Missing (MR=0.7)** | BERT           | 28.0   | 65.9   | 1.117  | 0.530  |\n|                         | RoBERTa             | 29.3   | 67.2   | 0.992  | 0.556  |\n|                         | **DeBERTa-V3**      | **32.2** | **69.5**   | **0.965** | **0.578**  |\n***\nIn summary, we believe our work provides a new perspective to the multimodal learning community in bridging complete and incomplete multimodal learning.\n\nWe sincerely hope that above responses could address your concerns and provide another perspective for the application of the proposed method.\n\n\n[1] Jin Li et.al. 2025. Generating with Fairness: A Modality-Diffused Counterfactual Framework for Incomplete Multimodal Recommendations. In Proceedings of the ACM on Web Conference 2025 (WWW'25).\n\n[2] Shicai Wei et.al. 2024. Robust Multimodal Learning via Representation Decoupling. 18th European Conference Computer Vision  (ECCV'24).\n\n[3] Pipoli, Vittorio et.al. 2025. IM-Fuse: A Mamba-based Fusion Block for Brain Tumor Segmentation with Incomplete Modalities. 28th International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI'25).\n\n[4] Rombach R et al. High-resolution image synthesis with latent diffusion models[C] Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (CVPR'2022).\n\n[5] Liu Z et al. Protecting your llms with information bottleneck[J]. Advances in Neural Information Processing Systems, 2024 (NIPS'2024)."}, {"review_id": "tBZbsTsj4x", "reviewer": "Reviewer_oakX", "summary": "The paper presents a novel framework, CyIN, designed to address the challenge of multimodal learning in scenarios with both complete and incomplete multimodal inputs.  The authors introduce a cyclic informative latent space that enables efficient cross-modal interaction and multimodal fusion.  This space is constructed using token-level and label-level Information Bottlenecks (IB) to capture task-relevant features and filter out noise.  Additionally, the paper proposes a cross-modal cyclic translation mechanism to reconstruct missing modalities through forward and reverse propagation, enhancing robustness in incomplete input scenarios.  Extensive experiments on four datasets demonstrate that CyIN achieves state-of-the-art performance in both complete and diverse incomplete multimodal learning settings, highlighting its effectiveness and generalization ability.  The contributions include the development of the informative latent space, the cyclic interaction and translation mechanisms, and the unified framework for joint optimization of complete and incomplete multimodal learning.", "questions": "see above", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths\n\n1. The paper proposes a novel Cyclic Informative Learning framework (CyIN) that bridges the gap between complete and incomplete multimodal learning by constructing an informative latent space using token-level and label-level Information Bottlenecks.\n\n2. The introduced cross-modal cyclic translation mechanism enhances the model's robustness and performance in scenarios with missing modalities by reconstructing the missing information through forward and reverse propagation.\n\nWeaknesses\n\n1. The paper assumes that all modalities contribute equally to the final task, but in practice, different modalities may have different importance to the task. For example, in some sentiment analysis tasks, text modalities may be more important than audio and visual modalities. This unbalanced contribution between modalities may affect the performance and generalization ability of the model\n\n2. Although the paper proposes recurrent interaction and translation mechanisms based on information bottlenecks, the computational efficiency and scalability of these mechanisms can be a potential issue when dealing with large-scale multimodal data. For example, when the number of modalities increases or the data dimension is high\n\n3. The experiments mainly focus on sentiment analysis and emotion recognition, and lack corresponding experimental verification for some broader multimodal tasks, such as multimodal question answering and multimodal recommendation. In addition, although the paper provides detailed hyperparameter Settings and training details, no in-depth analysis of the sensitivity of different hyperparameters is performed in the experiments. For example, the impact of the β parameter in the information bottleneck on the model performance and how to adjust β for the best performance under different missing rates can be further explored.", "was_revised": true, "final_justification": "The response resolves most of my concerns, and I will maintain the score.", "rebuttal": "We would like to express our sincere gratitude for your thoughtful questions and valuable feedback of our strengths including **\"novel framework\" and \"enhancing robustness\"**. Our responses to your concerns are presented in lists as follows. **We are eager to engage in further discussions with you to address your concerns and enhance the quality of our work.** \n\n# **Q1: About imbalance contribution of diverse modalities.** \n\n**A1:** In our work, we assumed equal contribution across modalities to ensure generality and stability, particularly under randomly missing modality scenarios. However, we acknowledge that in practice, different modalities may contribute unequally depending on the task [4], summarized as balanced multimodal learning [5], another issue in the field of multimodal learning.\n\nTo partially address this, we increase experiments by varying the number of cross-moda translator layers in Table I. We observe that allocating more layers in reconstruction allows the model to learn more semantics representation, where the results show more important contribution of text modality than others. \n\nWe plan to explore adaptive balancing strategies as part of our future work as stated in Section Limitations.\n***\n**Table I.** Performance comparison of CyIN using different CRA layer ratios for cross-modal translation ($la$:$lv$:$av$) under random missing rate $MR=0.7$ on the MOSI dataset.\n\n| CRA Layer Ratio ($la$:$lv$:$av$) | Acc7 ↑ | F1 ↑  | MAE ↓  | Corr ↑ |\n|-----|----|---|----|---|\n| 4:4:1   | **28.7** | **67.5** | **1.114** | 0.532  |\n| 2:2:1    | 28.5   | 67.2   | 1.122  | 0.521  |\n| **1:1:1** (Original)   | 28.0   | 65.9   | 1.117  | 0.530  |\n| 1:1:2  | 27.5   | 66.3   | 1.119  | 0.531  |\n| 1:1:4  | 27.4   | 65.6   | 1.116  | **0.534** |\n***\n\n# **Q2: About computational efficiency and scalability with modality number and data dimension.** \n\n**A2:** As shown in Table II, CyIN achieves the best inference efficiency and lowest FLOPs when handling three input modalitie, comparing with the state-of-the-art methods such as GCNet and IMDer.\n***\n**Table II.** Computational efficiency for the state-of-the-art methods and the proposed CyIN on MOSI dataset.\n\n| Model | Total Param (M) | Total Training Time (h) | Inference Time (s/iteration) | FLOPs (T) |\n|---|---|---|----|----|\n| GCNet  | 144.34 M  | **1.47h** | 70.62s  | 3.747  |\n| IMDer   | 168.31 M   | 1.89h  | 103.75s   | 5.466  |\n| **CyIN (ours)**  | **123.49 M**   | 1.61h  | **22.41s**  | **1.594** |\n***\nMoreover, CyIN is designed to be scalable as the number of modalities increases. The cross-modal translation mechanisms introduce only a minimal computational overhead relative to prior methods (Each cross-modal translator only increase 1.45 M parameters, $\\sim  10$% of the total model parameters). This is due to the modular nature of the architecture and the efficient reconstruction performance in the informative space. Since reconstruction is performed in this compressed latent space, The computational cost remains manageable in scenarios with high-dimensional data, where IB serves as a compact and informative representation extractor [6,7].\n\nTo further validate scalability, we conducted additional experiments on datasets involving 2 to 4 modalities, as detailed in the next response. These results demonstrate the robustness and efficiency of CyIN across varying modality configurations.\n\n# **Q3: About experimental verification for broader multimodal tasks.** \n\n**A3:** To further validate the applicability of the proposed method, we conduct extensive experiments on current industrial multimodal applications including:\n\n- **Multimodal Recommendation SOTA¹ (3 modalities: visual, audio, textual)** paper: [1] Jin Li et.al. 2025. *Generating with Fairness: A Modality-Diffused Counterfactual Framework for Incomplete Multimodal Recommendations*. In Proceedings of the ACM on Web Conference 2025 (WWW'25).\n\n- **Multimodal Face Anti-spoofing (3 modalities: RGB, Depth, IR) and Dense Prediction (2 modalities: RGB, Depth) SOTA²** paper: [2] Shicai Wei et.al. 2024. *Robust Multimodal Learning via Representation Decoupling*. 18th European Conference Computer Vision (ECCV'24).\n\n- **Multimodal Medical Segmentation (4 modalities: FLAIR, T1, T1c, T2) SOTA³** paper: [3] Pipoli, Vittorio et.al. 2025. *IM-Fuse: A Mamba-based Fusion Block for Brain Tumor Segmentation with Incomplete Modalities*. 28th International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI'25).\n\nThe experimental results shown below illustrate CyIN reaches the state-of-the-art performance.\n***\n**Table III.** Experiment comparison for **Multimodal Recommendation** task of accuracy and fairness performance (%) on three datasets, including Amazon Baby, Tiktok, and Allrecipes. The modality setting follows the random missing protocol with 0.4 missing rate as in the SOTA¹ paper.\n\n| Dataset       | Model             | Recall@10 ↑ | Recall@20 ↑ | Precision@10 ↑ | Precision@20 ↑ | NDCG@10 ↑ | NDCG@20 ↑ | F@10 ↑  | F@20 ↑  | F_fuse@10 ↑ | F_fuse@20 ↑ |\n|---|----|---|----|---|----|---|---|---|---|--|---|\n| Amazon Baby   | SOTA¹    | 5.26   | 8.54  | 0.56  | 0.45   | 2.76       | 3.60       | 87.24   | 90.12    | 1.11  | 0.90    |\n|   | **CyIN (ours)**   | **5.43**    | **8.56**    | **0.57**   | 0.45    | **2.89**   | **3.68**   | **90.10** | **92.70** | **1.14**      | 0.90          |\n| Tiktok        | SOTA¹     | 4.81  | 7.39 | 0.48    | 0.37    | 2.95  | 3.60       | 88.15   | 92.27    | 0.96          | 0.74          |\n|     | **CyIN (ours)**   | **4.91**    | **7.57**  | **0.49**  | **0.38**   | **3.15**   | **3.81**   | **89.02** | **92.31** | **0.98**  | **0.75**      |\n| Allrecipes    | SOTA¹    | 2.49   | 3.36   | 0.24  | 0.16   | 1.33    | 1.55    | 96.82   | 89.52    | 0.49   | 0.33    |\n|      | **CyIN (ours)**   | **2.61**    | **3.43**  | **0.26**  | **0.17**  | **1.45**   | **1.66**   | **99.57** | **91.94** | **0.52**   | **0.34**      |\n***\n**Table IV.** Experiment comparison for **Multimodal Face Anti-spoofing** task with Average Classification Error Rate (ACER ↓) on the CASIA-SURF dataset. The modality setting follows the fixed missing protocol as in the original SOTA² paper.\n\n|  RGB | Depth | IR | SOTA² (ACER ↓) | **CyIN (ours) (ACER ↓)** |\n|:---:|:---:|:--:|:----:|:----:|\n| ✓   |       |    | 7.33   | **4.48**  |\n|     | ✓     |    | **2.13**  | 2.83  |\n|     |       | ✓  | 10.41    | **7.75** |\n| ✓   | ✓     |    | **1.02** | 2.26   |\n| ✓   |       | ✓  | 3.88  | **3.06**  |\n|     | ✓     | ✓  | 1.38   | **1.20**   |\n| ✓   | ✓     | ✓  | 0.69  | **0.66**   |\n|     |    |     |     |     |\n| **Average** |     |     | 3.84  | **3.18**  |\n***\n**Table V.** Experiment comparison for **Multimodal Dense Prediction** task with mIoU (↑) on the NYUv2 dataset. The modality setting follows the fixed missing protocol as in the original SOTA² paper.\n\n|  RGB | Depth | SOTA² (mIoU ↑) | **CyIN (ours) (mIoU ↑)** |\n|:---:|:-----:|:----:|:-------:|\n| ✓   |       | **44.06**  | 43.93   |\n|     | ✓     | 41.82   | **43.84**    |\n| ✓   | ✓     | 49.89   | **50.46**   |\n|     |     |     |      |\n| **Average** |     | 45.26  | **46.07**   |\n***\n**Table VI.** Experiment comparison for **Multimodal Medical Segmentation** task with Dice Similarity Coefficient (DSC %) (↑) on the BraTS2023 dataset. The results are averaged over four modalities: FLAIR, T1, T1c, and T2, following the fixed modality setting in the original SOTA³ paper.\n| Class            | SOTA³ (DSC ↑) | **CyIN (ours) (DSC ↑)** |\n|-----|---|---|\n| Enhancing Tumor  | 74.6  | **75.1**   |\n| Tumor Core       | 85.0 | **85.8**   |\n| Whole Tumor      | 90.6  | **91.2**   |\n***\n# **Q4: About hyper-parameter sensitivity.** \n\n**A4:** We have conducted additional experiments presented in Table VII, which systematically evaluate the effect of varying the most important hyper-parameters $\\beta$ and $\\gamma$. The former controls the trade-off degree of mutual information in IB and the latter adjusts the contribution of translation loss.\n***\n**Table VII.** Hyper-parameter sensitivity on $\\beta$ and $\\gamma$ of the proposed CyIN on MOSI dataset with incomplete multimodal learning of random missing rate $MR=0.7$.\n| Model Variants | Acc7↑ | F1↑ | MAE↓ | Corr↑ |\n|----|----|---|---|----|\n| β=2 | 24.6 | 59.4 | 1.248 | 0.461 |\n| β=4 | 27.5 | 61.3 | 1.226 | 0.478 |\n| β=8 | **30.0** | 63.6 | 1.199 | 0.519 |\n| β=16 | 28.0 | 65.9 | **1.117** | **0.530** |\n| β=32 | 26.4 | **66.0** | 1.118 | 0.506 |\n| β=64 | 23.3 | 64.1 | 1.206 | 0.463 |\n|     |    |    |   |     |\n| γ=1 | 25.8 | 64.1 | 1.197 | 0.437 |\n| γ=5 | 27.5 | 63.1 | 1.135 | 0.488 |\n| γ=10 | **28.0** | **65.9** | **1.117** | **0.530** |\n| γ=15 | 26.8 | 62.9 | 1.272 | 0.519 |\n| γ=20 | 25.5 | 60.4 | 1.311 | 0.490 |\n***\n- Setting $\\beta=8-32$ gives better performance as a balanced trade-off for mututal information between $I(S;B)$ and $I(B;T)$. Very small or large values of $\\beta$ lead to clear degradation in all metrics, showing that improper bottleneck strength harms performance.\n\n- Setting $\\gamma=10$ yields the best overall performance across metrics. A small $\\gamma$ results in weak regularization, making it hard to align modalities under severe missing conditions while a large $\\gamma$ hurts performance greatly dur to neglecting building effective informative space.\n\n[4] Peng X, Wei Y, Deng A, et al. Balanced multimodal learning via on-the-fly gradient modulation. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (CVPR) 2022.\n\n[5] Zhang H, Wang W, Yu T. Towards robust multimodal sentiment analysis with incomplete data[J]. Advances in Neural Information Processing Systems (NeurIPS) 2024.\n\n[6] Shwartz-Ziv R, Tishby N. Opening the black box of deep neural networks via information. arXiv:1703.00810, 2017.\n\n[7] Saxe A M, Bansal Y, Dapello J, et al. On the Information Bottleneck Theory of Deep Learning. International Conference on Learning Representations (ICLR) 2018."}, {"review_id": "NRPM5KKZAk", "reviewer": "Reviewer_WC7q", "summary": "This paper addresses the gap between complete and incomplete multimodal learning by proposing the CyIN (Cyclic INformative Learning) framework. The core idea is to construct an “information-purified latent space” via a two-stage Information Bottleneck (IB) mechanism—Token-level IB and Label-level IB—and then perform cross-modal cyclic translation to reconstruct missing modalities within this space. The authors evaluate their approach on four datasets under three settings.", "questions": "Refer to the above weaknesses.", "limitations": "Yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths\n1. Unified handling of complete and arbitrarily missing inputs with a single model, avoiding the common practice of training separate models for different missing patterns.\n\n2. Clear writing, with Figure 1 providing an intuitive breakdown of the framework and loss components.\n\n3. The experiments are relatively comprehensive, and the results demonstrate their effectiveness.\n\nWeaknesses\n1. Both the Information Bottleneck and cyclic reconstruction approaches (CRA) are established methods; the main contribution lies in their combination. The authors need to explain the connection between the two techniques and why their combination constitutes an optimal solution.\n\n2. The paper does not quantify the cost introduced by the two-stage training, multiple translators, and multi-head cross-attention—specifically in terms of parameter count or inference latency.", "was_revised": true, "final_justification": "I appreciate the authors' efforts in addressing my concerns. I have carefully read the rebuttal. I will keep my score.", "rebuttal": "We would like to express our sincere gratitude for your thoughtful questions and valuable feedback of our strengths including **\"unified handling\", \"clear writing\" and \"comprehensive experiments\"**. Our responses to your concerns are presented in lists as follows. **We are eager to engage in further discussions with you to address your concerns and enhance the quality of our work.**\n\n# **Q1: Connection between IB and Cyclic Translation.**\n\n**A1:** Our key contribution lies in the novel integration for IB and cyclic translation within a unified framework designed for both complete and incomplete multimodal learning. \n\nSpecifically, the IB serves as an effective representation learning principle by encouraging compact and task-relevant latent spaces, which helps reduce the complexity of high-dimensional inputs while preserving discriminative modality-specific and -shared information. Cyclic translation, on the other hand, enhances cross-modal reconstruction by reinforcing consistency and structural alignment between modalities. It captures modality-shared features, facilitating robust reconstruction when some modalities are missing.\n\nWe have increased a theoretical analysis in combining these two approaches in our work as follows. Considering multimodal learning with two modalities without loss of generality, the proposed CyIN can be formulated as follows:\n\nS₁ → B₁ ⇄ B₂ ← S₂  \n   ↓  ↓  \n   T₁ T₂\n\nCombining IB with cyclic translation, we can deduce the theoretical loss as follows,\n\n$$\n\\mathcal{L} = \\mathcal{L} _ {IB} + \\gamma (\\mathcal{L} _ {rec} + \\mathcal{L} _ {cyc}) \n$$\n\n$$\n\\mathcal{L} = \\underbrace{[I(S _ 1; B _ 1) - \\beta I(B _ 1; T _ 1)]} _ {\\text{IB for modality 1}} + \\underbrace{[I(S _ 2; B _ 2) - \\beta I(B _ 2; T _ 2)]} _ {\\text{IB for modality 2}} + \\underbrace{[D _ {KL}(B _ 1; B ^ {rec} _ {2 \\rightarrow 1}) + D _ {KL}(B _ 2; B ^ {rec} _ {1 \\rightarrow 2})]} _ {\\text{Reconstruction from modality 1} \\rightleftharpoons \\text{2}} + \\underbrace{[D _ {KL}(B _ 2; B ^ {cyc} _ 2) + D _ {KL}(B _ 1; B ^ {cyc} _ 1)]} _ {\\text{Translating Back from modality 2} \\rightleftharpoons \\text{1}}\n$$\n\nMinimizing above reconstruction loss, we can obtain informative unimodal bottleneck latents $B _ u$ for each modality $u$, which contains inter-modal features to conduct productive multimodal fusion. Comparing with directly fusing unimodal representations $F _ u$, fusion process implemented on the bottleneck latents $B _ u$, denoted as $F _ M = D _ M(B _ 1, B _ 2)$, can benefit from the compression ability of information bottleneck and lead to less reconstruction difficulty in cross-modal translation.\n\n- When both modalities $u _ 1$ and $u _ 2$ are presented, known as complete multimodal learning, we can regard the cyclic translation $B _ 1 \\rightleftharpoons B _ 2$ as sort of cross-modal interaction enhancing the extraction of modality-shared dynamics. Such cross-modal synergies can benefit the multimodal fusion process $F _ M = D _ M(B _ 1, B _ 2)$ to attain the final discriminative representation.\n\n- When one of the modalities is missing, the framework is required to conduct incomplete multimodal learning. Assuming $u _ 1$ is missing, we can obtain bottleneck latents $B _ 2$ from modality $u _ 2$ to reconstruct the bottleneck latents $B ^ {rec} _ 1$, denoted as $B _ 2 \\rightharpoonup B ^ {rec} _ 1$, which can be further decoded as the supplementary information for the multimodal fusion process $F _ M = D _ M(B ^ {rec} _ 1, B _ 2)$.\n\nIn summary, IB provides compact, robust latent spaces for both understanding and generation, while cyclic translation enhances modality interaction and recovery. Their combination allows CyIN to unify complete and incomplete modality scenarios in the same framework. We believe this integration is not only novel but also essential for scalable and flexible multimodal learning.\n\n# **Q2: Cost for two-stage training and different modules.**\n\n**A2:** To address this concern, we have quantified the additional computational costs introduced by the two-stage training, VIB networks, multiple translators, and multimodal fusion (multi-head cross-attention) in Table I across GCNet, IMDer, and our proposed CyIN.\n***\n**Table I.** Comparison on model complexity for the state-of-the-art methods and the proposed CyIN, including the model parameter (M) and proportion percentage (Pc).\n| Model           | Total Param (M) | Increased Param (M/Pc)     | Detail Module Param (M/Pc)                | Total Training Time (h) | Inference Time (s/iteration) | Detail Module Latency (s)        |\n|-----------------|-----------------|----------------------------|------------------------------------------|---------------------|-----------------------------|---------------------------------|\n| GCNet           | 144.34 M        | 34.86 M / 24.15%           | -                                        | **1.47h**           | 70.62 s                     | -                               |\n| IMDer           | 168.31 M        | 51.89 M / 32.16%           | -                                        | 1.89h               | 103.75 s                    | -                               |\n| **CyIN (ours)** | **123.49 M**    | **14.0 M / 11.35%**        | VIB (1.74 M / 12.42%)                     | 1.61h | **22.41s** | VIB (0.37 s / 1.65%)                   |\n|                 |                 |                            | Translator (1.45 M / 10.35%)              |                     |                             | Translator (17.51 s / 78.13%)              |\n|                 |                 |                            | Fusion (9.64 M / 68.80%)                  |                     |                             | Fusion (0.26 s / 1.16%)                  |\n***\nCyIN achieves the lowest total parameter count (123.49M) and significantly lower inference time (22.41s) compared to GCNet (70.62s) and IMDer (103.75s). Besides, CyIN costs $17.4\\ \\mathrm{s/epoch}$ for Stage 1, $213.9\\ \\mathrm{s/epoch}$ for Stage 2, where Stage 1 for training informative space is relatively fast. \n\nThe added parameters and inference latency from the increased modular components as VIB and Translator modules are relatively minor. Specifically:\n\n- VIB introduces only 1.74M parameters (1.41\\% of the total, or 12.42\\% of increased parameters), and 0.37 s latency\n\n- Translator adds 1.45M (1.18\\% of the total, or 10.35\\% of increased parameters), and 17.51 s latency (Reconstruction for all missing modalities scenarios, 2.92s for each missing scenario)\n\n- Fusion module accounts for the majority increased parameters (9.64M, or 68.80\\% of increased parameters), and 0.26 s inference latency.\n\nThese results demonstrate that the proposed modular extensions are lightweight relative to the overall architecture. Most of the additional cost arises from the fusion mechanism, not from the cyclic translators or VIB modules. Despite these additions, CyIN remains the most efficient in terms of both parameter size and inference latency. We will explore faster translation way like one-step flow matching or consistency model to further speed up the translation process in the future."}, {"review_id": "RjYEAxzAL0", "reviewer": "Reviewer_cbWP", "summary": "The paper proposes CyIN, a unified framework to jointly handle both complete and incomplete multimodal learning via a cyclic informative latent space. The core contributions include (1) a dual Information Bottleneck (IB) module at token- and label-levels to extract semantically rich yet compact features, and (2) a cyclic cross-modal translation module that enables forward and reverse reconstruction of missing modalities using cascaded residual autoencoders. The method is evaluated on four datasets under complete, fixed missing, and random missing settings, demonstrating state-of-the-art performance. However, the paper does not clearly distill the core problem it aims to address, and further refinement is needed to improve its clarity and focus.", "questions": "* The motivation of the paper.\n* What theoretical guarantees, if any, does the proposed method offer?\n* Training overhead analysis.", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "**Strength:**\n* CyIN elegantly bridges the gap between complete and incomplete modality inputs with one model, avoiding the need to retrain for each missing-modality configuration.\n* The use of token-level and label-level Information Bottlenecks effectively filters noise and preserves task-relevant information, leading to better fusion and reconstruction.\n* CyIN achieves SOTA or near-SOTA performance on MOSI, MOSEI, IEMOCAP, and MELD, consistently outperforming baselines in both complete and incomplete setups.\n\n**Weaknesses:**\n* The authors state in the abstract that the proposed method aims to address the challenge of missing modalities. However, in the fourth paragraph, they enumerate several limitations of existing methods—such as insufficient exploitation of missing information, lack of unified modeling, and poor generalization and robustness. Despite listing these issues, the proposed method does not explicitly address **all of them**, which may lead to confusion regarding the scope and objectives of the paper.\n* While Information Bottleneck is a sound principle, the theoretical grounding of combining IB with cyclic autoencoding for modality reconstruction is not rigorously established.\n* Parameter count, FLOPs, and training time are not reported. Furthermore, all experiments focus on affective/emotion datasets; effectiveness on large-scale or non-affective tasks remains unknown.\n* Typo: builds -> build (in line 9)", "was_revised": true, "final_justification": "The authors have adequately addressed my main concerns in the rebuttal. Therefore, I will increase my score.", "rebuttal": "We would like to express our sincere gratitude for your thoughtful questions and valuable feedback of our strengths including **\"elegantly bridge the gap\", \"effectively filter noise and preserve task-relevant information\" and \"consistently outperforming baselines\"**. Our responses to your concerns are presented in lists as follows. **We are eager to engage in further discussions with you to address your concerns and enhance the quality of our work.**\n\n# **Q1: About the addressed limitation.**  \n**A1:** The fourth paragraph outlines several known limitations of prior work, which motivated the design of our method. Below, we explain how CyIN attempt to address each limitation:\n\n- **Insufficient exploitation of missing information**:  \nCyIN incorporates a cyclic cross-modal translation mechanism that allows available modalities to reconstruct missing ones via bottleneck latents, promoting deeper exploitation of cross-modal information. Unlike previous direct imputation method, our model leverages available modalities to reconstruct the missing ones in a compact informative space.\n\n- **Task-unrelated noise interruption**:  \nCyIN introduce IB to filter out irrelevant noise and redundancy by compressing representation. The informative space retains task-relevant information for multimodal representation, which not only reduce the reconstruction difficulty in incomplete multimodal learning, but benefit representation compression in complete multimodal learning.\n\n- **Lack of unified modeling for varying missing modality scenarios**:  \nCyIN can handle arbitrary missing modality patterns without retraining separately by explicitly modeling diverse mising modalities scenarios with cyclic translation in a shared informative latent space, which improves generalization under unknown missing circumstances in real-world.\n\n- **Sacrificing complete multimodal performance**:  \nCyIN consistently maintains state-of-the-art performance for both complete and incomplete modality inputs.\n\n# **Q2: Theoretical grounding of combining IB with cyclic translation.**  \n\n**A2:** We present further explanation for the theoretical analysis of CyIN. The optimization objective for combining IB with cyclic translation is deduced as: \n$$\n\\mathcal{L} = \\mathcal{L} _ {IB} + \\gamma ( \\mathcal{L} _ {rec} + \\mathcal{L} _ {cyc} )\n$$\n\nConsidering multimodal learning with two modalities without loss of generality, the proposed CyIN can be formulated as follows:\n\nS₁ → B₁ ⇄ B₂ ← S₂  \n   ↓  ↓  \n   T₁ T₂\n\n- The left and right side with $S_1 \\rightarrow B_1 \\rightarrow T_1$ and $S_2 \\rightarrow B_2 \\rightarrow T_2$ denote the chain of information bottleneck. Note that the source and target states have $S/T\\in$ { $F_{S_1},F_{S_2}$ }/{ $F_{T_1},F_{T_2}$ } in the token-level IB or $S\\in$ { $F_{S_1},F_{S_2}$ },$T \\in y$ in the label-level IB, both of which have been theoretically deduced in our paper, formulated as:\n$$\n\\mathcal{L}_{IB} = \\underbrace{[I(S _ 1; B _ 1) - \\beta I(B _ 1; T _ 1)]} _ {\\text{IB for modality 1}} + \\underbrace{[I(S _ 2; B _ 2) - \\beta I(B _ 2; T _ 2)]} _ {\\text{IB for modality 2}} \n$$\n- The middle part $B _ 1 \\rightleftharpoons B _ 2$ denotes the cyclic translation with \n$B _ 1 \\rightharpoonup B ^ {rec} _ {2 \\rightarrow 1} \\rightharpoonup B ^ {cyc} _ 1 \\sim B _ 1$ and \n$B _ 2 \\rightharpoonup B ^ {rec} _ {1 \\rightarrow 2} \\rightharpoonup B ^ {cyc} _ 2 \\sim B _ 1$, where \n$\\rightharpoonup$ denotes the translation process while $\\sim$ denotes alignment with the original bottleneck. Considering $\\Gamma$ as the translator network, the translation process can be divided into cross-modal reconstruction across diverse unimodal latents $B ^ {rec} _ {1 \\rightarrow 2}=\\Gamma _ {1 \\rightarrow 2}(B _ 1) $ and cyclic translation from the reconstructed latent back to the origin modality\n$B ^ {cyc} _ 2 = \\Gamma _ {2 \\rightarrow 1}(B ^ {rec} _ {1 \\rightarrow 2})$, optimized by:\n\n$$\n\\mathcal{L} _ {rec} = \\underbrace{D _ {KL}(B _ 1; B ^ {rec} _ {2 \\rightarrow 1})} _ {Reconstruction\\ from\\ modality\\ 1 \\rightharpoonup 2} + \\underbrace{D _ {KL}(B _ 2; B ^ {rec} _ {1 \\rightarrow 2})} _ {Reconstruction\\ from\\ modality\\ 2 \\rightharpoonup 1} ,\\quad \\mathcal{L} _ {cyc} = \\underbrace{D _ {KL}(B _ 2; B ^ {cyc} _ 2)} _ {Translating\\ Back\\ from\\ modality\\ 2 \\rightharpoonup 1} + \\underbrace{D _ {KL}(B _ 1; B ^ {cyc} _ 1)} _ {Translating\\ Back\\ from\\ modality\\ 1 \\rightharpoonup 2}\n$$\n\n- When both modalities are presented as complete multimodal learning, we can regard the cyclic translation $B_1 \\rightleftharpoons B_2$ as sort of cross-modal interaction enhancing modality-shared dynamics extraction. Such cross-modal synergies can benefit the multimodal fusion process $F_M=D_M(B_1,B_2)$ to attain the final discriminative representation. \n\n- When one of the modalities is missing, the framework is required to conduct incomplete multimodal learning. Assuming $u_1$ is missing, we can obtain bottleneck latents $B_2$ from modality $u_2$ to reconstruct the bottleneck latents $B^{rec}_1$, denoted as $B_2 \\rightharpoonup B^{rec}_1$, which can be further decoded as the supplementary information for the multimodal fusion process $F_M=D_M(B^{rec}_1,B_2)$. \n\nThe aforementioned two-modality scenarios can be generalized to arbitrary modality pairs without constraint, thereby facilitating efficient scaling abilityof CyIN.\n\n# **Q3: Parameter count, FLOPs, and training time.**\n\n**A3:** We have reported the computational efficiency in Table . Compared with the state-of-the-art methods, the proposed CyIN achieves the lowest total parameter count (123.49M) and FLOPs (1.594T). The inference time of CyIN is $\\sim3\\times$ faster than GCNet and $\\sim5\\times$ faster than IMDer. \n\nDue to the cyclic translation process during training, the training time of CyIN is slightly slower than GCNet leveraging Graph Neural Network in modality reconstruction, but still faster then IMDer utilizing diffusion models for reconstruction.\n***\n**Table I.** Computational efficiency comparision on MOSI dataset.\n| Model | Total Param (M) | Total Training Time (h) | Inference Time (s/iteration) | FLOPs (T) |\n|-|-|-|-|-|\n| GCNet | 144.34 M | **1.47h** | 70.62s | 3.747 |\n| IMDer | 168.31 M | 1.89h | 103.75s | 5.466 |\n| **CyIN (ours)**| **123.49 M** | 1.61h | **22.41s** | **1.594** |\n***\n# **Q4: Experiments on large-scale or non-affective tasks.**\n\n**A4:** We have extended experiments on 6 non-affective datasets with 2–4 modalities, including:\n\n- **Multimodal Recommendation SOTA$^1$** Paper: [1] Jin Li et.al. 2025. Generating with Fairness: A Modality-Diffused Counterfactual Framework for Incomplete Multimodal Recommendations. In Proceedings of the ACM on Web Conference 2025 (WWW'25).\n\n- **Multimodal Face Anti-spoofing and Dense Prediction SOTA$^2$** Paper: [2] Shicai Wei et.al. 2024. Robust Multimodal Learning via Representation Decoupling. 18th European Conference Computer Vision (ECCV'24).\n\n- **Multimodal Medical Segmentation SOTA$^3$** Paper: [3] Pipoli, Vittorio et.al. 2025. IM-Fuse: A Mamba-based Fusion Block for Brain Tumor Segmentation with Incomplete Modalities. 28th International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI'25).\n\nThe experimental results shown below illustrate CyIN reaches the state-of-the-art performance, further demonstrating the effectiveness of CyIN in generalizing to diverse mainstream architectures.\n\nThe scale of datasets varies from 1.4k samples (NYU v2) to 65k samples (Tiktok), 87k samples (CASIA-SURF), 139k samples (Amazon Baby), which partially show the scaling performance of the proposed CyIN. We leave exploration on larger scale datasets in the future work.\n***\n**Table.** Experiment for **Multimodal Recommendation** task of accuracy and fairness (%) under random missing protocol with 0.4 missing rate as in the SOTA¹ paper.\n| Dataset  | Model | Recall@10 ↑ | Recall@20 ↑ | Precision@10 ↑ | Precision@20 ↑ | NDCG@10 ↑ | NDCG@20 ↑ | F@10 ↑  | F@20 ↑  | F_fuse@10 ↑ | F_fuse@20 ↑ |\n|-|-|-|-|-|-|-|-|-|-|-|-|\n| Amazon Baby | SOTA¹ | 5.26 | 8.54 | 0.56 | 0.45 | 2.76 | 3.60 | 87.24   | 90.12 | 1.11 | 0.90 |\n| | **CyIN (ours)** | **5.43** | **8.56** | **0.57** | 0.45 | **2.89** | **3.68** | **90.10** | **92.70** | **1.14** | 0.90 |\n| Tiktok | SOTA¹ | 4.81  | 7.39 | 0.48 | 0.37 | 2.95  | 3.60 | 88.15 | 92.27 | 0.96 | 0.74 |\n| | **CyIN (ours)** | **4.91** | **7.57** | **0.49**  | **0.38**   | **3.15** | **3.81** | **89.02** | **92.31** | **0.98**  | **0.75**  |\n| Allrecipes | SOTA¹ | 2.49   | 3.36 | 0.24  | 0.16 | 1.33 | 1.55 | 96.82 | 89.52 | 0.49   | 0.33 |\n| | **CyIN (ours)** | **2.61** | **3.43** | **0.26** | **0.17** | **1.45** | **1.66** | **99.57** | **91.94** | **0.52** | **0.34** |\n***\n**Table.** Experiment for **Multimodal Face Anti-spoofing** task on CASIA_SURF dataset with ACER (↓) under fixed missing protocol as in the original SOTA² paper.\n|  RGB | Depth | IR | SOTA² | **CyIN (ours) ** |\n|:-:|:-:|:-:|:-:|:-:|\n| ✓ | | | 7.33 | **4.48** |\n| | ✓ | | **2.13** | 2.83 |\n| | | ✓ | 10.41 | **7.75** |\n| ✓ | ✓ | | **1.02** | 2.26 |\n| ✓ | | ✓ | 3.88 | **3.06** |\n| | ✓ | ✓ | 1.38 | **1.20** |\n| ✓ | ✓ | ✓ | 0.69  | **0.66** |\n| | | | | |\n| **Average** | | | 3.84 | **3.18** |\n***\n**Table.** Experiment for **Multimodal Dense Prediction** task on NYU v2 dataset with mIoU (↑) under fixed missing protocol as in the original SOTA² paper.\n| RGB | Depth | SOTA² | **CyIN (ours)** |\n|:-:|:-:|:-:|:-:|\n| ✓ | | **44.06**| 43.93 |\n| | ✓ | 41.82 | **43.84** |\n| ✓ | ✓ | 49.89 | **50.46** |\n| | | | |\n| **Average** | | 45.26  | **46.07** |\n***\n**Table VI.** Experiment for **Multimodal Medical Segmentation** task on BraTS2023 dataset with DSC (%) (↑), averaged over four modalities under fixed modality setting in the original SOTA³ paper.\n| Class | SOTA³ | **CyIN (ours)** |\n|-|-|-|\n| Enhancing Tumor | 74.6 | **75.1** |\n| Tumor Core | 85.0 | **85.8** |\n| Whole Tumor | 90.6 | **91.2** |\n***\n\n# **Q5: Typo.** \n**A5:** Thank you for pointing out. We will carefully check the manuscript."}]

Author rebuttal:

Meta-review: The paper introduces a method called “CyIN”, short for Cyclic INformative Learning framework, which builds a joint latent space using token- and label-level information bottlenecks applied cyclically across modalities. This space enables cross-modal reconstruction of missing modalities through forward and reverse propagation. The approach achieves good performance on several datasets.

__Strengths__:
- Addresses a real-world problem where multimodal systems often encounter missing or corrupted inputs during deployment
- Interesting combinations of information theory with cross-modal translation 
- Clear presentation with intuitive figures and comprehensive experimental evaluation

__Weaknesses__:
- The novelty lies primarily in combining existing techniques (e.g. information bottleneck and cyclic reconstruction) rather than introducing fundamentally new concepts
- The paper assumes equal importance across modalities, which may not reflect real-world scenarios where certain modalities dominate
- Some reviewers found the theoretical justification for combining information bottleneck with cyclic translation could be more rigorous
- The multi-stage training process and multiple components add complexity that may limit practical adoption


The consensus is borderline but on the side of ‘acceptance’ suggesting this paper is suitable for Neurips.
Decision: Accept (poster)