MANGO: Multimodal Attention-based Normalizing Flow Approach to Fusion Learning
Authors: Thanh-Dat Truong
Christophe Bobda
Nitin Agarwal
Khoa Luu
Venue: NeurIPS 2025 poster ()
Keywords: Multimodal Fusion
Multimodal Learning
Normalizing Flows
Primary area: applications
TL;DR: This paper introduces a novel Multimodal Attention-based Normalizing Flow approach to developing explicit, interpretable, and tractable multimodal fusion learning

Abstract: Multimodal learning has gained much success in recent years. However, current multimodal fusion methods adopt the attention mechanism of Transformers to implicitly learn the underlying correlation of multimodal features. As a result, the multimodal model cannot capture the essential features of each modality, making it difficult to comprehend complex structures and correlations of multimodal inputs. This paper introduces a novel Multimodal Attention-based Normalizing Flow (MANGO) approach to developing explicit, interpretable, and tractable multimodal fusion learning. In particular, we propose a new Invertible Cross-Attention (ICA) layer to develop the Normalizing Flow-based Model for multimodal data. To efficiently capture the complex, underlying correlations in multimodal data in our proposed invertible cross-attention layer, we propose three new cross-attention mechanisms: Modality-to-Modality Cross-Attention (MMCA), Inter-Modality Cross-Attention (IMCA), and Learnable Inter-Modality Cross-Attention (LICA). Finally, we introduce a new Multimodal Attention-based Normalizing Flow to enable the scalability of our proposed method to high-dimensional multimodal data. Our experimental results on three different multimodal learning tasks, i.e., semantic segmentation,  image-to-image translation, and movie genre classification, have illustrated the state-of-the-art (SoTA) performance of the proposed approach.

MANGO: Multimodal Attention-based Normalizing Flow Approach to Fusion Learning
Thanh-Dat Truong', Christophe Bobda', Nitin Agarwal Khoa Luu' CVIU Lab, University of Arkansas, USA UUiiersitty of Florida, USA $COSMOS Research Center, University of Arkansas, Little Rock, USA *ICSI, University of California, Berkeley, USA (tt032, khoaluu)@uark, edu cbobdadece ufl.edu, nxagarwalQualr edu https://nark- cviu.
witpieit.uuuuuu.uu.. github..hh..... hotpro/jeeoooo////MOOO
Abstract
Multimodal learning has gained much success recent years. However, current multimodal fusion methods adopt the attention mechanism of Transformers implicitly learn the underlying correlation of multimodal features. As a result, the multimodal model cannot capture the essential features of each modality, making it difficult to comprehend complex structures and correlations of multimodal inputs. This paper introduces a novel Multimodal Attention-basel Normalizing Flow (MANGO) approach to developing explicit, interpretable, and tractable multimodal fusion learning. In particular, we propose a new Invertible Cross-Attention (ICA) layer to develop the Normalizing Flow-based Model for multimodal data. To efficiently capture the complex, underlying correlations in multimodal data our proposed invertible cross-attention layer, we propose three new cross-attention mechanisms: Modality-to- Modalality Cross-Attention (MMCA), Inter-Modality Cross-Attention (IMCA), and Learnable Inter-Modality Cross-Attention (LICA). Finally, we introduce a new Multimodal Attention-bases based Normalizing Flow enable the scalability of our proposed method to high-dimensional multimodal data. Our experimental results on three different multimodal learning tasks, i.e., semantic segmentation, image-to-image translation, and movie genre classification, have illustrated the state-of-the-art (SoTA) performance proposed approach.
Introduction
Human perceptions interpret the surrounding world in multimodal way via multiple input channels, such as vision, text, or audio. The deep learning- based multimodal fusion methods have majorly improved the performance var- ious problems, e.g., classification (20
Secer Hodetty
Mirrhhll Aotdtie
51
X
38 37
58), action recognition 112
54
57
semantic
X
segmentation (65] (56] 59 61
60 object detec-
tion [72). The recent large multimodal mod-
els. e.g., ChatGPT (T). Gemini [52]. Chaemelon
[51], LLaMMA [53]. etc, introduced for general- assistant purposes have also shown impressive performance on these applications.
Hastrry de Socaety semtering
Snifacallyy Parttis
Figure 1: Our Cross-Modality Fusion Approach Via Multimodal Normalizing Flows with Invertible Cross-Attention.
The critical success of multimodal fusion meth- ods relies on the interaction and correlation mod-
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

eling mechanisms across input modalities. The recent methods 20 65| adopt the attention mech- anisms of Transformers 64 to implicitly model the cross- modslityy correlation. By training on large-scale data, the attention models can implicitly learn the underlying correlation represented in the
data. For example, the rision-language fusion models [28 27 48 use early fusion where the visual tokens and textual tokens are simultaneously fed into the Transformer model. Then, Transformers
will learn the correlation and alignment between visual and textual tokens via the second-order correlation learning of the attention mechanism. Under this form, these multimodal fusion methods are alignment-agnostic, where the cross-modal alignments and correlations are not fully exploited [65]. In addition, the implicit fusion method often associates information across modalities without distinctly modeling the unique characteristics and correlations of each modality. Then, it may over- look the contribution of specific modalities, mainly if one modality contains more data stronger signals, leading to suboptimal performance [65 20]. Since the implicit approach cannot individually model the importance of each modality, these methods could struggle to capture complex structures and complementary information represented in the multimodal data. Implicit modeling methods also lack interpretability since it is hard understand or represent the contributions of each modality to the outputs. Other methods [33 I g adopted the late fusion, where the features are fused after each of the modalities has been decided. However, late fusion ignores the low-level interaction across modalities. As a result, the direct adoption of fusion with attention could not improve performance compared to the unimodal methods [65 20
While most recent multimodal methods adopt attention to capture the multimodal correlations implicitly, the explicit modeling approach has been less investigated [6 23]. The normalizing flow-based model [5] 21 48) is a common approach to explicit modeling. By modeling the exact likelihoods of data via the bijective mapping between the data and latent spaces, the normalizing flow-based models allow for stable and reliable training, gaining better insight into the model representations of the underlying multimodal data distribution. In particular, by stacking set of bijective transformations, the explicit models can construct complex distributions, enabling them to capture multimodal data distributions with direct control over parameters. Thus, this explicit modeling approach enhances interpretability and enables a better understanding of multimodal features and correlations in the latent space, which can be challenging to access in prior methods 665 20). Compared to prior methods [65] 20), explicit modeling via normalizing flows provides a better multimodal fusion mechanism since it can capture the underlying structures and correlation of multimodal data without letting any single modality dominate. Therefore, explicit modeling enables more precise, flexible, and robust multimodal fusion, improving performance in tasks requiring understanding and good alignment of multimodal data.
The Challenges in Multimodal Normalizing Flows.
While explicit modeling is a potential approach to multimodal fusion, developing multimodal normalizing flows requires several efforts. Indeed, there are two significant limitations in the current normalizing flow-based models. First, while the affine coupling layer (5) [2 allows for the properties of tractability and invertibility, this layer limits the expressiveness of the models. Unlike the attention mechanism in Transformers [64], the coupling layer cannot capture the wide-range data dependencies and correlation in multimodal data 48 Second, scaling the normalizing flow-based models to high-dimensional data is a challenging problem. It requires stacking more bijective layers in the models, leading to high computational cost and hard convergence during training [5) While the implicit modeling approaches have alleviated the computational overhead using latent models (e.g.. Latent Diffusion (42). there are limited studies to address this overhead problem in normalizing flow-based approaches. Therefore, there is an urgent need to address these limitations to develop an efficient multimodal normalizing flow-based model.
Contributions of this Work. This paper introduces the new Multimodal diention-based Normalizing Flows (MANGO), an explicit, interpretable, and tractable approach, to multi modaiiym fusion prob- lems (Fig. D. To the best of our knowledge. this is one of the first studies that develops a Normalizing Flow approach to multimodal fusion learning. Our contributions can be summarized as follows. First, we propose a new Invertible Pross-Attention (ICA) layer for Normalizing Flow-based Models. The proposed ICA layer can efficiently address the limitations of coupling layers in the standard Nor- malizing Flows while maintaining its tractability and invertibility properties. Second, to capture the correlation and alignment across modalities, we present three new partitioning pross-attention mecha- nisms, including Mitalittytt-Modallll Cross-Attention (MMCA), Inder- Cross-Attention (IMCA), and Learnable Inter-Modalin modal Attention- based Normalizing Flow approach with a latent model to enable its scalability to
(LICA). Third, we present novel Multi-
2

---

high-dimensional multimodal data fusion. Our approach can address the limitations of computa- tional overhead while efficiently modeling complex correlations in multimodal data. Finally, our experiments on three multimodal learning tasks, i.e., semantic segmentation, image-to-image transla- tion, and movie genre classification, have shown the effectiveness of MANGO different aspects, demonstrating its State-of-the- (SoTA) performance compared to prior multimodal models.
2 Related Work and Background
2.1 Related Work
Attention Models. The attention mechanism in Transformers has shown outstanding performance in unimodal and multimodal learning [64 28 65 Using the second-order correlation, the attention mechanism can capture the long-term relation across input modalities. There are two common types of attention in Transformers, i.e., self-attention and pross-attention. While self-attention focuses learning correlations within a single input modality [64], cross-attention models relationships across modalities, allowing the model to analyze complex correlations from modality to another [68]. Transformers have hecome a dominant approach and have profound impacts in developing various multimodal tasks, e.g., large vision-language model 28 27]. RGB-D object segmentation [65]
Multimedal Fusion. Multimodal fusion learning has shown its outstanding advantage over the uni- modal counterparts in various tasks, e.g., semantic segmentation 665 20 image-to-image translation 18]. action recognition [1). object detection 772.. etc. The early approaches of multimodal fusion learning adopted simple feature concatenation to fusion the information from multiple modalities 7 76). Then, later works further improved the cross- modss-m fusion by using deep fusion via a neural network, e.g., RNN 22) LSTM [50], Attention [55] [5). etc. Another approach adopted the neural architecture search to search for appropriate networks for multimodal fusion [ 73 [0]. The current state-of-the-arr fusion approaches utilize early fusion to capture cross- cross-modality interactions at the data level via Transformers 665
By combining all modalities at an initial stage via input tokens, the Transformers will learn to model the correlation across modalities via self-attention
20
28
[64]. The later work further improved the early fusion method using pixel- wise fusion [20), pruning techniques [65] dynamic multimodal fusion [70]. However, it should be noted that these current multimodal fusion methods are an implicit modeling approach.
Explicit Modeling via Normalizing Flows. To develop the invertible network, RealNVP [5] first introduced an affine coupling layer where its reverse version and the log-determinant of the Jacobian matrix be easily computed. Later work [4] 21] further improved the coupling layers by introducing non-linear independent component estimation [4], invertible convolution 21 34], activation normalization (21), autoregressive modeling 117. multi-scale architectures [5]. equivariant normalizing flows [9] Another approach [15] 48 enhanced the expressiveness of the coupling layer by using Transformers in the scaling and translation network. However, it still cannot address the problem of long-range dependencies and complex cross-modality correlation in the data. Recent studies further developed the conditional flow-based approach, e.g., conditional image synthesis using conditional invertible networks 47 or two invertible networks 49
31 32
2.2
Limitations of Normalizing Flows
The typical normalizing flow model 5 21 is designed via the invertible affine couple layer as:
X X. partition(X)
Y, X,, Y. = X2 @ exp exp (8(X1)) T(X.)
(I)
Y merge([Y.. Yal)
where X is an input, partition is a partition method, e.g., RealNVP 5 adopts checkerboard par- titioning method, S and T are deep neural networks, merge is merging function, and @ is the element-wise matrix multiplication.
Limitations. The success of flow-based model relies on the design of the invertible layers. However, the current affine coupling layers remain inefficient modeling complex data. First, the expressiveness of the coupling layer is limited due to its simple design. The design of S and T via residual networks [5) could not capture the complex relationships represented in the high-dimensional data. Thus, it still struggles to capture highly intricate dependencies correlations in the data,

---

especially in multimodal data. Second, scaling to high-ding sigh-dimensional data increases the complexity of the flow-based model, which can make the training unstable and inefficient. If the number of coupling layers is shallow, the model may fail to capture the complex relationships and dependencies in the multimodal data. This leads to poor performance in tasks like density estimation or fusion modeling. Thus, the high-dimensional data also requires more layers to capture all the necessary correlations among all tokens, increasing computational cost. In this paper, we will develop a new ereetion-bbsed Normalizing Flow approach to addressing these prior limitations in normalizing flows and multimodal fusion.
3 The Proposed Multimodal Attention-based Normalizing Flow (MANGO) Approach
Most recent multimodal models adopt Transformers with an attention mechanism to learn the cross-modality correlations [65 20, 27.. However, prior research suggested this fusion approach inefficient [35]. Indeed, the correlations learned via supervveeri or weak supervision cannot provide explicit attention modeling across modalities and will be ineffective when the information of multimodal inputs is sparse. In addition, as cross-modality correlations are generally high- dimensional and complex, developing a multimodal model capable of capturing complex correlations challenging.
Therefore, to address this problem, this paper will model the cross-modality correlations as the joint distributions. Then, the joint distributions can be further modeled using the Normalizing Flow-based Model, a tractable yet powerful approach to modeling complex distributions with bijective mapping functions. Fig. 2 iillustrates the overview of our proposed Multimodal Attention-basee Normalizing Flow-based framework. Formally, let X be the multimodal input (e.g.. RGB and Depth images), G be the bijective network that maps the inputs into the latent space, i.e., Z G(X). The prediction Y can be obtained via the projection head as Y TaskHead (). where TaskHead is the projection head that produces the task-specific outputs (e.g., semantic segmentation). Then, the multimodal data distribution p p(X) can be formed via the Normalizing Flow-based Model G' as in Equ.
(G(() ax
p (Z)
(2)
where T (Z) is the prior Normal distribution. In our approach, we assume that inputs X can be tokenized as X = (x1...,,xy X where N is the number of tokens. For simplicity, we assume the X consists of two input modalities (e.g.. RGB and Depth images), i.e., X = [x1, [x,...,X X,, XM+1, "r. Xx] where [x,,..,, XM) and [xM+1,.... XN] belong to the first and second modality.
3.1 The Proposed Invertible Cross-Attention (ICA)
We introduce a novel Invertible Cross-Attention to address the prior limitations in Normalizing Flow-based models. The success of attention mechanisms relies the capability of exploring the relationship among features via sccond-orde, correlations. In particular, the design of the attention
layers can be formulated as Attention( Q, K. V) softmax (9*K" Va
V where Q, K, and V are the query, key, and value features obtained by applying linear projection to the input X, and is the scale-dot product. The query and key are used to learn the attention weights via a scaled dot product. Then, this attention information is accumulated into the value vector, which allows the final outputs
wartbir Creae Attenton Layar
Hode-odaity Pantton
Wne b Gratee Amtoitnn Layer
Inten Mo 1a Partiion
Wnventa wwwrttaa Crate Siteriiinn Layer
Creodes
Modalty to Wedilty Autioo
Oooplig How
JuNum
Merg
Weige
XoNomm
Lasrable
Dipth Exc22
Figure 2: Our Proposed Multimodal Attention-baser Fusion Learning.
Normalizing Flows (MANGO) Approach to

---

(QxK" M be Va the cross attention matrix. Thanks to the auto-regressive modeling, the inverse matrix of A always exists since A is the upper triangular matrix. It should be noted that the diagonal of A is always greater than 0 due to the softmax properties. Therefore, approach can efficiently ensure the will be a learnable parameter to to carry the correlation among tokens. Inspired by this attention design, we propose the ICA within the coupled layer as in Equ.
X, X2 partition((x:, xx))
IN(LP((1)), IN(LL((X)))
[QK*
(3)
X1, softmax
V
vd
I.. Ya))
where LN is the layer norm, LP is the linear projection, and d is the feature dimension. This cross-attention mechanism aims to model the inter-token interaction via the attention weights. The attention information in the first patch of inputs (X1) is embedded into the second patch of inputs (X2). By scaling into multiple invertible cross-aftention layers and alternating the token partitions, our proposed approach can efficiently capture the correlation among inputs, especially in multimodal data, since the attention information across input partitions is exchanged interwisely.
The success of the current state-of- the-art of lcrge-scale generative models, e.g., Large Language Models (LLM)
Invertibilty.
Eerword
Invers
[53]. Large
V-
Vision-Language Models (LVM) 28 27 re- lies on the auto-regressive modeling. Indeed, the auto-regressive form naturally aligns with the nature of the data, where each input token depends on the previous ones. This modeling approach can model the highly complex dependencies within the multimodal data and maintain consistency and coherence. our learning approach, we propose to model the invertible attention layer via the auto-regressive form. In particular, our ICA layer in Eqn. 3 can be reformed as in Eqn.
Figure 3: Our Invertible Pross-Attention (ICA).
Q Q K M
Y2 softmax
(4)
Vd
where M is the upper triangular matrix to ensure the auto-regressive modeling property. Under this form, the inverse process of our ICA can be formulated as in Eqn. (5).
Y1. Y2 partition([y YN))
LN(LP(Y,)), K LN(LP(Y.)), V=Y,
Q
(5)
X, X,, X.
softmax
M
V
Vd
= mergel [X Xal)
Fig- 3 illustrates the forward and inverse process of the ICA layer. Let A softmax
invertibility of the cross-attention layers. Inspired by 48 664 capture general scale.
Tractability. One of the crucial properties required by the Normalizing Flow-based model is the tractability of the determinant of the Jacobian matrix, i.e., det. (ox). Formaaly, the determinant of the Jacobian matrix of our ICA can be formed as in Eqn. (6).
det
BY
(det(A)) = det softmmax
( M) N/2
M)
(6)
Va
Since A is an upper block triangular matrix due the autoregressive form, the determinant can be simply computed as the product along the diagonal of the matrix.
3.2 The Partitioning Approaches to Cross-Modality Attention
As shown Eqn. B the partitioning method plays a vital role learning the correlation across modalities since it will decide which attention information will be exchanged within the invertible
5

---

cross-attention layers. To support the correlation learning across modalities, we propose to design three different partitioning approaches capture different types cross-modality attention (Fig. (
For simplicity, we rewrite the multimodal input X = [xi:..,, XM, XM+1, xN] as X x
(x/,,..,, xf ,... xx] where K is the number of tokens of the second modality, i.e., N = M K.
Attes-Atttntion (MMCA). To capture the attention from the first to the second modality (or vice versa), the partition function in Eqn. can be formed
Modality-to-Modaliy
X1
X1
Figure 4: Our Proposed Partitioning Ap-
X2
X2
proaches: Modality-oo-Modality Cross-Attention
AT A
partision 2
AEC
(Left). Inter-Modality Cross-Attention (Middle).
MMCA where partition A-B and partition
MMCA are Hgge Aydgl-Moddaiiii S BaantinrdillngginAAARARAee
While the first partition method partition MMCA allows the ICA layers to capture the correlation of the first modality to the second modality, partition MMrtitionMMMAA MMCA will model the attention in the reverse direction, i.e., from the second the first modality. Under this approach, the intra-attention informa- tion can be exchanged across modalities effectively. Then, the merging method of the corresponding partitioning function can be formulated as Eqn. (8).
merrge (Y) Y2]) [Y,, Y2) mmerge (Y Y2)) 2Y2 Y
partition 2 X
parcition 4
This merging method aims to maintain the consistency of the token positions by reorganizing the positions of the output tokens corresponding to their original ones in input X.
Inter-Modality Atees-Attention (IMCA). To model the inter-attention across modalities, our partitioning function can be formulated as follows,
X1
[x]
M/2 x
x/22
partition IMGA
(9)
X2 = (xM/2+1, XM 1/22+
x3
x3
where partition IMCA is the inter-modality partitioning method. Our partitioning method has four different ways to divide partitions, i.e., (X1, X2) E ([X) A } XX),X))). ([X),, X2rl X, (IX [x,,
.[33, ,x,,xx inter-modality partitioning approach allows the cross-attention layer to capture the correlation across modalities efficiently. Then, the modality attention in the first partition (X,) can be
Our
embedded into the second partition (X2). Then, the merging method of the partitioning function partition' /MCA can be formulated as in Eqn. TOP.
merge Y1, Y2) -Y A Y Y B.
Y B
(10)
where Y A
Y2 B: Y are the corresponding outputs of X X X B. X3 produced by ICA.
Cross-Attention (LICA). To further improve IMCA learning, we intro- duce a new Learnable Inter-Modality Cross-Attention as follows,
Learnable
X1 = [X Xyyzl
X' =
X x = XW per
partition LICA
(11)
X2 = (//2211 Xxe
where W
per the learnable permutation matrix.
To maintain the permutation property of the matrix W per, we adopt the LU Decomposition [21 as W per PL(U + diag(s)). where P is the fixed permutation matrix, L and U are the learnable lower and upper triangular matrices with ones and zeros on the diagonal, and s is the learnable vector. Since W per is the permutation matrix, the inverse permutation matrix W-1 can be computed and
the Jacobian determinant of 8X can be determined via the vector i.c., log det
ax"
ax' 8X
=E |s]).
Our approach can efficiently capture the underlying cross-attention across input modalities using the proposed LICA approach. Then, the merging method of the learnable partitioning function can be formed via the inverse permutation W as follows:
merge ( Y2)) YY,, W
(12)

---

3.3 Multimodal Latent Normalizing Flows
The typical likelihood- based model has two stages. First, the perceptual compression stage focuses on removing fighffreuencyy details while learning little semantic information. Sec- ond, the semantic compression stage will learn the semantic and conceptual composition repre- sented in the data [4]] As a result, the second stage plays more important role since it is an actual generative model that learns the semantic structures and cross- tross-modality correlations rep- resented in the multimodal data. The original data is often represented in high-dimensional space, e.g., high-resolution images or long se- quence data. However, the semantic information of the data can be represented in a much lower- dimensional space since the input space has re- dundant dimensions. Therefore, based on the intrinsic dimensionality of the input data, we aim to find a perceptually equivalent but computationally efficient space for our multimodal normalizing flow-based approach.
Table I: Comparison of RGB-D Semantic Segmen- tation Performance on NYUDv2 and SUN RGB-D with Prior Methods. Our metries include Pixel Ac- curacy (Pixel Acc.) (%), Mean Accuracy (mAcc.) (%), Mean Intersection over Union (mloU) (%).
Ingurs
Pool
NYCD52
Piesl
SUNROID
FCN-325 EU
RGB
60.0
4222
25.2
68.4
41.1
25.0
RefiscNec 220
RGB
74.4
50.6
47.6
81.1
57.7
47.0
50.4
76.3
48.1
RDFNet
RGB:D
76.0
81.5
Assmiliaioon
RGB40
71.0
66.0
51.2
CEN (5)
RGB+0
77.7
65.0
$3.5
63.
632
$1.1
DPENN
RGB44
IMSANN M1
RG]l++
50.9
WD Fasion
[0]
RGB
75.2
02.5
49.7
62.3
6066
47.0
Ieaanno Catrai
b
76.5
65.8
40.8
K2.8
61.4
47.9
Tedenfiiccmm (Tmy)
RGGlle
78.6
16.3
6600
61.1
W/O fusion (Smal)
63.0
Concel (Smal)
RGBB
TedumFiacam (Simall
6:]
RGB00
70.0
54.2
64.
61.1
55.0
Gammiiiniioe (-51
KGl+e
60.3
20.4
53.3
838
65.3
55.5
STANGO
KGBRD
43.5
71A
39.
K39
073
561
Perceptual Compression. Inspired by the success of prior work 42 propose to project the data into a much representation. Let & be the encoder that maps the input X in to the latent feature F, i..., F = E(X). Then, the decoder D will map the features back into its original data space, i.e., X D(F). The
feature space but with more meaningful information in the
design of encoder & and the decoder DD can be varied, e.g., PCA, Autoencoder 41 14. However, to achieve the best capability of perceptual compression, we adopt the autoencoder approach [41 [4] to develop the encoder and the decoder. This approach can provide equivalent to the data space while maintaining the lower-dimensional space.
input space perceptually
Latent Normalizing Flow-based Model. Instead of modeling the multimodal data X on its original high-dimensional space, we propose to model the data distribution via its multimodal feature F on the latent space as p(F) - π (Z)
(G(F)
OF We named this method the Multimodal Attention-based
Normalizing Flow Approach with Latent Model. With our approach, the flow-based model does not need learn to perform perceptual compression on digeesiinnsio data. Instead, our normalizing flow approach will focus on learning the semantic information and correlation of multimodal data. As result, our model exhibits better scaling properties while using an efficient computational cost. In addition, the bijective network G designed via our Invertible Cross-Arre -ttts-Attention layers offers better multimodal modeling via second-order correlation learning.
A Attention-base Normalizing Flow Network. Our bijective network G (Fig. consists of L blocks where each block consists of eight invertible cross-attention layers and coupling layer (5). The first two cross- cooss-attention layers adopt the MMCA partitioning. The following four cross-attention layers perform different IMCA partitioning approaches. The next two layers utilize LICA Cross-Attention layers. Then, the coupling layer is adopted to increase the inner expressiveness of the bijective blocks.
Learning MANGO With Task Specific. Given the multimodal input X and the label of a specific task Y, MANGO can be jointly optimized via the negative log-likelihood and the task-specific learning objective in Eqn. (3)
min Ex [- log 7(2) log
8G(F)
OF
where Z = G))( Y is the prediction of the corresponding task, Lrask is the loss of the corresponding prediction task, and is the parameters of the model.
Experimental Results
4.1 Implementation and Benchmarks
Implementation. Our bijective network G consists of T. = 12 cross-attention blocks. For the perceptual compression encoder E, adopt the visual encoder of [4] for both RGB and Depth
7

---

images. We utilize the text encoder from 40 for the textual data. For fair comparisons, we use the task heads of semantic segmentation, image translation, and movie genre classification from [70] 65]. Our experiments are conducted on the 4 NVIDIA A100 GPUS. Our training uses the same learning hyper-parameters from (65] and an input image size of 256 256 for fair comparisons.
Semantic Segmentation. This task uses the two homogeneous inputs of RGB and Depth images to predict the segmentation maps. We perform experiments on NYUDv2 (36) and SUN RGB-D (45). While NYUDv2 consists of 795/654 images for training and testing splits, SUN RGB-D includes 5,285/5,050 samples for training and testing.
Image-to-Image Translation. Following the standard protocol in [65], we adopt the Taskonomy [75 for the multimodal image translation task. This large-scale indoor scene dataset provides over ten multimodal, e.g., RGB, Depth, Normal, Shade, Texture, Edge, etc. We use a subset of 1,000 high-quality images for training and 500 for validation.
MM-IMDB Movie Genre Classification. MM-IMDB a large-scale multimodal dataset for movie genre classification. We adopt the training and testing split of [70] for fair comparisons. In particular, the data in our experiments consists of 15,552 data for training and 2,608 for validation. In this multimodal learning task, we use the inputs from two modalities of images and texts.
4.2 Comparison with State-of-the-Art Methods
Semantic Segmentation. Table I presents our results compared to prior multimodal methods on multimodal semantic segmentation. Our results show the proposed approach achieves state-of-the-an performance the NYUDv2 and SUN RGB-D datasets. Our model con- sistently outperforms the prior methods in all evaluation metrics and datasets. In particular, the mIoU results of our proposed approach are higher than GeminiFusion by 1.5% and 0.6% on the two datasets. Our results have illustrated that our explicit modeling of multimodal fusion has shown a clear advantage over the prior fusion methods [20, 651.
both
Figure 5: Qualitative Comparison on
NYUDv2 Benchmark.
Fig. 5 visualizes the results of our fusion approach via our normalizing flows compared to the prior fusion method, i.e., TokenFusion 65
Image-to-Image Translation. We present our results on the five different learning settings of multimodal Image-to-Image Translation as Our results consistently out- perform prior methods in five different multi- modal learning settings. In particular, compared to prior GeminiFusion (20). our models have
Table 2: Comparison of Multimodal Image Trans- lation Perfor- mance on Taskonomy with Prior Multimodal Methods. We use evaluations of
shown in Table 2
FID//KD /ID (x10-2) for the RGB target and MAE (x10-1)/MM //ME (x10-1) for Normal, Shade, and
Depth targets.
BSade Textard Tall:TTTeeDeepelllaea DaplaNNenaa RGBx RGB CCB-NNeellf CCB-NenelRRGBBEAA RGB+
gained better FID scores, i.c., lower than Gem- iniFusion by 1.71 and 29.37 on benchmarks of Shade+Texture - RGB and Depth+Norma + RGB. These results further confirm the outstand- ing capability of our approach to capture com- plex correlations across modalities.
1397235
125
Selt-Argggenn
73
1.20/2.20
1188/
50.4.20
105
21/22.1
50
CEN (e)
63.33
B4 33/32220
112/251
1.051.73
0.56 6L
Hileniaase (lias) 63
14.40/1.23
78.35/299
0/25/1300
7.54
0.21
Fostan Coool (Ste 1115
72:45/2.36
$6051409
111/2.75
1.90/200
0 OO
0.751
41.32/111
TEANSO
BYAT 1.4
MM-IMDB Movie Genre Classification. Table 3
presents the results our approach on the multimodal classification benchmarks. As shown in Table 3, our pro- posed approach oulperforms the prior methods on both micro-average and maero-average F1 scores and achieves ofte-of-the--a performance. Particularly, our results of Micro and Macro Fl scores remain higher than the prior method [69 by 3.5% and 4.9%. These results illustrate that our approach performs better on homogeneous and heterogeneous inputs. Fig. [6 visualizes our results on the Image-to- Image-to-Immae translation benchmark.
Figure 6: Qualitative Comparison on Image-to-Image Benchmark.
8

---

4.3 Ablation Studies
Effectiveness of Invertible Cross-Attention Layers. To illustrate the impact of our proposed invertible cross- attention layers, we conduct experiments to compare our
Table 3: Comparison of Movie Genre Classification Performance on the MM- IMDB dataset with Prior Multimodal Methods. Our metries include the Micro-
proposed layers with other flow models, i.e., Affine Cou-
pling Layer [5). Glow [21] Flow++ [15). and AttnFlow 48. As shown in Table 4 our proposed invertible cross- attention layer consistently outperforms the prior coupling methods. In particular, the mIoU results of our method achieved up to 59. 99 and 54.1% on both NYUDv2 and SUN RGBD benchmarks. These results clearly illustrate the advantages of our proposed method for modeling cor- relations and complex structures in multimodal data.
Average and Macro-Average F1 Scores.
Miem
Macro
Method
Modality
(S)
Fl (%%
Image Network 70
40.0
25.3
Text Network 770
59.2
47.2
Late Fusion
A4
I+1
59.6
51.0
LKTF 20
1+T
59.2
49.3
MI-Matrix 19
I+T
58.5
48.4
DynMM
[+T
60.4
51.6
COCA 24
I+T
67.7
62.6
MFM
Effectiveness of Different Partitioning Approach. Table 5 prrsennts the experimental results of different partitioning approaches. As shown in the results, using Modality-to- Modality and Inter-Modality Cross-attention, the mIoU results on both NYUDv2 and SUN RGBD benchmarks have achieved 58.0% and 53.7%. Moreover, when the Learnable Inter-Modality Cross- Attention is adopted, our mIoU results are further improved by 59.2% and 54.1% compared those without LICA. The experimental results have confirmed the effectiveness of our proposed approach in modeling correlation across modalities via our cross- cross-attenticn attention mechanism.
BLIP 22
I+T
67.4
62.8
ReFNer
58.7
Bridge Tow (69)
L+T
68.2
63.3
MANGO
Effectiveness of Latent Model. These experi- ments study the effectiveness of our latent model approach. As shown in T the performance of our multimodal normalizing flow-based mod- els is consistently improved on both seman- tic segmentation benchmarks using the latent
Table 4: Effectiveness of Invertible Lavers.
Layer
Pixtel Ave.
NYUDVI
mAce.
mloU
Pivle] Aec.
SUN RCBD
mare.
mloU
Coupling Loper
7000
63.4
2038
793
39.5
45.3
Glow
27.0
66.4
53.0
80.3
61.9
69.1
flow+
03
77.5
68.1
54.2
81.5
62.0
50.5
Amfitw
79.5
69.8
56.5
82.5
65.1
52.2
MANGO
81.5
71.6
59.2
67.3
model. The proposed method achieves the SoTA results where the mloU results of our best model have achieved 59.2% and 54.1% on NYUDv2 and SUN RGBD benchmarks. The results have highlighted the advantages of using the perceptual compression encoder to produce a lower but more efficient representation space.
Effectiveness of Number of Cross-Attention Blocks. Table G illustrates the results of our approach using different numbers (L) of cross- attention blocks. As in our results, the perfor- mance of multimodal segmentation models us- ing a deeper network results in better performance. In particular, using L = 12 blocks of invertible cross-attention blocks, the mloU performance on NYUDv2 and SUN RGBD benchmarks has reached up to 59.2% and 54.1% 1%, respectively. While fewer blocks may result in lower computational costs. the deeper model can exploit better correlation of features in multimodal data.
Table 5: Effectiveness of Partitioning Approaches.
MMCA IMCA LICA
NYUD22
SUN RGBD
Pideel Acc. mAre. mloU Fixld Acc.
mAre. mloU
19.3
56.4
82.
64.6
51.3
80.2
70,8
58.0
833
66.2
53.7
81.5
71.6
59.2
83.9
67.2
54.1
5 Conclusions
Our paper has introduced a new explicit model- ing approach multimodal fusion learning via
Table 6: Effectiveness of Latent Model.
the Attention-based Normalizing Flow-Based Model. Our proposed ICA layers with three different cross-attention mechanisms have effi- ciently captured the complex structure and un- derlying correlations in multimodal data. We have also introduced new latent approach to normalizing flows to increase our scalability to multi- modal data. Our intensive experiments on three standard benchmarks, i.e., Semantic Segmentation, Prage-to-Immae Translation, and Movie Genre Classification, have shown the effectiveness of our approach. Our study has demonstrated the effectiveness of invertible cross-attention layers in multi- modal learning under selected hyperparameters and benchmarks. However, it still remains limitations in objective balancing and scalability. The detailed limitations are discussed in our appendix.
Blocks
Model
Pisiel Ace.
NYUD!
mAcc.
mdaU)
Piskel Ace.
SUN ROBD
mAce.
mioU
15.9
63.
31.0
591
43.5
6588
52.3
60.5
53.1
65.3
54.1
6001
51.4
58.0
65.8
59.5

---

Acknowledgment. This work is partly supported by NSF CAREER (No. 2442295), NSF SCH (No. 2501021), NSF E-RISE (No. 2445877), NSF SBIR Phase 2 (No. 2247237) and USDA/NIFA Award. We also acknowledge the Arkansas High- Pigh-Peefforancc Computing Center (HPC) for GPU servers. Nitin Agarwal's participation was supported by U.S. NSF (0IA-194639) OIA-1920920), AFOSR ARO (W911NF-23-1-0011 (N00014-21-1-2121,, N00014-21-1-2765, 100014-22-1-2318) AFRL, DARPA, Australian DSTO
(FA9550-22-1-0535 12--0333),
24-02--0000 W91 W911NF-25-1-0147, ONR
Strategic Policy Grants Program, Arkansas Research Alliance, the Jerry L. Maulden/Entergy Endow- ment, and the Donaghey Foundation at the University of Arkansas at Little Rock.
References
[1] Achiam, Adler, S. Agarwal, L. Ahmad, Akkaya, F. L. Aleman, D. Almeida, J. Altenschmidt, S. Altman, S. Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08874. 2023.
[2] Arevalo, Solorio, M. Montes-y Gómez, and A. González. Gated multimodal units for information fusion. arXiv preprint 1702.197,,,,9992, 2017.
[3] I.. Braz, V. Teixeira, H. Pedrini, and Z. Dias. Image-text integration using multimodal fusion network module for movie genre classification. In 11th International Conference of Pattern Recognition Systems (ICPRS 2021), volume 2021, pages 200-205, 2021.
[4] L. Dinh, D. Krueger, and Y. Bengio. Nice: Non-linear independent components estimation. arXiv preprint arXivv 2014.
[5] L. Dinh, J. Sohl-DDikstein, and S. Bengio. Density estimation using real nvp. arXiv preprint 1rXiv:1605.08803 2016.
[6] S. Dong, Y. Feng, Q. Yang, Y. Huang, D. Liu, and H. Fan. Efficient multimodal semantic segmentation via dual-prompt learning, arXiv preprint arXiv:2312.00360, 2023.
[7] A. Ephrat, I. Mosseri, O. Lang. T. Dekel, K. Wilson, A. Hassidim. W. T. Freeman, and M. Rubinstein. Looking to listen the cocktail party: A speaker-independeer audio-visual model for speech separation. arXiv preprint arXiv: 808610160.6....... 2018.
[8] G. Evangelopoulos, A. Zlatintsi, A. Potamianos, P. Maragos, K. Rapantzikos, G. Skoumas, and Avrithis. Multimodal saliency and fusion for movie summarization based on aural, visual, and textual attention. IEFF Transactions Multimedia, 15(7);1553- 2013.
[9] V. Garcia Satorras, E. Hoogeboom, F. Fuchs, I. Posner, and M. Welling. E (n) equivariant normalizing flows. Advances in Neural Information Processing Systems, 34:4181-4199 2021.
[10] M. I. E. Ghebriout, H. Bouzidi, S. Niar, and H. Ouarnoughi. Harmonic-nas: Hardware-aware multimodal neural architecture search on resource-constrained devices. In Asian Conference Machine Learning, pages 374-389. PMLR, 2024.
[11] M. Glodek, S. Tschechne, G. Layher, M. Schels, T. Brosch, S. Scherer, M. Kächele, M. Schmidt, H. Neumann, G. Palm, et al. Multiple classifier systems for the classification of audio-visual emotional states. In Affective Computing and Intelligent Interaction: Fourth International Conference, ACII 2011, Memphis, TN USA, October 9-12, 2011, Proceedings, Part II, pages 359-368. Springer, 2011.
[12] X. Gong, S. Mohan, N. Dhingra, J.-C. Bazin, Y. Li, Z. Wang, and R. Ranjan. Mmg-ego4d: Multimodal generalization in egocentric action recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6481-649) 2023.
[13] C. Hazirbas, L. Ma. C. Domokos, and D. Cremers. Fusenet: incorporating depth into semantic segmentation via fusion-based enn architecture. In Asian Conference on Computer Vision, November 2016.
[14] K. He, X. Chen, S. Xie, Y. Li, P. Dollár, and R. Girshick. Masked autoencoders are scalable vision learners. In Proceedings of the IEEF/CVE conference on computer vision and pattern recognition, pages 16000-16009, 2022.
10

---

[15] J. Ho, X. Chen, A. Srinivas, Y. Duan, and P. Abbeel. Flow++: Improving flow-based generative models with variational dequantization and architecture design. In International conference on machine learning, pages 2722-2730. PMLR, 2019.
[16] C. Hu, B. Fu, P. Yu, L. Zhang, X. Shi, and Y. Chen. An explicit multi-modal fusion method for sign language translation. In ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 3860-3864. IEEE, 2024.
[17] C.-W. Huang, D. Krueger, Lacoste, and A. Courville. Neural autoregressive flows, In International conference machine learning, pages 2088-2087. PMLR, 2018.
[18] X. Huang, M.-Y. Liu, S. Belongie, and J. Kautz. Multimodal unsupervised image-to-iman translation. In Proceedings of the European conference on computer vision (ECCV), pages 172-189, 2018.
[19] S. M. Jayakumar, W. M. Czarnecki, J. Menick, J. Schwarz, J. Rae, Osindero, Y. W. Teh, T. Harley, and R. Pascanu. Multiplicative interactions and where to find them. In International Conference on Learning Representations, 2020.
[20] D. Jia, J. Guo, K. Han, H. Wu, C. Zhang, C. Xu, and X. Chen. Geminifusion: Efficient pixel-wise multimodal fusion for vision transformer. arXiv preprint arXiv:2406.01210, 2024.
[21] D. P. Kingma and P. Dhariwal. Glow: Generative flow with invertible 1x1 convolutions. Advances neural information processing systems, 31, 2018.
[22] Li, D. Li, C. Xiong, and S. Hoi. Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation. In ICML, 2022.
[23] M. Liang. J. Hu, C. Bao, H. Feng, F. Deng, and T. L. Lam. Explicit attention-enhaneed fusion for rgb-thermal perception tasks. IEEE Robotics and Automation Letters, 8(7):4060-4067, 2023.
[24] P. P. Liang, Y. Lyu, X. Fan, Z. Wu, Y. Cheng, J. Wu, I. Y. Chen, P. Wu, M. A. Lee, Y. Zhu, al. Multibench: Multiscale benchmarks for multimodal representation learning. In Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 1), 2021.
[25] X. Liang, Q Guo, Y. Qian, W. Ding, and Q. Zhang. Evolutionary deep fusion method and its application in chemical structure recognition. IEEE Transactions on Evolutionary Computation, 25(5):883-893, 2021.
[26] G. Lin, A. Milan, C. Shen, and I. Reid. Refinenet: Multi-path refinement networks for high- resolution semantic segmentation. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017.
[27] H. Liu, C. Li, Y. Li, and Y. Lee. Improved baselines with visual instruction tuning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 26296-26306, 2024.
[28] H. Liu, C. Li, Q. Wu, and J. Lee. Visual instruction tuning. Advances in neural information processing systems, 36, 2024.
[29] Z. Liu, Y. Shen, V. B. Lakshninarasimhan, Liang, A. Zadeh, and L.-P. Morency, Efficient low-rank multimodal fusion with modality-specific factors. In Annual Meeting of the Association for Computational Linguistics, 2018.
[30] J. Long, E. Shelhamer, and T. Darrell. Fully convolutional networks for semantic segmentation. In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition (CVPR), June 2015.
[31] Y. Lu and B. Huang. Structured output learning with conditional generative flows. In Pro- ceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 5005-5012, 2020.
11

---

[32] A. Lugmayr, M. Danelljan, L. Van Gool, and R. Timofte. Srflow: Learning the super-resolution space with normalizing flow. In Computer Vision-ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part V 16, pages 715-732. Springer, 2020.
[33] E. Morvant, A. Habrard, and S. Ayache. Majority vote of diverse classifiers for late fusion. In Structural, Syntactic, and Statistical Pattern Recognition: Joint IAPR International Workshop. S+ SSPR 2014, Joensuu, Finland, August 20-22, 2014, Proceedings, pages 153-162. Springer, 2014.
[34] S. Nagar, M. Dufraisse, and G. Varma. Cine flow: Characterizable invertible 3x3 convolution. arXiv preprint arXiv:2107.01358 2021.
[35] A. Nagrani, S. Yang, A. Amab, A. Jansen, C. Schmid, and C. Sun. Attention bottlenecks for multimodal fusion. Advances in neural information processing systems, 34:14200- 14213, 2021.
[36] P. K. Nathan Silberman, Derek Hoiem and R. Fergus. Indoor segmentation and support inference from rgbd images. In ECCV, 2012.
[37] H.-Q. Nguyen, T.-D. Truong, X. B. Nguyen, A. Dowling, X. Li, and K. Luu. Insect-foundation A foundation model and large-scale Im dataset for visual insect understanding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 21945-21955, 2024.
[38] X.-B, Nguyen, C. N. Duong, X. Li, S. Gauch, H.-S. Seo, and K. Luu, Micron-bert: Bert-based facial micro-expressinn recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1482-1492) 2023.
[39] S.-J Park, K.-S. Hong, and S. Lee. Rdfnet: Rgb-d multi-level residual feature fusion for indoor semantic segmentation. In The IEEE International Conference on Computer Vision (ICCV), Oct 2017.
[40] A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, Mishkin, J. Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pages 8748-8763 PMLR, 2021.
[41] A. Razavi, A. Van den Oord, and O. Vinyals. Generating diverse high-fidelity images with vq-vae-2. Advances in neural information processing systems, 32, 2019.
[42] R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages
2022.
[43] S. Sankaran, D. Yang, and S.-N. Lim. Refining multimodal representations using a modality- centric self-supervised module, 2022.
[44] D. Seichter, S. Fischedick, M. Köhler, and H.-M. Gross. Efficient multi-task rgb-d scene analysis for indoor environments. In IEEE International Joint Conference on Neural Networks (IJCNN, pages 1-10, 2022.
[45] S. Song, S. P. Lichtenberg, and J. Xiao. Sun rgb-d: A rgb-d scene understanding benchmark suite. In Proceedings of the IEEE conference computer vision and pattern recognition, pages 567-576, 2015.
[46] S. Song, J. Liu, Y. Li, and Z. Guo. Modality compensation network: Cross-modal adaptation for action recognition. IEEE Transactions on Image Processing. 29:3957-3969, 2020.
[47] M. Sorkhei, G. E. Henter, and H. Kjellström. Full-glow: Fully conditional glow for more realistic image generation. In DAGM German Conference Pattern Recognition, pages 697-711. Springer, 2021.
[48] R. S. Sukthanker, Z. Huang, S. Kumar, R. Timofte, and L. Van Gool. Generative flows with invertible attentions. In Proceedings of the IEEE/CVE Conference on Computer Vision and Pattern Recognition, pages 11234-11243, 2022.
12

---

[49] H. Sun, R. Mehta, H. H. Zhou, Z. Huang, S. C. Johnson, V. Prabhakaran, and V. Singh. Dual- glow: Conditional flow-based generative model for modality transfer. Proceedings of the IEEF/CVF International Conference on Computer Vision, pages 1061 10611-10620, 2019.
[50] Z.-X. Tan, A. Goel, T.-S. Nguyen, and D. C. Ong. A multimodal Istm for predicting listener empathic responses over time. In 2019 14th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2019), pages 1-4. IEEE, 2019.
[51] C. Team. Chameleon: Mixed-modal early-fusion foundation models. arXiv preprint
arXiv:2405.098/6 2024.
[52] G. Team, R. Anil, S. Borgeaud, J.-B. Alayrac, J. Yu, R. Soricut, J. Schalkwyk, A. M. Dai, A. Hauth, K. Millican, et al. Gemini: family of highly capable multimodal models. arXiv preprint arXiv:23/2./1880) 2023.
[53] H. Touvron, T. Lavril, G. Izacard, X. Martinet, M.-A. Lachaux, T. Lacroix, B. Rozière, N. Goyal, E. Hambro. F. Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.1339
2023.
[54] T.-D. Truong. Q.-H. Bui, C. N. Duong, H.-S. Seo, S. L. Phung, X. Li, and K. Luu. Direcformer: directed attention transformer approach to robust action recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 20030-20046 2022.
[55] T.-D. Truong, C. N. Duong, T. De Vu, H. A. Pham, Raj, N. Le, and K. Luu, The right to talk: An audio-visual transformer approach. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 1105-1114, October 2021.
[56] T.-D. Truong, N. Le, B. Raj. J. Cothren, and K. Luu. Fredom: Fairness domain adapta- tion approach semantic scene understanding. In IEEE/CVF Computer Vision and Pattern Recognition (CVPR), 2023.
[57] T.-D. Truong and K. Luu. Cross-view action recognition understanding from exocentric to egocentric perspective. Neurocomputing, 614: 128731, 2025.
[58] T.-D. Truong, H.-Q. Nguyen, X.-B. Nguyen, A. Dowling, X. Li, and K. Luu. Insect-foundation: A foundation model and large multimodal dataset for vision-language insect understanding. International Journal of Computer Vision, pages 1-26. 2025.
[59] T.-D. Truong, H.-Q. Nguyen, B. Raj, and K. Luu. Fairness continual learning approach to semantic scene understanding in open-world environments. Advances in Neural Information Processing Systems, 36, 2024.
[60] T.-D. Truong, U. Prabhu, Raj. J. Cothren, and K. Luu. Falcon: Fairness learning via contrastive attention approach to continual semantic scene understanding. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 15065-15075, 2025.
[61] T.-D. Truong, U. Prabhu, D. Wang, B. Raj, S. Gauch, Subbiah, and K. Luu. Eagle: Efficient adaptive geometry- learning in cross-view understanding. Advances in Neural Information Processing Systems,
2024.
[62] A. Valada, R. Mohan, and W. Burgard. Self-supervised model adaptation for multimodal semantic segmentation. International Journal of Computer Vision (IJCV. jul 2019. Special Issue: Deep Learning for Robotic Vision.
[63] A. Valada, R. Mohan, and W. Burgard. Self-supervised model adaptation for multimodal semantic segmentation. International Journal of Computer Vision, 12365): 2020.
[64] A. Vaswani. Attention is all you need. Advances in Neural Information Processing Systems, 2017.
[65] Y. Wang, X. Chen, L. Cao, W. Huang, F. Sun, and Y. WWang Multimodal token fusion for vision transformers. In Proceedings of the IEEF/CVE conference on computer vision and pattern recognition, pages 12186-12195, 2022.
13

---

[66] Y. Wang, W. Huang. F. Sun, T. Xu, Y. Rong, and J. Huang. Deep multimodal fusion by channel exchanging, Advances neural information processing systems, 33:4835-4844 2020.
[67] Y. Wang, Sun, M. Lu, and A. Yao. Learning deep multimodal feature representation with asymmetric multi-layer fusion. In ACM International Conference Multimedia (ACM MM), 2020.
[68] X. Wei, T. Zhang, Y. Li, Y. Zhang, and F. Wu. Multi-modality cross attention network for image and sentence matching. In Proceedings of the IEEE/CVF conference computer vision and paltern recognition, pages 10941-10950, 2020.
[69] X. Xu, C. Wu, S. Rosenman, V. Lal, W. Che, and N. Duan. Bridgetower: Building bridges between encoders in vision-language representation learning. arXiv preprint arXiv:2206.08657, 2022.
[70] 7. Xue and R. Marculescu. Dynamic multimodal fusion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pages 2575-2584, 2023.
[71] B. Yin, X. Zhang, Z. Li, L. Liu, M.-M. Cheng, and Q. Hou. Dformer: Rethinking rgbd representation learning for semantic segmentation. arXiv preprint arXiv:2309.0900 09666, 2023.
[72] Yin, Shen, R. Chen, W. Li, R. Yang, P. Frossard, and W. Wang. Is-fusion: Instance-scene collaborative fusion for multimodal 3d object detection. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 14905-14915, 2024.
[73] Y. Yin, S. Huang, and Zhang. Bm-nas: Bilevel multimodal neural architecture search. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 8901-8909 2022.
[74] Yu, Z. Wang, Vasudevan, L. Yeung, M. Seyedhosseini, and Y. Wu. Coca: Contrastive captioners are image-text foundation models. arXiv preprint arXiv:2205.0199 019117 2022.
[75] A. R. Zamir, A. Sax, Shen, J. Guibas, Malik, and S. Savarese. Taskonomy: Disentangling task transfer learning. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 3712-3722, 2018
[76] H. Zhao, C. Gan, A. Rouditchenko, C. Vondrick, J. McDermott, and A. Torralba. The sound of pixels. In Proceedings of the European conference on computer vision (ECCV), pages 570-586, 2018.
14

---

Appendices
A Additional Ablation Studies
Effectiveness of Number Cross-Attention Blocks. We conducted an ablation study with 16 cross coss-attenttion blocks. As shown in Table although using more cross-attention blocks will increase the computation, it helps to enhance the model performance.
Table 7: Effectiveness of Number of Cross- Attention Blocks.
Blocks
NYUv2
SUN RGBD
Acc.
mAcc.
mloU
Asc.
mAcc.
mIoU
17.3
63.8
52.3
79.6
60.3
48.1
78.1
65.3
54.1
84.4
12
81.5
71.6
59.2
83.9
67.2
54.1
16
83.1
75.1
61.7
85.4
68.7
55.6
Computational Cost. As shown in Table the parameters, GFLOPs, and inference time of our method competitive with prior methods. Meanwhile, achieved state-of-the-art performance on two segmentation benchmarks.
Table 8: The Comparision of Computational Cost.
Methed
XUUD2
mi00
SUN
aIDU
PARAMS
GFLOPS
Infereace
TobraFacom
162
110
159M
136
Gemisiooie
20
527
75884
174
155 ms
Attention Visualization. As shown in Figure 7 our Invertible Cross-Attention layer can capture the attention interaction from the region the depth image (red box) to the RGB image. This result has illustrated the effectiveness of our proposed attention layer in capturing the correlation across modalities.
Depth
Attention Interaction to RGB
Depth
Attention Interaction to RGB
Figure 7: The Attention Visualization of ICA Layer.
B Dicussion of Limitations
Our experiments have chosen set of learning hyper-parane apper-parameters and benchmarks to support our hypothesis. However, our work could contain several limitations. Our work studied the effectiveness of our proposed invertible cross-attention layers in multimodal learning. Thus, investigation of balance weights among learning objectives has not been fully exploited, and we leave this experiment as our future work. Due to computation limitations, our experiments are limited to the standard scale of the benchmarks. However, hypothesize that the proposed approaches generalize to larger-scale data and benchmark settings according to the fundamental thcories presented in our paper.
15

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The claims declared in the abstract match with the contributions, experimental results, and scope of the paper.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
The limitations of the paper are discussed in the appendix.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
16

---

Justification:
The description of the formula is provided in the paper.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: The details of datasets and implementations are presented in the experimental sections.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, No answer this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
17

---

Answer: [NA]
Justification:
The code will published may the paper be accepted.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips. cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
/
wwt.Subbmisiioooiiioooooooio for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data,
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification:
The details of training and testing are presented in the experimental section.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core the paper to a level detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined other appropriate information about the statistical significance of the experiments?
Question:
Answer: [NA]
Justification: Following the standard evaluation of semantic segmentation, image-to-imad to-igge translation, and classification, we evaluate our model by the standard mloU, accuracy, and MSE metrics.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emror bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
18

---

It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative еπхо rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: The computational resources used in our experiments are presented in the experimental section.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't make into the paper).
9. Code ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics
Answer: [Yes]
Justification:
The content of the paper and datasets strictly follows the NeurIPS Code of
Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (c.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification:
The paper does not have negative societal impact.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
19

---

The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: The paper does not have risk. The released models will be available may the paper accepted.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification:
The paper provides all the references code, data, and models used in the paper.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version the asset used and, possible, include URI
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g.. website). the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms use in package should be provided. For popular datasets, paperswithcode com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
20

---

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: The paper does not introduce the new dataset. The code of the paper will be published may the paper be accepted.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: The research in this paper does not involve crowdsourcing research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: The research in this paper does not involve crowdsourcing nor research with human subjects. Thus, there is no requirement for IRB.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
21

---

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology. scientific rigorousness, or originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification:
The paper does not utilize the LLMs.
Guidelines:
The answer NA means that the method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy battps://neurre http//rrppscccomofeeereeee/////p Cc/Coonferences/2225//LI. for what should or should be described.
22


Reviews:
[{"review_id": "qADHyzrWoi", "reviewer": "Reviewer_dFiC", "summary": "This paper proposes MANGO, a novel multimodal fusion framework that combines normalizing flows with a tailored Invertible Cross-Attention (ICA) mechanism. MANGO explicitly models the joint distribution of multimodal data through invertible transformations, enabling interpretable fusion and improved performance across tasks such as semantic segmentation, image-to-image translation, and movie genre classification. It introduces three novel partitioned attention mechanisms (MMCA, IMCA, LICA) and demonstrates SoTA results on several benchmark datasets.", "questions": "1.\tWhy have you not provided attention maps or saliency maps to show how the attention mechanisms in this paper capture cross-modality correlations, validating the interpretability claim?\n2.\tYou should clarify MANGO’s scalability and provide quantitative comparisons of computational costs against other similar and SoTA models.\n3.\tYou should enhance architectural diagrams and include visualizations to show better how different MANGO functions work", "limitations": "Yes.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n1.\tIntroduces a novel approach for multimodal fusion using Normalizing Flows and ICA, addressing key challenges in cross-modality correlation.\n2.\tStrong performance across multiple tasks (semantic segmentation, image-to-image translation, etc.).\n3.\tClear implementation details and experimental setups allow for easy reproduction of results.\nWeaknesses:\n1.\tNo formal proof regarding the correctness or convergence of ICA, which weakens the paper’s theoretical foundation.\n2.\tClaims of interpretability are not demonstrated with concrete visualizations or quantitative analyses, leaving the interpretability claim unsubstantiated.\n3.\tContradictory claims regarding scalability and lack of quantitative comparisons with other models.\n4.\tVisual aids are unclear, and more effective diagrams or illustrations would improve the understanding of the model.", "was_revised": true, "final_justification": "The authors have properly addressed my concerns on visualizations or quantitative analyses of interpretability and scalability by adding sufficient explanations and the required visualization diagrams in the revision and appendix. However, the proof of the correctness or convergence of ICA should have been formally added to the revision. I will raise my score accordingly.", "rebuttal": "Dear Reviewer dFiC,\n\n\nWe greatly appreciate your insightful review and valuable feedback. We are pleased you think **our proposed approach is novel and achieves the strong performance across benchmarks**. We appreciate your constructive comments and would like to address these points as follows.\n\n\n[Q1] **Formal proof regarding the correctness or convergence of ICA**\n\n\n[A1] Our proposed Invertible Cross-Attention (ICA) layer is theoretically grounded through its architectural design. In particular, as shown in Eqns. (4)–(6), ICA adopts an autoregressive formulation with an upper triangular attention matrix, ensuring invertibility due to the strictly positive diagonal entries of the softmax matrix. This structure ensures the existence of an inverse and facilitates the efficient computation of the Jacobian determinant, thereby meeting the tractability requirement of normalizing flows. Our approach follows established methods in flow-based models, such as RealNVP [1], AttnFlow [2], and Neural Autoregressive Flows [3], which similarly rely on structural properties rather than formal convergence proofs. Additionally, as in Table 6, we empirically demonstrate convergence and stability across tasks and increasing model depths. Our proposed invertible attention design is a promising direction for future work.\n\n\n\n\n[Q2] **Visualizations or quantitative analyses of interpretability**\n\n\n[A2] Our interpretability is theoretically supported through the explicit modeling of cross-modal dependencies via our normalizing flows with Invertible Cross-Attention (ICA) layers. Unlike prior implicit fusion models, ICA provides an explicit and tractable learning approach to modeling cross-modal interactions, as demonstrated in Eqns. (4)–(6), where the autoregressive formulation ensures invertibility and allows closed-form computation of the Jacobian determinant through the upper triangular structure of the attention matrix. This property enables us to trace and quantify the contribution of each modality to the final representation. Moreover, the partition-based design of MMCA, IMCA, and LICA enables modality-specific attention routing, providing a transparent mechanism for analyzing how each modality contributes to the fusion process. We further substantiate this by reporting performance gains through ablations (Tables 4–6), which quantitatively show how different attention configurations and layers affect model behavior. While figures are not permitted in the rebuttal, we refer the reviewer to Figure 1 in the appendix, which visualizes the attention maps and clearly illustrates how ICA captures modality-specific interactions (e.g., from depth to RGB). We will include additional visualization in our revised versions.\n\n\n\n\n[Q3] **Contradictory claims regarding scalability and lack of quantitative comparisons**\n\n\n[A3] We clarify that the scalability claim refers to both computational efficiency and the model’s ability to handle high-dimensional multimodal data. In particular, our approach introduces a multimodal latent normalizing flow (Section 3.3), which reduces input dimensionality via perceptual compression (Eqn. 13), thereby lowering the computational burden. This design enables the model to focus on semantic content rather than raw input redundancy. In addition, as shown in Table 2 in the appendix, we compare our model to TokenFusion and GeminiFusion in terms of parameter count, GFLOPs, and inference time. MANGO achieves higher mIoU (59.2% on NYUDv2) while maintaining comparable or better efficiency (e.g., 72.9M params vs. 75.8M in GeminiFusion, and 144 ms vs. 153 ms). It demonstrates that our approach not only scales well in practice.\n\n\n| Method           | NYUDv2 mIOU | SUN RGB-D mIOU | PARAMS | GFLOPS | Inference Time |\n|------------------|-------------|----------------|--------|--------|----------------|\n| TokenFusion      | 54.2        | 53.0           | 45.9M  | 108    | 126 ms         |\n| GeminiFusion     | 57.7        | 53.3           | 75.8M  | 174    | 153 ms         |\n| **MANGO**        | **59.2**    | **54.1**       | 72.9M  | 152    | 144 ms         |\n\n\n\n\n[Q4] **Visual aids are unclear**\n\n\n[A4] We thank the reviewer for the suggestion to improve the clarity of the visual aids. Due to the constraints of the rebuttal format, we are unable to include revised diagrams or illustrations in this response. In the revised version of the paper, we have revised and updated figures to improve readability. We also point the reviewer to Figure 1 in the appendix, which provides a concrete visualization of attention maps from the ICA layer, further supporting the interpretability and functional design of our model.\n\n\n[Q5] **Concerns about attention maps or saliency maps**\n\n\n[A5] While the main paper focuses on theoretical formulation and empirical results, we have provided qualitative evidence through attention visualizations in Figure 1 in the appendix, which illustrates how the Invertible Cross-Attention (ICA) layer captures cross-modal interactions between deep and RGB images. These visualizations support our claim that ICA explicitly models modality-aware dependencies.\n\n\n[Q6] **MANGO’s scalability and provide quantitative comparisons of computational costs**\n\n\n[A6] We provided a detailed comparison in Table 2 (Appendix) against recent SoTA models such as TokenFusion and GeminiFusion (Please refer to our answers in [A3]). MANGO achieves superior segmentation performance (e.g., 59.2% mIoU on NYUDv2) while maintaining comparable efficiency in inference time (144 ms), GFLOPs (152), and parameter count (72.9M). These results validate that MANGO scales effectively without compromising performance. We will make these points more prominent in the final version of the paper.\n\n\n[Q7] **Enhance architectural diagrams and include visualizations**\n\n\n[A7] We thank the reviewer for this helpful suggestion. We will enhance the architectural diagrams of MANGO in the final revised version.\n\n\n\n\n**References**\n\n\n[1] Dinh, et al. Density estimation using real nvp. ICLR, 2017.\n\n\n[2] Sukthanker et al. Generative flows with invertible attentions. CVPR 2022.\n\n\n[3] Huang et al. Neural autoregressive flows. ICML, 2018."}, {"review_id": "MhZy5QuXcu", "reviewer": "Reviewer_uDbX", "summary": "This paper introduces MANGO framework which is for multimodal fusion that leverages normalizing flows combined with an invertible cross-attention (ICA) mechanism.  MANGO incorporates three partitioning strategies (MMCA, IMCA, and LICA), with LICA introducing a learnable token permutation via LU decomposition. Experiments across semantic segmentation, image translation, and multimodal classification tasks demonstrate state-of-the-art performance.", "questions": "- Could the author provide a comparative analysis of computational cost? Like a table comparing MANGO to key baselines on metrics such as total parameters, training time, and/or inference speed on a standard input size.\n- Can MANGO handle generation tasks (e.g., image captioning or text generation), or is it limited to fusion-only applications?", "limitations": "Yes", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 4, "significance": 4, "originality": 4, "strengths_and_weaknesses": "Strengths\n- The paper is generally well-written, with a clear structure.\n- Experimental results are compelling, showing state-of-the-art performance across multiple tasks including segmentation, image translation, and classification.\n- The learnable partitioning strategy (LICA) using LU decomposition is an innovative way to address the rigidity of traditional flow partitions.\n    \nWeaknesses\n- No runtime or efficiency comparison is provided. While the latent model approach is designed to improve efficiency, the overall architecture is likely to be computationally expensive compared to simpler fusion methods. \n- Although the paper introduces a method to improve scalability, the authors themselves concede it remains a limitation. The complexity of the model might hinder its adoption in resource-constrained environments or for applications requiring very low latency inference.", "was_revised": true, "final_justification": "I recommend Accept. The paper's core contribution is novel and the state-of-the-art results are compelling.\n\nMy primary initial concern about the lack of computational cost analysis was fully resolved by the authors' rebuttal. They provided data demonstrating that their method is not only more accurate but also computationally competitive with other leading models.", "rebuttal": "Dear Reviewer uDbX, \n\nWe sincerely thank the reviewer for their thoughtful and constructive feedback. We are pleased that you found that **our paper is well-written with a clear structure**, and **we truly appreciate your recognition of innovative of LICA using lU decomposition and our strong experimental results**. We appreciate your constructive comments and would like to address these points as follows.\n\n\n\n\n[Q1] **Runtime Analysis**\n\n\n[A1] Our proposed approach is carefully designed with computational efficiency in mind. In particular, our multimodal latent normalizing flow approach (Section 3.3) projects high-dimensional inputs into a compact semantic space, allowing the model to avoid processing raw inputs directly and thereby reducing memory and computational cost. To address the concern about runtime, we have already provided quantitative comparisons in Table 2 (Appendix). MANGO achieves 59.2% mIoU on NYUDv2 while maintaining a competitive inference time of 144 ms, compared to 126 ms for TokenFusion and 153 ms for GeminiFusion. Similarly, MANGO uses 72.9M parameters and 152 GFLOPs, which is comparable to GeminiFusion (75.8M, 174 GFLOPs) despite offering higher performance. These results confirm that, while more expressive, MANGO remains practically efficient and scalable.\n\n\n| Method           | NYUDv2 mIOU | SUN RGB-D mIOU | PARAMS | GFLOPS | Inference Time |\n|------------------|-------------|----------------|--------|--------|----------------|\n| TokenFusion      | 54.2        | 53.0           | 45.9M  | 108    | 126 ms         |\n| GeminiFusion     | 57.7        | 53.3           | 75.8M  | 174    | 153 ms         |\n| **MANGO**        | **59.2**    | **54.1**       | 72.9M  | 152    | 144 ms         |\n\n\n[Q2] **The complexity of the model might hinder its adoption in resource-constrained environments or for applications requiring very low latency inference.**\n\n\n[A2] We have already discussed the limitations of our approach (Appendix Section B) related to scalability. This is a common challenge among expressive fusion models that aim to explicitly model high-order multimodal relationships. However, in our approach, we have taken concrete steps toward efficiency through the use of perceptual compression, which reduces input dimensionality. Invertible cross-attention layers ensure tractability while avoiding the need for dense fusion operations. As shown in Table 2 (appendix), MANGO achieves competitive inference time and computational cost (e.g., 144 ms, 152 GFLOPs) while outperforming SoTA models.\n\n\n[Q3] **Can MANGO handle generation tasks (e.g., image captioning or text generation), or is it limited to fusion-only applications?**\n\n\n[A3] We thank the reviewer for this insightful question. While MANGO is primarily designed for multimodal fusion tasks, its underlying architecture, based on invertible normalizing flows, is not inherently limited to fusion-only applications. In principle, the bijective mapping between input and latent spaces enables both inference and generation, provided appropriate task heads and decoders are used. However, this extension is beyond the scope of the current work and remains an exciting direction for future research. We leave this investigation to our future work."}, {"review_id": "uXOjceDwdO", "reviewer": "Reviewer_eadU", "summary": "The paper proposes a multimodal learning method using normalizing flows. Three fusion approaches are proposed to control the degree of modality mixing, which are placed in the model to progressively increase the mixing degree. Three applications are tested: semantic segmentation, image-to-image translation, and movie genre classification, and superiority in performance is shown.", "questions": "Please see the weaknesses.", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "* Strengths\n\n1. The writing is mostly fine.\n2. It seems that there are not many prior methods using normalizing flows for multimodal learning.\n\n* Weaknesses\n\n1. It is interesting to use invertible operations for multimodal learning, but it is not clear why invertibility is required/beneficial for multimodal learning. It seems the reverse direction is not used at all anyway.\n\n2. It is said that the existing approaches lack explicit modeling of inter-modal correlation. However, the proposed method relies on ICA, which is similar to the popular cross-modal attention except for the assignment of each modality to query, key, and value. So the modeling capability of the proposed method doesn't seem to go beyond the popular cross-modal attention. It is not clearly explained that except for the assignment, how the proposed method is distinguished, and which part of the proposed method allows explicit modeling unlike existing methods. \n\n3. It is expected to justify better the motivation of the operation in the proposed ICA. Basically, the attention scores are calculated from one modality (green in Fig. 3) and are applied to combine tokens of the other modality (orange in Fig. 3). How can this resolve the research question (i.e., how to explicit model correlation better than existing methods)?\n\n4. It seems that the authors intend to emphasize the universality of the proposed method through multiple applications. While the experimental results show better performance of the proposed method, it is not quite clear why the proposed method works well in different applications. For instance, semantic segmentation requires modeling interaction between depth and RGB, whose embeddings would have high spatial correlation, whereas in the movie genre classification, a pair of image and text would have almost no spatial relationship but more complicated semantic relationship. It would be better to both conceptually explain and analyze learned models/representations to understand how the proposed method allows flexible modeling of different inter-modal relationship.\n\n5. Discussion on how the proposed method is distinguished from existing methods is rather insufficient. In the introduction, the limitations of existing approaches are mentioned only briefly and roughly. The argument at the end of Sec. 2.1 about the limitations of existing approaches (unable to model long-range dependency and complex cross-modal correlation) should be supported better by explicitly demonstrating whether the proposed method can overcome those limitations and how. It is not quite clear whether the three applications of the experiments actually require long-range dependency and complex cross-modal correlation and whether the proposed method actually improves the performance specifically due to better modeling of long-range dependency and complex cross-modal correlation. \n\n6. In the three applications, further theoretical/experimental analysis on how the proposed method is distinguished from the compared methods. For instance, how different the proposed method is from the fusion approaches in GeminiFusion or BridgeTower. \n\n7. Overall, the novelty of the proposed method is rather unclear to me.", "was_revised": true, "final_justification": "The authors' rebuttal is appreciated. Some of my previous concerns are cleared, but some are still not. \n\nI more or less agree on the conceptual claim that the invertibility can maintain important inter-modal correlation, but still have difficulty in understanding how such a structure actually leads to improved task performance in various tasks. It would be better to provide some other evidence or deeper analysis (other than the final task performance) that can help readers understand more about whether/how the conceptual advantages of the proposed method actually work, such as what the model actually learned, how the model encoded long-range dependency, what kind important information is kept through the invertibility whereas conventional methods lose, how adaptively the model learns different types of inter-modal interactions appearing in different tasks, etc.\n(I guess the additional question by reviewer XZY1 is also in a similar context.)\n\nSince some concerns are cleared, I raise my rating to borderline accept.", "rebuttal": "Dear Reviewer eadU,\n\nWe greatly appreciate your insightful review and valuable feedback. We are very happy you think **our writing is good, and our normalizing flows for multimodal learning are distinct from prior methods**. We appreciate your constructive comments and would like to address these points as follows.\n\n[Q1] **Benefits of invertibility in multimodal learning**\n\n[A1] The use of invertible operations in MANGO is motivated by both theoretical and practical advantages in multimodal learning. Unlike prior Transformer-based models that rely on implicit attention mechanisms, our approach explicitly models the joint distribution of multimodal inputs via Normalizing Flows with Invertible Cross-Attention (ICA). This design enables MANGO to preserve modality-specific information and capture complex cross-modal dependencies in a mathematically principled and lossless manner. In contrast, Transformers are not invertible, which can lead to information loss, particularly in fine-grained features or subtle cross-modal correlations during feature aggregation. The invertibility of our proposed approach ensures a bijective, lossless mapping between the input and latent spaces, guaranteeing that no information is discarded. This is an important property when fusing heterogeneous modalities with varying signal strength, as it prevents dominant modalities from overwhelming weaker ones during fusion. Moreover, invertibility enables exact likelihood estimation, providing a tractable and interpretable training objective that standard fusion models often lack. As in Section 4, our proposed approach consistently outperforms Transformer-based fusion models across tasks, validating the effectiveness of our invertible formulation.\n\n[Q2] **How the proposed method is distinguished and allows explicit modeling unlike existing methods.**\n\n[A2] The role and functionality of our ICA layers within our framework are fundamentally different from prior methods. In standard Transformers, attention is used within an implicit modeling paradigm, where cross-modal correlations are learned in a latent space without an explicit probabilistic structure. In contrast, our method embeds ICA within a normalizing flow, which explicitly models the joint probability distribution of multimodal inputs (Eqn. 2). In our approach, ICA is not merely used to refine features, but to construct a bijective, invertible transformation that aligns with the theoretical foundations of normalizing flows. Through this formulation, each ICA layer contributes directly to the likelihood computation (Eqn. 6), enabling tractable, mathematically grounded modeling of inter-modal dependencies. Furthermore, the invertibility of ICA guarantees lossless transformation, allowing us to retain and analyze modality-specific contributions, which is not possible in standard attention modules. The ability to compute exact log-likelihoods and preserve modality-wise information flow is what makes our approach explicit, interpretable, and principled, distinguishing it from prior attention-based fusion methods.\n\n[Q3] **Motivation of the operation in the proposed ICA**\n\n[A3] Our motivation lies in the need for explicit, lossless modeling of cross-modal correlations, which existing Transformer-based fusion methods fail to offer. In prior Transformer-based methods, attention mechanisms operate implicitly, without modeling the joint probability distribution of the modalities. They tend to focus on contextual alignment, often overlooking subtle or global inter-modal dependencies, and do not explicitly preserve modality-specific information. Additionally, Transformer attention is not invertible, which can lead to information loss, especially of fine-grained correlations during feature fusion. In contrast, our ICA is designed within a normalizing flow approach, where each layer contributes to a bijective, mathematically grounded transformation. The directional design of ICA (e.g., attention scores computed from modality A and applied to modality B) allows us to explicitly model how one modality modulates the representation of the other, enabling principled and directional interaction that reflects true conditional dependencies. This operation is key to modeling the joint distribution of multimodal inputs, as shown in Eqns. (2)–(6), and is crucial for computing exact log-likelihoods, which the traditional attention mechanisms cannot achieve. Moreover, our approach introduces structured partitioning (MMCA, IMCA, LICA) to ensure that attention is not only directional but also aligned with specific modeling intents (modality-to-modality, inter-modality, or learnable interactions). The invertibility of ICA ensures lossless mapping between modalities and latent space, preserving all correlation structures. As in Section 4, our design yields consistent performance improvements across tasks, outperforming Transformer-based baselines.\n\n[Q4] **Universality of the proposed method through multiple applications**\n\n[A4] One of our goals is to demonstrate the universality and flexibility of MANGO across diverse multimodal tasks, each with distinct types of inter-modal relationships. The core strength of MANGO lies in its ability to explicitly model joint distributions between modalities via a tractable, invertible architecture. Unlike prior methods that use fixed attention mechanisms across all tasks, our design leverages (1) ICA layers, which allow the model to compute directional cross-modal influences in a lossless, learnable manner, and (2) adaptive partitioning strategies (MMCA, IMCA, LICA), which enable flexible structuring of attention flow depending on the modality pair. For example, in semantic segmentation, the spatial correlation between RGB and depth is naturally exploited via MMCA and IMCA; in contrast, for movie genre classification, where image and text have weaker spatial but richer semantic alignment, LICA enables the model to learn an optimal token permutation for aligning semantic patterns. Our multimodal latent normalizing flow design further supports this flexibility by operating in a compressed semantic space, where modality-specific and cross-modal features are jointly modeled. As shown in Tables 1–3, MANGO consistently improves performance across tasks with homogeneous and heterogeneous modality structures.\n\n[Q5] **Discussion on how the proposed method is distinguished from existing methods is rather insufficient.**\n\n[A5] Existing Transformer-based fusion models rely on implicit attention mechanisms that do not explicitly model joint distributions, and often struggle to capture long-range dependencies or complex inter-modal correlations, especially in settings with unbalanced modality strengths or limited supervision. Our method addresses these limitations by introducing normalizing flows with ICA. Specifically, the ICA layers use autoregressive attention matrices (Eqns. 4–6) to model cross-modal influences in a directional, lossless, and long-range manner. They are embedded in a bijective flow architecture that enables tractable modeling of the joint probability distribution across modalities. This design allows our model to capture both global and local dependencies in a mathematically grounded way. Regarding the experimental tasks: semantic segmentation benefits from long-range spatial dependencies between RGB and depth; image translation requires global cross-modal correlation to reconstruct coherent targets from complementary modalities; and movie genre classification depends on aligning complex semantic signals from visual and textual content. As shown in our experiments and ablations (Tables 1-6), our model consistently outperforms flow-based and Transformer-based baselines. \n\n[Q6] **Further theoretical/experimental analysis on how the proposed method is distinguished from the compared methods**\n\n[A6] Our approach is fundamentally different in both architectural design and modeling principles compared to GeminiFusion and BridgeTower. In particular, GeminiFusion performs early fusion and relies on self-attention layers to implicitly learn cross-modal correlations. Meanwhile, BridgeTower integrates vision and language encoders through interleaved attention blocks without modeling the joint distribution explicitly. In contrast, MANGO adopts an explicit modeling paradigm by using normalizing flows to estimate the joint probability distribution of multimodal inputs. Our Invertible Cross-Attention (ICA) layers contribute directly to tractable likelihood computation (Eqns. 2–6), and our partitioning strategies (MMCA, IMCA, LICA) offer structured, interpretable modeling of modality-specific and inter-modal dependencies. As shown in Tables 1-3, MANGO consistently outperforms GeminiFusion and BridgeTower by a large margin. Our results have validated that our explicit, invertible modeling framework brings measurable advantages in real-world multimodal tasks.\n\n[Q7] **The novelty of the proposed method is rather unclear to me.**\n\n[A7] We would like to emphasize that our method introduces a new class of explicit multimodal fusion models that combine normalizing flows with invertible attention mechanisms, which, to our knowledge, has not been explored in prior work. Unlike standard Transformer-based approaches (e.g., GeminiFusion, BridgeTower) that rely on implicit attention fusion, our method embeds a novel Invertible Cross-Attention (ICA) mechanism within a tractable normalizing flow framework. ICA is not only attention-based but also invertible and autoregressive, enabling exact joint likelihood modeling and lossless information preservation, a key theoretical and practical distinction. Our contributions are further supported by Reviewer dFiC, who acknowledges our approach as “novel ... addressing key challenges in cross-modality correlation”, and Reviewer XZY1, who agrees that “the proposed method from the perspective of explicit modeling is relatively novel.”"}, {"review_id": "1AedcO4eZV", "reviewer": "Reviewer_XZY1", "summary": "This paper identifies a key limitation in existing multimodal fusion approaches: leveraging Transformer-based attention mechanisms for fusion hinders the model's ability to effectively capture the complex structures and correlations inherent in multimodal data. To address this issue, the authors propose a method called Multimodal Attention-based Normalizing Flow (MANGO), which builds a multimodal modeling framework using Invertible Cross-Attention (ICA) layers. By introducing three types of cross-attention mechanisms, MANGO effectively captures the intricate structures and relationships within multimodal inputs. Experiments conducted on three different multimodal tasks validate the effectiveness of the proposed method.", "questions": "N/A", "limitations": "Yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths：\n\n1）The methodology and evaluation metrics used in the paper are appropriate.\n\n2）The authors propose a multimodal fusion method from the perspective of explicit modeling, which is relatively novel.\n\nWeaknesses：\n\n1）Although the authors point out in the Introduction that \"implicit modeling fails to fully capture the alignment and structural relationships across modalities,\" the discussion on the underlying causes and theoretical foundations of this phenomenon is relatively weak. The paper leans more toward phenomenological description, lacking rigorous mathematical definitions and theoretical explanations, which makes this concept feel more like an intuition rather than a well-grounded theoretical proposition.\n\n2）In Section 4.2 \"Semantic Segmentation,\" the authors claim that “Our model consistently outperforms the prior methods in all evaluation metrics and datasets.” However, upon reviewing the experimental results in Table 1, it is evident that the performance on the Pixel Accuracy metric for the SUN RGB-D dataset is not the best. Moreover, the visualizations in Figure 5 do not clearly demonstrate the advantages of the proposed method.\n\n3）In the Introduction, the authors mention that existing implicit multimodal fusion methods tend to overly rely on a single modality when it provides more data or stronger signals, leading to suboptimal performance. However, the paper does not include any experiments to verify whether MANGO effectively addresses this issue.", "was_revised": true, "final_justification": "The authors have well responsed the first two problems.\n\nFor the third problem, the author responded \"In the MM-IMDB classification task, our method outperforms strong baselines, which suggests that MANGO can effectively leverage both modalities without overfitting the stronger modality.\" However, in multimodal tasks, determining whether this issue has been resolved should NOT solely rely on the final performance, as this could be due to the performance boost from better leveraging the stronger modality. We believe the verification should proceed as follows: Since the author claims that existing methods have the above issue, the first step should be to present the accuracy of each modality in a multimodal model trained with the fusion approach of existing methods. Then, the accuracy of each modality in the multimodal model trained with the proposed method should be shown. If the accuracy of the weaker modality improves while the accuracy of the stronger modality does not decrease, it would strongly demonstrate the effectiveness of the proposed method.", "rebuttal": "Dear Reviewer XZY1, \n\nWe would like to express our gratitude for your careful reading and valuable feedback. We are glad you encourage that **our multimodal fusion method from the perspective of explicit modeling is novel**. We appreciate your constructive comments and would like to address these points as follows.\n\n\n\n\n[Q1] **Theoretical foundations of limitations of prior implicit modeling**\n\n\n[A1] The core limitation of Transformer-based fusion models lies in their implicit modeling nature, where attention mechanisms operate without a formal probabilistic framework. These models do not explicitly model the joint distribution of multimodal inputs; instead, they focus on contextual alignment, often overlooking subtle or global inter-modal dependencies and failing to preserve modality-specific information. In contrast, our method is grounded in the theory of normalizing flows, which explicitly models the joint distribution of multimodal inputs through invertible, bijective transformations with tractable likelihood computation. Our ICA layer is a mathematically defined transformation whose Jacobian determinant (Eqn. 6) directly contributes to the overall data likelihood (Eqn. 2), enabling exact and interpretable modeling of multimodal structure while guaranteeing lossless information propagation. Moreover, ICA’s autoregressive formulation introduces directed attention flows, allowing the model to efficiently capture cross-modal dependencies. These properties form a rigorous theoretical foundation for MANGO’s ability to model complex multimodal correlations, which are consistently validated through our experiments and ablation studies.\n\n\n\n\n[Q2] **Performance on the Pixel Accuracy metric for the SUN RGB-D dataset and Visualization of Figure 5**\n\n\n[A2] While it is true that our method does not achieve the highest score on Pixel Accuracy for the SUN RGB-D dataset, we would like to clarify that Mean Intersection-over-Union (mIoU) is the primary and widely accepted metric for evaluating semantic segmentation performance. Unlike pixel accuracy, which can be biased toward large or dominant classes and may overestimate performance due to background or majority-class pixels, mIoU provides a class-balanced evaluation by measuring the overlap between predicted and ground truth regions across all classes. This makes it a more reliable indicator of true segmentation quality, particularly in multimodal settings with varied object distributions. Our approach consistently achieves the highest mIoU across all datasets, which reflects its effectiveness in modeling cross-modal dependencies.\n\n\nRegarding the visualizations in Figure 5, we appreciate the reviewer’s feedback and acknowledge that the differences may not be immediately obvious due to the challenging nature of the NYUDv2 benchmarks, which contain complex indoor scenes, frequent occlusions, and subtle semantic boundaries. In the final version, we will enhance Figure 5 by adding highlight boxes to draw attention to regions where our method corrects boundary errors or recovers finer details, thereby making the qualitative advantages more interpretable.\n\n\n\n\n[Q3] **Existing implicit multimodal fusion methods tend to overly rely on a single modality**\n\n\n[A3] This challenge has been reflected in our experimental setup. In particular, the MM-IMDB classification task involves highly imbalanced modality signals, where text typically carries stronger semantic information than images. Indeed, as shown in the first two rows of Table 3, using text only yields a significantly higher macro-F1 score (47.2%) than using image only (25.3%), confirming that text carries a much stronger semantic signal in this benchmark. In this benchmark, our method outperforms strong baselines, i.e., BridgeTower and BLIP, by +3.5% micro-F1 and +4.9% macro-F1, indicating that MANGO is able to leverage both modalities effectively without overfitting to the stronger one. This suggests that our explicit modeling of the joint distribution via invertible attention within normalizing flows mitigates the issue of modality dominance by preserving complementary signals."}]

Author rebuttal:

Meta-review: The paper proposes a Multimodal Attention-based Normalizing Flow framework that explicitly models the joint distribution of multimodal inputs through a tractable and invertible attention-based network. The paper is clearly structured and easy to follow, and it presents strong performance across multiple tasks. The authors’ rebuttal successfully addressed some concerns regarding computational cost, as well as the clarity of interpretability and scalability. These revisions should be incorporated into the final version.
Decision: Accept (poster)