Extremely Simple Multimodal Outlier Synthesis for Out-of-Distribution Detection and Segmentation
Authors: Moru Liu
Hao Dong
Jessica Ivy Kelly
Olga Fink
Mario Trapp
Venue: NeurIPS 2025 poster ()
Keywords: Outlier Synthesis
Multimodal Learning
OOD Detection
OOD Segmentation
Primary area: applications
TL;DR: We introduce Feature Mixing, an extremely simple and fast method for synthesizing multimodal outliers with theoretical support.

Abstract: Out-of-distribution (OOD) detection and segmentation are crucial for deploying machine learning models in safety-critical applications such as autonomous driving and robot-assisted surgery. While prior research has primarily focused on unimodal image data, real-world applications are inherently multimodal, requiring the integration of multiple modalities for improved OOD detection. A key challenge is the lack of supervision signals from unknown data, leading to overconfident predictions on OOD samples. To address this challenge, we propose Feature Mixing, an extremely simple and fast method for synthesizing multimodal outliers with theoretical support, which can be further optimized to help the model better distinguish between in-distribution (ID) and OOD data. Feature Mixing is modality-agnostic and applicable to various modality combinations. Additionally, we introduce CARLA-OOD, a new multimodal dataset for OOD segmentation, featuring synthetic OOD objects across diverse scenes and weather conditions. Extensive experiments on SemanticKITTI, nuScenes, CARLA-OOD datasets, and the MultiOOD benchmark demonstrate that Feature Mixing achieves state-of-the-art performance with a $10 \times$ to $370 \times$ speedup. Our source code and dataset are available at https://github.com/mona4399/FeatureMixing.

Extremely Simple Multimodal Outlier Synthesis for Out-of-Distribution Detection and Segmentation
Moru Liu!*
Hao Dong²*
Jessica Kelly²
Olga Fink
Mario Trapp 1.3
'Technical University of Munich
2ETH Zürich FFaunhoferr IKS 4EPFL
Abstract
(OOD) detection and segmentation crucial for deploying machine learning models in safety-critica applications such as autonomous driving and robot-assisted surgery. While prior research has primarily focused unimodal image data, real-world applications are inherently multimodal, requiring the in- tegration of multiple modalities for improved OOD detection. A key challenge is the lack of supervision signals from unknown data, leading to overconfident predictions OOD samples. To address this challenge, propose Feature Mixing, an extremely simple and fast method for multimodal outlier synthesis with theoretical support, which can be further optimized to help the model bet- ter distinguish between in-distribution (ID) and OOD data. Feature Mixing is modnlity-agnostic and applicable various modality combinations. Additionally, we introduce CARLA-OOD. novel multimodal dataset for OOD segmentation, featuring synthetic OOD objects across diverse scenes and weather conditions. Ex- tensive experiments on SemanticKITTI, nuScenes, CARLA-OOD datasets, and the MultiOOD benchmark demonstrate that Feature Mixing achieves state-of-the-an performance with 10X to 370x speedup. Our source code and dataset will be available http://ggiubbb.ommmoo444444Faaa
Introduction
Classification and segmentation are fundamental computer vision tasks that have seen significant advancements with deep neural networks [22, 40]. However, most models operate under a closed-set assumption, expecting identical class distributions training and testing. In real-world applications, this assumption often fails, out-of-distribution (OOD) objects frequently appear. Ignoring OOD instances poses critical safety risks in domains like autonomous driving and robot-assisted surgery, motivating research on OOD detection [36] and segmentation [7] to identify unknown objects that are unseen during training.
Most existing OOD detection and segmentation methods focus on unimodal inputs, such as im- ages [36] point clouds [7], despite the inherently multimodal nature of real-world applica- tions. Leveraging multiple modalities can provide complementary information to improve per- formance [15, 14]. Recent work by Dong et al. [17] introduced the first multimodal OOD detection benchmark and framework and also extended the framework to the multimodal OOD segmentation task. key challenge OOD detection and segmentation is the tendency of neural networks assign high confidence scores to OOD inputs [44] due to the lack of explicit supervision for unknowns during training. While real outlier datasets [25] can help mitigate this, they are often costly and impractical to obtain. Alternatively, synthetic outliers [19, 48, 43] have been proven effective for regularization, but existing methods are designed for unimodal scenarios and struggle multimodal settings [17]. Dong
"gqual contribution.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

et al. [17] proposed multimodal outlier synthesis technique using nearest-neighhot information, but its computational cost remains prohibitive for segmentation tasks.
To address this, we propose Feature Mixing, an extremely simple and efficient multimodal outlier synthesis method with theoretical support. Given in-distribution (ID) features from two modalities, Feature Mixing randomly swaps subset of N feature dimensions between them to generate new multimodal outliers. By maximizing the entropy of these outliers during training, our method effectively reduces overconfidence and enhances the model's ability to distinguish OOD from ID samples. Feature Mixing is modality-agnose and applicable to various modality combina- tions, such as images and point clouds or video and optical flow. Moreover, its lightweight de- sign enables 10x speedup for multimodal OOD detection and a 370x speedup for seg- mentation compared to [17] (Fig. I).
0OD Seg Porformazzee (AUROC f)
91
370x
90
89-
88
Mixup
We conduct extensive evaluations across eight datasets and four modalities to validate the ef- fectiveness of Feature Mixing. For multimodal OOD detection, we use five datasets from the MultiOOD benchmark [17] with video and op- tical flow modalities. For multimodal OOD segmentation, we evaluate on large-scale real- world datasets, including SemanticKITTI [3] and nuScenes [6], with image and point cloud modalities. To address the lack of multi- modal OOD segmentation datasets, we intro- duce CARLA-OOD, a synthetic dataset gener- ated using CARLA simulator [18]. featuring di- verse OOD objects in various challenging scenes and weather conditions (Fig. 6). Our experiments on both synthetic and real-world datasets demonstrate that Feature Mixing outperforms existing outlier synthesis methods in most cases with significant speedup. In summary, the main contributions of this paper are:
+4.49
NF-Mix
87
Feature Mixing (ours)
Computational Cost of Outlier Synshesis per Sample (s) Figure I: Mixup [52] is efficient for outlier synthe- sis but performs poorly in OOD segmentation. In contrast, NP-Mix [17] achieves strong OOD seg- mentation but computationally expensive. Our Feature Mixing combines both speed and perfor- mance, benefiting from its simple yet effective de- sign. Results are on SemanticKITTI dataset.
1. We introduce Feature Mixing, an extremely simple and fast method for multimodal outlier synthesis, applicable to diverse modality combinations.
2. We provide theoretical insights in support of the efficacy of Feature Mixing.
3. We present the challenging CARLA-OOD dataset with diverse scenes and weather condi- tions, addressing the scarcity of multimodal OOD segmentation datasets.
4. We conduct extensive experiments across eight datasets and four modalities demonstrate the effectiveness of our proposed approach.
2 Related Work
2.1 Out-of-Distribution Delection
OOD detection aims detect test samples with semantic shift without losing the ID classification accuracy. Numerous OOD detection algorithms have been developed. Post hoc methods [24, 23, 36] aim to design OOD scores based on the classification output of neural networks, offering the advantage of being easy to use without modifying the training procedure and objective. Methods like Mahalanobis [32] and k-nearest neighbor [47] use distance metrics in feature space for OOD detection, while virtual-logit matching [50] integrates information from both feature and logit spaces to define the OOD score. Additionally, some approaches propose to synthesize outliers [19, 48) or normalize logits [51] to address prediction overconfidence by training-time regularization. However, all these approaches are designed for unimodal scenarios without accounting for the complementary nature of multiple modalities.

---

2.2 Out-of-Distribution Segmentation
OOD segmentation focuses on pixel- or point-level segmentation of OOD objects and has been widely studied in medical images [1], industrial inspection [46], and autonomous driving [5] in recent years. Approaches to pixel-level segmentation are generally categorized into uncertainty-
based [27, 49], outlier exposure [38, 31], and reconstruction-bast -based methods [2, 42]. Point-level segmentation has gained attention in recent years due to its practical applications in real-world environments. For example, Cen et al. [7] address OOD segmentation on LiDAR point cloud and train redundancy classifiers to segment unknown object points by simulating outliers through the random resizing of known classes. Li et al. [33] separate ID and OOD features using prototype- based clustering approach and employing a generative adversarial network [21] to synthesize outlier features. Similarly, these techniques are exclusively focused on unimodal contexts, neglecting the inherent complementarity among different modalities.
2.3 Multimodal OOD Detection and Segmentation
Multimodal OOD detection and segmentation are emerging research areas with limited prior work. Dong et al. [17] introduced the first multimodal OOD detection benchmark, identifying modality prediction discrepancy as a key indicator of OOD performance. They proposed the agres-toe algorithm to amplify this discrepancy during training and developed a multimodal outlier synthesis method that expands the feature space using nearest-neighbor class information. Their approach was later extended to multimodal OOD segmentation on SemanticKITTI [3]. More recently, Li et al. [34] introduced dynamic prototype updating, which adjusts class centers to account for intra-class variability in multimodal OOD detection. In this work, we propose a novel multimodal outlier synthesis method applicable to both OOD detection and segmentation tasks.
3 Methodology
3.1 Problem Setup
In this work, we focus on multimodal OOD detection and segmentation, where multiple modalities are involved to help the model better identify unknown objects. We define the problem setups for each task below.
Multimodal OOD Segmentation aims to accurately segment both ID and OOD objects in a point cloud using LiDAR and image data. Given a training set with classes y 2.., C), unlike traditional closed-set segmentation where test classes match training classes, OOD segmentation introduces unknown classes U {C + 1} in the test set. A paired LiDAR point cloud and RGB image can be represented as D = (P. y) where P = (PI, p2. Pw) denotes the LiDAR point cloud consisting of M points, with each point p represented by three coordinates and intensity p= (z. y, 2, i). Let X E 3 HWW represent the RGB image, where H and W denote height and width. The label y {y1, 122 ...: yM) provides semantic labels for each point, where y E y for the training data and y E UU for the test data.
Given model M trained under the closed-set assumption, with its outputs O = M(P, X)
R MxC
within the domain of y. During deployment, M should accurately classify known samples in y as ID and identify unknown samples in Uf as OOD. A separate score function S(p) is typically used as an OOD module to decide whether a sample point p E P is from ID or OOD:
ID
S(p) 2 1 S(p) <n
Gp)
(1)
OOD
where samples with higher scores S(p) are classified as ID and vice versa, and 7) is the threshold.
Multimodal OOD Detection aims to identify samples with semantic shifts in the test set using video and optical flow, where unknown classes are introduced. The setup similar to segmentation but differs in input and output types. The input consists of paired video V and optical flow F, represented as D = [V, yy, where y is the sample-level label rather than point-level label in segmentation. Similarly, the model produces sample-level outputs O MMV, F) RCC instead of point-level. The remaining setup follows that of segmentation, and we refer the reader to [17] for detailed definition.
3

---

14
OOD
20.0
OOD
12-
ID
17.5
ID
10
Deusity
Deasity
12.5
10.0
6
7.5
4-
5.0
2.5
0.0
0.0
0.2
0.4
0.6
0.8
0.2
0.3
0.4
U.5
0.6
0.7
0.8
Soore
Score
(a) Without Outlier Optimization
(b) With Outlier Optimization
Figure 2: (a) incertainty-basee OOD methods face overconfidence issues, resulting in significant overlap between the score distributions of ID and OOD samples. (b) After training with outlier optimization, the confidence scores for ID and OOD samples become more distinct, enabling the model better differentiate them. Results are on CARLA-OOD dataset.
3.2 Motivation for Outlier Synthesis
Uncertainty-based OOD detection and segmentation methods [24, 36, 37] are computationally efficient but suffer from overconfidence issues, illustrated in Fig. 2 (a). Outlier exposure- based methods [25, 38, 31] mitigate this issue by training models using auxiliary OOD datasets to calibrate the confidences of both ID and OOD samples. However, such datasets are often unavailable, especially in multimodal settings. To address this challenge, we introduce Feature Mixing, an extremely simple and efficient multimodal outlier synthesis method that operates in the feature space with negligible computational overhead (Sec. 3.3). These synthesized outliers are further optimized via entropy maximization, enhancing the model's ability distinguish ID from OOD data (Sec. 3.4). As shown in Fig. 2 (b), training with outlier optimization results in well-separated confidence scores, leading to improved OOD detection and segmentation.
3.3 Feature Mixing for Multimodal Outlier Synthesis
Existing Methods. Some prior works [49, 8] generate outliers in the pixel space by extracting OOD objects from external datasets and pasting them into inlier images. However, such methods are impractical for multimodal scenarios, where we need to generate outliers for paired multimodal data. Instead, generating outliers in the feature space is more effective and scalable. Mixup [52] interpolates features randomly selected samples to generate outliers but inadvertently introduces noise samples within the ID distribution (Fig. 4 (a)). VOS [19] samples outliers from low-likelihood regions
of the class-conditional feature dis- tribution but is designed for uni- modal settings and struggles with mul- timodal data. Moreover, it gener- ates outliers too close to ID sam- ples (Fig. (b)) and slow for high- dimensional features. NP-Mix [17] explores broader embedding spaces using nearest-neigg neighbo class informa- tion but remains computationally ex- pensive for segmentation tasks and in- troduces unwanted noise (Fig. 4 (c)).
Feature of Modality
Outlier Feature of Modality
2 5
3
6
Feature Mixing
2
2
Feature of Modality 2
Outlier Feature of Modality
v: randomly selceted feature for mixing Figure 3: Illustration of Feature Mixing.
Our Solution. To overcome these limitations, we propose Feature Mixing, an extremely simple yet effective approach that generates multimodal outliers directly in the feature space. Our method ensures that the synthesized features remain distinct from ID features (Theorem 1) while preserving semantic consistency (Theorem 2). Given ID features F [Fe; FiJ, where F is from modality and F; is from modality 2, Feature Mixing randomly selects subset of N feature dimensions from each modality and swaps them to obtain new features F. and F,, which are then concatenated form the

---

Dulles
Ourles
Oellis
Outtber
Mixup
VOS
NP-Mix
(d) Feature Mixing (ours)
Figure 4: I-SNE Visualization of multimodal outlier synthesis results the HMDB51 dataset. Our Feature Mixing excels at generating outlier samples by spanning wider embedding spaces without injecting noise extremely fast speed.
Seg. or Classifi. Head
Outlier Optimization
MOS
Fc
Sey.
0OD Modale
Extractor
ID Segmeniation
ODD Seymentation
Modality
Classiti: Head
OOD
Fi
or
Extractor
ID Classiiiattion
Detection
Modaliry
Concatenation
MOS
Multimodal Outlier Synthesis
Training only
Figure 5: Overview of the proposed framework that integrates Feature Mixing for multimodal OOD detection and segmentation.
multimodal outlier features F. = Foj Ft). Fig. and Algorithm provide detailed illustrations of the outlier synthesis process.
Algorithm 1 Feature Mixing
As shown in Fig. (d), Feature Mixing excels at generating multimodal outliers by covering broader embedding space without introducing noisy samples. The generated outliers exhibit two key properties: (1) These outliers share the same embedding space with the ID) features but lie in low-likelihood regions (Theorem 1). (2) Their deviation from ID features is bounded, preventing excessive shifts while maintaining diversity (Theorem 2). These properties ensure that the outliers align with real OOD charac- teristies and can be supported by the following theorems. Due to space limits, the proofs are provided in the Appendix.
Input: ID feature F | Fij. where F. is from channels. F from modality with N, channels; number selected feature dimen- sions for mixing N.
modality
Python-like Code:
select. random. wwwpllaraagllleeaaaa.a
selecl: random. wow.doom...mmlleerngg....
F. Fe.clome()
Fi.clone()
F. selecte, +, F, Fsleectt, 4 F; F. : = F. F. [selecte,, i,
F. = ([rc Fi), dim 0)
Output: Multimodal outlier feature F..
Theorem 1. Outliers F, synthesized by Feature Mixing lie in low-likelihood regions of the distribution of the ID features F, complying with the criterion for real outliers.
Theorem 2. Outliers F. are bounded in their deviation from F, such that FF. - -F22 S /2N
-F(5)
where = max;,j
Feature Mixing enables the online generation of multimodal outlier features and be seamlessly integrated into existing training pipelines (Sec. 3.4). Besides, its simplicity makes it modality-agnose allowing application diverse multimodal setups, such images and point clouds video and optical flow.
3.4 Framework for Outlier Optimization
Multimodal outlier features generated by Feature Mixing can be optimized using entropy maximiza- tion to help the model better distinguish between ID and OOD data, similar to outlier exposure- methods [25, 38, 31]. Fig. 5 illustrates our framework, which integrates Feature Mixing into multi-
5

---

modal OOD detection and segmentation. comprising two key components: Basic Multimodal Fusion and Multimodal Outlier Synthesis and Optimization.
Basic Multimodal Fusion. Our framework employs a dual-stream network to extract features from different modalities using separate backbones. For example, for multimodal OOD segmentation, features extracted from the image and point cloud backbones, denoted as F E IR RgN.xHXX and Fr XX respectively, are concatenated form the fused representation Ff. F3 contains both 2D and 3D scene information, which is then passed to a segmentation head for ID class segmentation. To enable efficient OOD segmentation inference, we append an uncertainty-base
OOD detection module to compute confidence scores for each prediction. This module supports various post- -ost-hoc OOD scoring methods [27, 24, 36], offering flexibility design choices. Furthermore, the simple late-fusion design facilitates the integration of ad advanced cross-modal training strategies [26, 17] and generalizes easily to other modalities and tasks.
Multimodal Outlier Synthesis and Optimization. To mitigate overconfidence in uncertainty-basee OOD detection, we incorporate outlier samples during training. These can be generated using existing methods such as Mixup [52], VOS [19], NP-Mix [17], or our proposed Feature Mixing. We then apply entropy-based optimization in Eq. (5) maximize the entropy of outlier features. In this way, we can better separate the confidence scores between ID and OOD samples (Fig. 2), thereby improving the model's 's ability distinguish OOD samples.
3.5 Training Strategy
Our training objective is to enhance OOD detection and segmentation while maintaining strong ID classification and segmentation performance.
Multimodal OOD Segmentation. Since accurate ID segmentation crucial for the effectiveness
of post-hoc OOD detection methods, optimizing ID segmentation loss [35] and Lovász-softmax loss [4], which are widely used in existing segmentation work [10, 53]. The focal loss foe addresses class imbalance by focusing on hard examples and is defined
priority. We employ focal
M C
foc
Σ
Σ
a.l fym c)FL(Om,.))
M m=
where FL(p) = -11 log(p) denotes the focal loss function and a is the weight w.r.t the c-th class. 1[-} is the indicator function. The Lovász-Softmax loss Liov directly optimizes the mean IoU and is expressed as:
Lior
Σ A { C
(3)
where
1 Om,c
if c ym- O m. otherwise.
mm (c)
(4)
AJ indicates the Lovász extension of the Jaccard index for class m(c) [o, 1IM indicates the vector of errors. For the generated multimodal outlier feature F., we obtain a prediction output ~
Ô E MMC using the segmentation head and aim to maximize the prediction entropy of the outlier features:
M Σ
Lent
Ô Öm,e log Öm
(5)
M
m= c=1
The final loss defined as:
C= == + Ltow + 7/1 Cent:
where 71 is weighting factor that balances the contributions each loss term.
Multimodal OOD Detection. The OOD detection loss combines classification loss with entropy regularization, which is defined
L L Lents
(7)
where the cross-entropy loss is used for Lete.

---

Experiments
We evaluate Feature Mixing across eight datasets and four modalities demonstrate its versatility. Specifically, we use nuScenes, SemanticKITTT, and our CARLA-OOD dataset for Multimodal OOD Segmentation using image and point cloud data. Additionally, we utilize five action recognition datasets from MultiOOD benchmark [17] for Multimodal OOD Detection, employing video and optical flow modalities.
4.1 Experimental Setup
Datasets and Settings. For multimodal OOD segmentation, we follow [17] to treat all ve- hicle classes as OOD the SemanticKITTI [3] and nuScenes [6] datasets. During training, the labels of OOD classes are set to void and ignored. During inference, we aim to segment ID classes with high Intersection over Union (IoU) while detecting OOD classes unknown.
We also introduce the CARLA-OOD dataset, created using the CARLA simulator [18], which includes RGB images, LiDAR point clouds, and 3D semantic segmentation ground truth, comprising a total of 245 samples. We select 34 anomalous objects as OOD, which are ran- domly positioned in front of the ego-vehicle across varied scenes and weather conditions, as shown in Fig. 6. Further details on CARLA- OOD are provided in the Appendix. The model is trained on the KITTI-CARLA [12] dataset with the same sensor setup and evaluated on CARLA-OOD with OOD objects. For multi- modal OOD detection, we use HMDB51 [30], UCF101 [45], Kinetics-600 [28], HAC [16], and EPIC-Kitchens [11] datasets from the Multi- OOD [17] benchmark. We evaluate using video and optical flow, where we train the model on one ID dataset and treat other datasets as OOD during testing.
RGB
Point
3D SS GT
Clear
Sunshine
Wel
Overcast
Figure 6: The proposed CARLA-OOD dataset for multimodal OOD segmentation. Points with red color are OOD objects.
Implementation Details. For multimodal OOD segmentation, our implementation follows [17] to build upon the fusion framework proposed PMF [53], utilizing ResNet-34 [22] as the camera backbone and SalsaNext [10] as the LiDAR backbone. After feature extraction and fusion, employ two 2D convolution layers as the segmentation head for segmentation. The OOD detection module uses MaxLogit [23] the default scoring function. For multimodal OOD detection, adopt the framework proposed in MultiOOD [17] and replace the multimodal outlier generation method with Feature Mixing. Additional implementation details are provided in the Appendix.
Evaluation Metrics. For OOD segmentation, we evaluate both closed-set and OOD segmentation performance at the point level. For closed- set evaluation, we use the mean Intersection over Union for known classes (mloU.). For OOD performance, we report the area under the receiver operating characteristic curve (AUROC), the area under the precision-recall curve (AUPR), and the false positive rate at 95% true positive rate @F5) For multimodal OOD detection, we report average accuracy (ACC) instead of mloU, for closed-set evaluation, well as AUROC and FPR @95 for OOD performance.
4.2
Main Results
4.2.1
Evaluation on Multimodal OOD Segmentation
We first evaluate our method on multimodal OOD segmentation. For baselines without outlier optimization, we consider basic Late Fusion, A2D [17] and xMUDA [26), with A2D being the state-of-the-art method. For baselines incorporating outlier optimization, we integrate Mixup [52]. NP-Mix [17], and our Feature Mixing into the framework. Due its inefficiency. VOS [19] is
7

---

SemanticKITTI FPR095) AUROC" AUPRI mloU,1 FPRG95) AUROCT AUPRT mloU, FPR AUROCT AUPR mloU,
nuSeenes
CARLA-OOD
Method
wlo Outlier Optimization
Late Fusion
53.43
86.98
46.02
61.43
47.55
26.42
76.79
57.24
20.56
61.84
XMUDA [26]
55.37
89.67
51.41
60.61
44.32
K3.47
20.20
78.79
97.00
57.86
10.35
65.15
A2D [17]
49.02
91.12
55.44
61.98
44,27
23.55
77.69
64.21
22.45
63.79
w/ Outtier Optimization
Mixup [52]
52.04
86.81
48.05
61.36
42.94
27.80
57.94
9.02
NP-Mix [17]
48.57
90.93
56.85
60.37
41.69
84.88
28. 54
76.16
41,81
88.45
29.68
62.56
Feature Mixing (ours)
92.98
33.37
XMUDA + FM (ours)
36.6.
91.54
53.89
60.43
39.49
85.29
28. 74
77.69
30.35
92.45
33.44
65.92
A2D (ours)
31.76
92.83
60.41
32.92
93.37
Table 1: Evaluation results on multimodal OOD segmentation datasets. FM: Feature Mixing.
OOD Datasets
Methods
Kinetics-600
UCF101
EPIC-Kitchens
HAC
Average FPR AUROCT
ID ACC
FPR AUROC" 6P95
AUROC" 6P5, AUROC1
FPR095] AUROCT
Baseline
32.95
44,93
87.95
97.70
32.95
29.73
92.60
87.23
Mixup [52]
25.31
94,10
36.37
90.49
14.37
96.40
22.57
94.85
24.67
93.96
86.89
[19]
31.70
93.22
38.77
89.93
15.39
96.
31.58
93.03
29.36
93.25
NPOS [48]
25.31
93.94
37.17
89.71
13,00
96.50
27.17
93.94
24.91
93.52
87.12
NP-Mix [17]
24.52
93.96
36.49
89.67
97.53
22.92
94.41
22.72
93.89
Feature Mixing (ours)
19.61
94.72
34.32
90.06
10.15
96.34
15.96
95.54
20.01
94.17
87.00
Table 2: Multimodal OOD Detection using video and optical flow, with HMDB51 as ID. Energy is
used as the OOD score.
excluded from the segmentation task. Additionally, we combine A2D and xMUDA with Feature Mixing to demonstrate its versatility.
As shown in Tab. 1, Late Fusion without outlier optimization suffers from overconfidence, leading to high FPR @95 values, indicating poor ID-OOD separation. While A2D improves performance in most cases, it remains suboptimal. Integrating outlier optimization yields significant improvements for both NP-Mix and Feature Mixing, underscoring the importance of outlier synthesis. On SemanticKITTI, Feature Mixing improves Late Fusion by 15.33% on FPR @95, 4.49% on AUROC, and 12.72% on AUPR. On nuScenes, Feature Mixing improves the Late Fusion baseline by 7.07% on FPR @95, 4.23% on AUROC, and 12.38% on AUPR. At the same time, Feature Mixing introduces a negligible negative impact on mloU, value.
Notably, all baselines without outlier optimization perform poorly on CARLA-OOD, with FPR@9: exceeding 97%. highlighting the dataset's 's difficulty and the overconfidence issue in uncertainty- based OOD methods. Feature Mixing significantly enhances Late Fusion on CARLA-OOD, reducing FPR @95 by 72.9%%, improving AUROC by 33.74%, and increasing AUPR by 12.81%. Furthermore, A2D + Feature Mixing achieves the best results in most cases, demonstrating our framework's adaptability advanced cross-modal training strategies.
4.2.2 Evaluation Multimodal OOD Detection
To assess the generalizability of Feature Mixing across tasks and modalities, we evaluate it on MultiOOD for multimodal OOD detection in action recognition, where video and optical flow serve as distinct modalities. We replace the outlier generation method in MultiOOD framework with Feature Mixing and compare it against Mixup [52], VOS [19], NPOS [48], and NP-Mix [17]. Models are trained on HMDB51 [30] or Kinetics-600 [39], and other datasets are treated as OOD during testing. As shown in Tab. 2, our Feature Mixing outperforms other outlier generation methods in most cases, achieving the lowest FPR @95 of 20.01 % and the highest AUROC of 94.17% on average when using HMDB51 1 as ID. Due to space limits, we put the results on Kinetics-600 in the Appendix. These results highlight the effectiveness of Feature Mixing in improving OOD detection across various tasks and modalities. Similarly, Feature Mixing introduces negligible impact on ID ACC.
4.3 Ablation Studies
Computational Cost. Tab. 3 compares the computational cost of different outlier synthesis methods. For OOD detection, the reported time corresponds to generating 2048 multimodal outlier samples of shape 4352. For OOD segmentation, it represents the time to synthesize x56 352 samples of shape 48.

---

RGB Image
Ground Truth
A2D
A2D + FM (ours)
Figure 7: Visualization results on different datasets. From top to down: SemanticKITTI, nuScenes, and CARLA-OOD. Points with red color OOD objects. Our method can segment OOD objects accurately. More visualization results are the Appendix.
While Mixup [52] is efficient, it performs poorly in OOD detection. In contrast, NP-Mix [17] achieves strong performance but is computationally expensive. Feature Mixing, benefiting from its simple design, both highly efficient and effective. Compared to NP- Mix, Feature Mixing provides 10x speedup for multimodal OOD detection and a 370x speedup for segmentation, making well-suited for real-world applications.
0OD
Detection OOD Segmentation
Mixup [52]
0.038
0.019
VOS [19]
152.05
61.49
NP-Mix [17]
0.545
4.81
Feature Mixing (ours)
0.058
0.013
Table 3: Computational cost of outlier synthe- sis methods (s).
Visualization. Fig. 7 presents the visualization of OOD segmentation results across different datasets, showcasing the RGB image, 3D semantic ground truth, and predictions from A2D and our best- performing A2D+FM model. The baseline method A2D struggles to identify OOD objects, whereas our method accurately segments OOD with minimal noise, demonstrating the effectiveness of the proposed framework. Additional visualizations can be found in the Appendix.
Hyperparameter Sensitivity. We evaluate the sensitivity of Feature Mixing to the hyperparameter N, using HMDB51 as ID dataset and Kinetics-600 as OOD dataset. Our findings, illustrated in Fig. demonstrate that Feature Mixing is ro- bust and consistently outperforms the baseline across all parameter settings.
Buslan
Bacia:
Mistng
97-
Mken
96
AUTOCC+
92
Feature Mixing in Tri- and Uni- modal Settings. To further demon- strate the generality of our method, we conduct additional experiments us- ing Feature Mixing (FM) in both tri- modal and unimodal settings.
924
Rastrro
Mixcing
Proturo Mikinng
Figure Ablation on the number N in Feature Mixing.
Tri-Modal Setting (Video Optical Flow + Audio). We extend FM to tri-modal setting on the EPIC-Kitchens dataset, where modalities include video, optical flow, and audio. FM is applied by randomly selecting subset of N feature dimensions from each modality and performing a cyclic swap: video + audio, audio > optical flow, and optical flow - vidco. This procedure synthesizes outlier features for all three modalities. As shown in Tab. 4a, FM consistently outperforms NP- Mix across all evaluation metrics, demonstrating robustness and versatility in complex multimodal scenarios.
Unimodal Setting (Video Only). FM is inherently modality-agnostic. operating purely in feature space without relying on input-specific assumptions. This makes it naturally extendable to unimodal scenarios. We implement a unimodal variant by splitting the video feature embedding into two halves and randomly swapping N dimensions between them, effectively synthesizing outliers within single

---

Method
FPR @ AUROCÍ ACC1
Method
FPR@95L AUROC1
ACC1
Baseline
69.22
72.39
73.13
Baseline
64.05
83.14
86.93
NP-Mix
62.69
74.95
71.46
NP-Mix
62.53
83.62
87.15
Feature Mixing
61.38
77.23
73.69
Feature Mixing
57.30
84.41
88.89
(a) EPICC-Ktchhes (Tri-modal)
(b) HMDB51 (Unimodal)
Table 4: OOD detection results with Feature Mixing in tri-modal and unimodal settings.
modality. Evaluated on the HMDB51 dataset, this approach outperforms the NP-Mix baseline across
all metrics, confirming FM's applicability even in the absence of multiple modalities (Tab. 4b).
Different Classes OOD. For multimodal OOD segmentation, we follow [17] and desig- nate all vehicle classes as OOD, Here, we ex- periment with different OOD category assign- ments: "ground" (road, sidewalk, parking, other- ground), "structure" (building, other-structure), and "nature" (vegetation, trunk, terrain). As shown in Tab. 5, Feature Mixing remains robust across all splits, consistently outperforming the
Method
FPR@95|
AUROC1 AUPR mloU.1
"ground" classes OOD
A27
71.15
74.92
69.13
66.57
A2D NP-Mix
53.60
94.71
95.30
65.07
A2D FM (ours)
36,30
93.89
96.04
65.88
"structure" classes as 0OD
A2D [17]
23.50
95,20
75.23
61.38
A2D NP-Mix Mix
22.14
95.41
76.36
60.54
A2D FM (ours)
18.05
96.09
79.85
61.79
"nature" classes 00D
A2D [17]
37.97
92.74
90.22
62.76
baseline by significant margin.
A2D NP-Mix A2D FM (ours)
30.88
98.18
92.51
61.21
20.60
96.13
94.08
62.67
Table 5: Ablation different classes as OOD on SemanticKITTI.
5 Conclusion
In this work, we introduce Feature Mixing, an extremely simple and fast method for multimodal outlier synthesis with theoretical support. Feature Mixing is modality-agnostic and applicable to various modality combinations. Moreover, its lightweight design achieves a 10x to 370x speedup over existing methods while maintaining strong OOD performance. To mitigate overeonfidence, outlier features are optimized via entropy maximization within our framework. Additionally, we present CARLA-OOD, a challenging multimodal dataset featuring synthetic OOD objects captured under diverse scenes and weather conditions. Extensive experiments across eight datasets and four modalities validate the versatility and effectiveness of Feature Mixing and our proposed framework.
Limitations and future work. Feature Mixing uses a random selection of feature dimensions to swap between modalities. While this is highly efficient and theoretically grounded, it may not always target the most informative feature regions for outlier synthesis. Future work could explore adaptive or learnable selection mechanisms that dynamically identify features that maximize OOD separability.
Societal impact. In our work, Feature Mixing advances multimodal OOD detection and segmentation by enabling more effective and efficient outlier exposure during training. This contributes directly to the reliability and safety of autonomous systems, including self-driving cars, by helping models handle unfamiliar or unexpected scenarios in open-world environments. Beyond autonomous driving, the method also has broader socictal impacts in other safety-critical domains such as healthcare and security, where robustness to unknown or anomalous data is crucial. By improving model reliability in the presence of distribution shifts, our approach supports the deployment of AI systems in dynamic, high-stakes environments where failure could have significant consequences.
Acknowledgments
The authors acknowledge that this work was supported by the Federal Ministry of Education and Research (BMBF) as part of LANNHEIM--AtODevSSaal was also funded by the Bavarian Ministry for Economic Affairs, Regional Development and Energy as part of a project to support the thematic development of the Institute for Cognitive Systems. The authors also acknowledge the support of "In-service diagnostics of the catenary/pantograph and wheelset axle systems through intelligent algorithms" (SENTINEL.) project, supported by the ETH Mobility Initiative.
(reference number 011S22087E).
10

---

References
[1] Bao, J., Sun, H., Deng, H., He, Y., Zhang, Z., Li, X.: Bmad: Benchmarks for medical anomaly detection. In: CVPR (2024)
[2] Baur, C., Wiestler, B., Albarqouni, S., Navab, N.: Deep autoencoding models for unsupervised
anomaly segmentation in brain mr images. arXiv preprint 1xXiv:1804.044886 (2018)
[3] Behley, J., Garbade, M., Milioto, A., Quenzel, J., Behnke, S., Stachniss, C., Gall, J.: Se- mantickitti: dataset for semantic scene understanding of lidar sequences. In: ICCV (2019)
[4] Berman, M., Triki, A.R., Blaschko, M.B.: The lovász-softmax loss: A tractable surrogate for the optimization of the intersection-over--iirr measure in neural networks, In: CVPR (2018)
[5] Blum, H., Sarlin, P.E., Nieto, J., Siegwart, R., Cadena, C.: The fishyscapes benchmark: Measur- ing blind spots in semantic segmentation. International Journal of Computer Vision 12)(11),
3119-318 (2021)
[6] Caesar, H., Bankiti, V., Lang, A.H., Vora, S., Liong, V.E., Xu, Q., Krishnan, A., Pan, Y., Baldan, G., Beijbom, O.: nuscenes: multimodal dataset for autonomous driving. In: CVPR (2020)
[7] Cen, J., Yun, P., Zhang, S., Cai, J., Luan, D., Tang, M., Liu, M., Yu Wang, M.: Open-world semantic segmentation for lidar point clouds. In: ECCV (2022)
[8] Choi, H., Jeong, H., Choi, Y.: Balanced energy regularization loss for out-of-distribution detection. In: CVPR (2023)
[9] Cordts, M., Omran, M.. Ramos, S., Rehfeld, T., Enzweiler, M., Benenson, R., Franke, U., Roth, S., Schiele, B.: The cityscapes dataset for semantic urban scene understanding. In: CVPR (2016)
[10] Cortinhal, T., Tzelepis, G., Aksoy, E.E.: Salsanext: Fast, uncertainty-aware semantic segmenta- tion of lidar point clouds for autonomous driving. arXiv preprint arXiv:2003.0365; (2020)
[11] Damen, D., Doughty, H., Farinella, G.M., Fidler, S., Furnari, A., Kazakos, E., Moltisanti, D.,
Munro, J., Perrett, T., Price, W., Wray, M.: Scaling egocentric vision: The epic-kitchens dataset. In: ECCV (2018)
[12] Deschaud, J.E.: KITTI-CARL: a KITTI-like dataset generated by CARLA Simulator. arXiv preprint Ntt::2109.00999 (2021)
[13] Dhamija, A.R., Günther, M., Boult, T.: Reducing network agnostophobia. In: NeurlPS (2018)
[14] Dong, H., Chatzi, E., Fink, O.: Towards multimodal open-set domain generalization and adaptation through self-supervision. arXiv preprint arXiv:2407.01s. 01151 (2024)
[15] Dong, H., Liu, M., Zhou, K., Chatzi, E., Kannala, J., Stachniss, C., Fink, O.: Advances in multimodal adaptation and generalization: From traditional approaches to foundation models. arXiv preprint 1ix::2501.18899 (2025)
[16] Dong, H., Nejjar, I., Sun, H., Chatzi, E., Fink, O.: SimMMDG: A simple and effective
framework for multi-modal domain generalization. In: NeurIPS (2023)
[17] Dong, H., Zhao, Y., Chatzi, E., Fink, O.: Multiood: Scaling -distii-dittiiitiiinn detection for multiple modalities. In: NeurIPS (2024)
[18] Dosovitskiy, A., Ros, G., Codevilla, F., Lopez, A., Koltun, V:: CARLA: An open urban driving simulator. In: CoRL (2017)
[19] Du, X., Wang. Z., Cai, M., Li, Y.: Vos: Learning what you don't 't know by virtual outlier synthesis. In: ICLR (2022)
[20] Feichtenhofer, C., Fan, H., Malik, J., He, K.: Slowfast networks for video recognition. In: ICCV (2019)
[21] Goodfellow. I., Pouget-Abadie, J., Mirza, M., Xu, B.. Warde-Farley, D., Ozair, S., Courville, A.. Bengio, Y.: Generative adversarial nets. In: NeurlPS (2014)
[22] He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: CVPR (2016)
[23] Hendrycks, D., Basart, S., Mazeika, M., Zou, A., Kwon, J., Mostajabi, M., Steinhardt, J., Song, D.: Scaling out-of-distribution detection for real-world settings. arXiv preprint 1IXiv:1911.11132 (2019)

---

[24]
Hendrycks, D., Gimpel, K.: A baseline for detecting misclassified and out-of-distribution examples in neural networks. arXiv preprint arXiv:1610.02136 (2016)
[25]
Hendrycks, D., Mazeika, M., Dietterich, T.: Deep anomaly detection with outlier exposure. arXiv preprint 1 1..44406 (2018)
[26] Jaritz, M., Vu, T.H., Charette, R.d., Wirbel, E., Pérez, P.: xmuda: Cross-modal unsupervised domain adaptation for 3d semantic segmentation. In: CVPR (2020)
[27] Jung, S., Lee. J., Gwak, D., Choi, S., Choo, J.: Standardized max logits: simple yet effective
approach for identifying unexpected road obstacles in urban-scene segmentation. In: ICCV (2021)
[28] Kay, W., Carreira, J., Simonyan, K., Zhang, B., Hillier, C., Vijayanaraiimhnn, S., Viola, F,, Green, T., Back, T., Natsev, P., et al.: The kinetics human action video dataset. arXiv preprint New::170.00995 (2017)
[29] Kingma, D.P.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.698 (2014)
[30] Kuehne, H., Jhuang, H., Garrote, E., Poggio, T., Serre, T.: Hmdb: large video database for human motion recognition. In: ICCV (2011)
[31] Lee, J., Oh, S.J., Yun, S., Choe, J., Kim, E., Yoon, S.: Weakly supervised semantic segmentation using out-of-distribution data. In: CVPR (2022)
[32] Lee, K., Lee, K., Lee, H., Shin, J.: A simple unified framework for detecting out-o--distribution samples and adversarial attacks. In: NeurlPS (2018)
[33] Li, J., Dong, Q.: Open-set semantic segmentation for point clouds via adversarial prototype framework. In: CVPR (2023)
[34] Li, S., Gong. H., Dong, H., Yang. T., Tu, Z., Zhao, Y:: Dpu: Dynamic prototype updating for multimodal out-of-distribution detection. arXiv preprint arXiv:241 1 1.08227 (2024)
[35] Lin, T.Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. arXiv preprint arXiv: 17xiiv:108..221 708.02202 (2017)
[36] Liu, W., Wang, X., Owens, J., Li, Y.: Energy-based aut-of-distributinn detection. In: NeurlPS (2020)
[37] Liu, X., Lochman, Y., Zach, C.: Gen: Pushing the limits of softmax-based out-of distribbbbtinn detection. In: CVPR. pp. 23946-29900 23955 (2023)
[38] Liu, Y., Ding, C., Tian, Y., Pang. G., Belagiannis, V., Reid, I., Carneiro, G.: Residual pattern
learning for pixel-wise cntoof-distribution detection in semantic segmentation. In: ICCV (2023)
[39] Long, F., Yao, T., Qiu, Z., Tian, X., Luo, J., Mei, T: Learning to localize actions from moments. In: ECCV (2020)
[40] Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. In: CVPR (2015)
[41]
Loshchilov, I., Hutter, F: Sgdr: Stochastic gradient descent with warm restarts. arXiv preprint arXiv:1608.00 009983 (2016)
[42] Masuda, M., Hachiuma, R., Fujii, R., Saito. H., Sekikawa, Y.: Toward unsupervised 3d point
cloud anomaly detection using variational autoencoder. In: ICIP (2021)
[43] Nejjar, I., Dong, H., Fink, O.: Recall and refine: A simple but effective source-free open-set domain adaptation framework. arXiv preprint 1.. (2024)
[44] Nguyen, A., Yosinski, J., Clune, J.: Deep neural networks are easily fooled: High confidence
predictions for unrecognizable images. In: CVPR (2015)
[45] Soomro, K.: Ucf101: dataset of 101 human actions classes from videos the wild, arXiv preprint arXiv: (2012)
[46] Sun, H., Cao, Y., Fink, O.: Cut: A controllable, universal, and training-free visual anomaly generation framework. arXiv preprint arXiv:2406.01078 (2024)
[47] Sun, Y., Ming, Y., Zhu, X., Li, Y.: Out-of-distributiion detection with deep nearest neighbors. ICML (2022)
12

---

[48] Tao, L., Du X., Zhu, X., Li, Y: Non Von-parametric outlier synthesis. arXiv preprint Stt1:2303.02966 (2023)
[49] Tian, Y., Liu, Y., Pang, G., Liu, F., Chen, Y., Carneiro, G.: Pixel-wise energy-biased abstention learning for anomaly segmentation on complex urban driving scenes. In: ECCV (2022)
[50] Wang, H., Li, Z., Feng, L.,, Zhang, W.: Vim: Out-of-distribution with virtual-logit matching. In:
CVPR (2022)
[51] Wei, H., Xie, R., Cheng, H., Feng. L., An, B., Li, Y.: Mitigating neural network overconfidence with logit normalization. In: ICML (2022)
[52] Zhang, H., Cisse, M., Dauphin, Y.N., Lopez-Paz, D.: mixup: Beyond empirical risk minimiza- tion. In: ICLR (2018)
[53] Zhuang, Z., Li, R., Jia, K., Wang, Q.. Li, Y., Tan, M.: Perception-awee aware multi-sensor fusion for 3d lidar semantic segmentation. In: ICCV (2021)
13

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
Our abstract and introduction clearly state the claims made.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results
be expected to generalize other settings.
is fine to include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
In the Appendix, we discussed limitations and future work.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low images are taken in low lighting. Or speech-to-text system might be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question:
For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
14

---

The paper provides the full set of assumptions and complete proof in the Appendix.
Justification:
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: Our paper discloses all the information needed to reproduce the main experi- mental results. Our source code and benchmark datasets will also be made publicly available upon acceptance.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer this question will not perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm.
(b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way access this model for reproducing the results way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reprodueibility may be tricky in some cases, in which authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path reproducing verifying the results.
5. Open access to data and code
15

---

Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: Our source code and benchmark datasets will also be made publicly available upon acceptance.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips. cc/
ceaninisssoooooooiii oeeiionooooooooo olloollio icy for more details.
While encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https: / //nips. cc/ wwcppbllc///aeeecceeeeceecceelee http//uudee//eeeee Sttps///ooSeumiieicc cessonnoolcyy for more details.
The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification:
We specify all the training and test details in the paper.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification:
We run all experiments three times and report the average value.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi-
dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for
example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
16

---

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
should be clear whether the error bar is the standard deviation or the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables figures symmetric error bars that would yield results that are out of range (c.g. negative emror rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification:
We provide details in implementation details part.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't 't make it into the paper).
9. Code ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https: : neurrps....pp// oc cc/public/ httpppulll//thhleeee htt.ttttiiisiiieleeeeeeeaaeeaaae
Answer: [Yes]
Justification: Our paper conforms, in every respect, with the NeurlPS Code of Ethics.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: There is no socictal impact of the work performed.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
17

---

The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility MI..
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: The paper poses no such risks.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Lieenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: They properly credited in our paper.
Guidelines:
The answer NA means that the paper does
existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com com/dataseta has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
18

---

If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification:
The new assets introduced in the paper are well documented.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/mmoel as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question:
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
16. Declaration of LLM usage
19

---

Question: Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods this research? Note that if the LLM used only for writing, editing, or formatting purposes and does not impact the core methodology. scientific rigorousness, or originality the research, declaration is not required.
Answer: [NA]
Justification: The core method development in this research does not involve LLMs any important, original, or non-standard components.
Guidelines:
The answer NA
that the method development in this research does involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy (https : h/tp://ies...//eeeeeeeeee http///rrs..c//ooeeeeeeeeeeeeeee / for what should or should be described.
20

---

ID
Otthers
Outlets
N 96
128
ID Outliers
ID Outlless
N 256
= 512
Figure 9: Feature Mixing introduces bias that shifts F, away from the ID distribution F, which is proportional to N. A large N promotes outliers to lie in low-likelihood regions of the ID feature distribution. Therefore, N can be adjusted to generate targeted outliers.
A Theoretical Insights for Feature Mixing
Problem Setup. Let F F,
E 2d be the concatenated in in-distrbbution (ID) features from two
modalities, where Fe- with mean μ. and covariance E., F; ~ Q with mean μ4 and covariance E. Me M4. The joint distribution of F has mean μ where Eg encodes cross-modal dependencies.
M and covariance Σ
Σ. Σd
Σ cl Σ
Feature Mixing swaps N features between F, and F, to generate perturbed features F. and F,, then concatenates them to form F.
which can written as:
F
F- F. 3 (1 M1) + F, O M1,
(8)
F = F, (1 M2) + + O M2,
(9)
where M1, M2 E {0, 1 is binary mask with N ones.
Theorem 1 Outliers F. synthesized by Feature Mixing lie in low-likelihood regions of the distribution of the ID features F, complying with the criterion for real outliers.
N Proof 1: After swapping N features, the feature of each modality has a a probability of being swapped from the other modality. Therefore, the mean of F is weighted average of μt and μr. and similarly for Fy:
N
N
F cl
He
d
ui:
(10)
N
N
F
+ d He
(11)
d
21

---

The perturbed mean po of F. becomes weighted combination of the original modality means:
F[
[11
Me - N m4
Ho F
R p4 + 14
(12)
F.
d
The deviation from the original mean becomes:
N
11 14e M 14
Aa=Ma-H - d
(13)
Since n # 141, Au & 0, introducing bias that shifts F. away from the IDD distribution proportional to N (Fig. 9) and [ Mzl. The Mahalanobis distance measures how far F, deviates from the mean Ia of the original joint distribution F, weighted by the inverse covariance --1
(2(F = (F. - -1)² " -14).
(14)
By defining F,- as (F.- 14)) + (1o 14), we get:
(F.) (F. no + AR) Σ IFF(F..F - 14 + Ap).
(Fo
(15)
After expanding the quadratic form, we get:
(F.).) = (F. (o) " (F. ()) + (Au)" D Apt AA
(16)
mo)
The first term captures the deviation of F. from its perturbed mean oo weighted by E E The second term the bias from the mean shift Au,, which grows with N and m - - The last term the cross-term that involves both perturbation noise and mean shift.
The original covariance E encodes intra- and cross-modal correlations. After swapping, Cov Cov(F.) becomes a mix of E. and Σ,, similarly for Cov(Fi). Besides, swapped features disrupt dependencies between F, and Fú, invalidating Eet- Therefore, the perturbed features F, have a new covariance structure E. E and this mismatch inflates the first term in Eq. (16). F.- 14, represents deviations under the perturbed distribution Σ., which are not aligned with the original covariance structure E. This misalignment causes E to assign incorrect weights to the deviations, leading to larger values in the quadratic form. Besides, the mean shift Au in the second term and the last cross-term can also lead to large values for D2 (FF... .). For Gaussian Gaussian-distribune distributed F, the likelihood of F, decays exponentially with (F.):
P(F.) exp 2 (-IDiTE))
(17)
Therefore, the inflated (²(F.) from covariance mismatch, mean shift, and cross-term forces p (F.) to be small, satisfying the low-likelihood criterion for outliers.
Here, we show mathematically that the expected Mahalanobis distance of F, exceeds that of ID samples F:
(FD)(F
)] > (F),
(18)
where (x) (x-xx) -xx-x))-='(x-- 14). For samples F, the squared Mahalanobis distance follows a chi-squared distribution with 2d degrees of freedom with expectation:
E[D²(F)) = EE [Tr (>=1(FF
- =n)²)]
(19)
=Tr = Tr((2s)= = 2d,
where Tr(-) is the trace of matrix. For outliers F.,. from Eq. (16) we know E[DD((F)) is the sum of the expectation of three terms. Since F, [o] = 0, the expectation of the last term is and:
E[D² (F.) = Tr( 'E.) + (M, -+)= -1 TI -1).
(20)
Let AΣ = E, - Σ, the trace term becomes:
Tr Tx)) Tr(E
= 2d + TT AE).
(21)
Now we prove that Feature Mixing ensures Tr( 0. Assume Σ is diagonal (without loss of generality via eigendecomposition) with Σ- = diag(o .......
.iag(o?....,,3,,, o²a). Feature Mixing increases variances
in dimensions with low of and decreases them in dimensions with high o/. Let:
AΣ diag( Azd), A 0 0,2
(22)
22

---

For of S o0. A, 2 0, and
A is large.
of
For of o.. A 0, and
is small in magnitude.
of
The trace becomes:
24
A
Tr
(23)
The positive terms dominate because Σ weights low-variance dimensions more heavily, Thus, Tr(E A 0. For example, if o? increases o = and of decreases o = where 0 < S b:
A b
a
a
(24)
(b a)² 2 0.
ab
Since Feature Mixing randomly swaps features from two modalities, the probability of increasing or decreasing of is the same, and therefore Tr(E - -AE) 2 0. The second term in Eq. (20) is strictly positive for H, # p.:
(Ho -1) (H4 11) 0. BID'(F.)) 2d Tr(E + (mo Σ = 11)
(25)
Combining all terms:
(26)
2d (FD2(F))
Since for Gaussian-distributed F, the likelihood of F. decays exponentially with D22 (F.).. p(F.) is much smaller than (F)) and therefore F, lie in low-likelihood regions.
Theorem 2 Outliers Fo are bounded in their deviation from F, such that FF, F/2 S /2N
where 8 maxi.
F
F(5)
Proof 2: While F, is statistically anomalous, it remains geometrically proximate to and is bounded in their deviation from F. The Euclidean distance between F, and F is:
F. F12
F.
F Fel2 + F, Frl..
(27)
For each modality, the deviation is bounded by the maximum feature difference 5
maxij F(O (
F Fel2 M @ (F: - Fe)12 S VN... 6, J (F. Fi)l2 S VN 8. F Fil2
(28)
(29)
Therefore:
F. 1 F S / (VN 8)2 + (VN 8)2 2N 6.
(30)
Since N & 22 6 remains small, ensuring F, stays near and preserves semantic consistency. In conclusion, both F, and F share the same embedding space, but F, lies in lowllikelhoood regions of the distribution of F.
B Additional Visualization Results
Fig. 10 to Fig. 12 present visualizations of multimodal OOD segmentation results across different datasets, showcasing the RGB image, 3D semantic ground truth, and predictions from various baselines. The baseline method struggles to identify OOD objects, whereas our method accurately segments OOD objects with minimal noise.
23

---

RGB Image
Ground Truth
A2D
A2D FM (ours)
Figure 10: More visualizations on SemanticKITTT dataset. Points with red color are OOD objects.
Our method accurately segments OOD objects, outperforming the baseline method.
RGB Image
Ground Truth
A2D
A2D FM (ours)
Figure 11: More visualizations on nuScenes dataset. Points with red color are OOD objects. Our method accurately segments OOD objects, outperforming the baseline method.
C More Details on Datasets
C.1 Realistic Datasets
We evaluate our approach on two widely-used autonomous driving datasets, including nuScenes [6] and SemanticKrTTi [3]. Both datasets provide paired LiDAR point cloud and RGB image, along with point-level semantic annotations. The nuScenes dataset contains 28, 130 training frames and 6, 019 validation frames, annotated with 16 semantic classes. SemanticKITTI consists of 21, 000 frames from sequences 00- 10 for training and validation, annotated with 19 semantic classes. Following [53, 7], we use sequence 08 for validation. The remaining sequences (00-07 and 09-10) are used for training. our OOD segmentation setting, we follow [17] map vehicle categories to a single unknown class to represent out-of diitr--diitiiittionn (OOD) objects in both datasets. Specifically, in
24

---

RGB Image
Ground Truth
A2D
A2D FM (ours)
Figure 12:
More visualizations on CARLA-OOD dataset. Points with red color are OOD objects. Our method accurately segments OOD objects, outperforming the baseline method.
SemanticKITTI, the categories (car, bicycle, motorcycle, truck, and other-vehicle| are remapped, while nuScenes, the remapped categories include /bicycle, bus, car, construction vehicle, trailer, truck, and motorcyclej. All other categories are retained as in-distribution (ID) classes and follow the standard segmentation settings. During training. we set the labels of OOD classes void and ignore them. During inference, we aim to segment the II) classes with high Intersection over Union (IoU) and detect OOD classes as unknown.
C.2 Synthetic Dataset
Limitations of the realistic datasets. The evaluation of multimodal OOD segmentation on datasets like nuScenes [6] and SemanticKITTI [3] often relies on manually remapping existing categories to simulate OOD classes. This methodology. while prevalent, suffers from two significant drawbacks. Firstly, the categories designated as OOD are often common objects (c.g., vehicles, ground, structures) that may not faithfully represent the characteristics of genuine, unseen anomalies encountered in real-world scenarios. Secondly, despite these designated OOD classes being nominally ignored during training (e.g., by excluding them from loss computation for known classes), the model is nevertheless exposed to these objects within the training data. This creates a substantial risk of data leakage, as the model may implicitly learn characteristics of these supposedly 'unseen' OOD classes.
Inspired by the development of existing 2D OOD segmentation benchmarks, where models are trained on Cityscapes [9] and tested on Fishyscapes [5] with the same class setup but additional synthetic OOD objects, we create the CARLA-OOD dataset for multimodal OOD segmentation task. We use the KITTI-CARLA dataset [12] train the base model. which is generated using the CARLA simulator [18] with paired LiDAR and camera data. The CARLA-OOD dataset aligns with the KITTI-CARLA sensor configurations but incorporates randomly placed OOD objects in diverse scenes and weather conditions for testing. The KITTI-CARLA dataset consists of sequences, each containing 000 frames captured from distinct CARLA maps and annotated with 22 classes for the LiDAR point cloud. We select 1,000 evenly sampled frames from each sequence, resulting in a total of 7, 000 frames for training and validation. The dataset split into a training set (Town01, Town03-Townd() and a validation set (Town02). with testing performed on our CARLA-OOD dataset. The CARLA-OOD dataset consists of 245 paired LiDDAR and camera samples captured across 5 CARLA maps and 6 weather conditions, each sample containing at least one OOD object.
25

---

Cartags
Box
Plantpot
Phire
contaiiee
Glask contaliner
Doghouse
Slide
barrier
Street
Trush can
greet sign
Table
Plastic
table
Figure 13: Example of OOD objects in our CARLA-OOS dataset.
Sensor
Position (x, z) [meters]
Configurations
LiDAR
1.80)
Channels: 64
Range: 80.0 meters
Upper FOV: degrees
Lower FOV: -24.8 degrees
RGB Camera (0.30, 0, 1.70)
FOV: 72 dogrees
Semantic Camera (0.30, 1.70)
FOV: 72 degrocs
Table 6: Sensor configurations for CARLA-OOD dataset.
To avoid class overlap with KITTI-CARLA, 34 OOD objects are selected and randomly placed within the scenes during dataset generation. The dataset is annotated with 22 classes aligned with KITTI-CARLA, along with an additional unknown class for OOD objects.
C,3 Generation of CARLA-OOD Dataset
The CARLA-OOD dataset created using the CARLA simulator, with a sensor setup aligned to the KITTI-CARLA dataset, consisting of camera and LiDAR the ego-vehicle. Detailed sensor configurations are provided in Tab. 6, with positions defined relative to the ego-vehicle. Thirty-four obstacles from CARLA's 's dynamic and static classes are randomly placed in front of the ego-vehicle varying distances as OOD objects (Fig. 13). The simulation spans diverse scenes (Town01, Town02, Town04, Town05, Town10) 10) and weather conditions (e.g., clear, wet, foggy, sunshine, overcast), capturing both semantic and covariate shifts. The dataset includes RGB images with resolution of 1392 1024 pixels, LiDAR point cloud, point-level semantic labels, and transformation matrices between sensors.
C.4 MultiOOD benchmark
In-Distribution
Kinetics-600
MultiOOD [17] is the first benchmark designed for Multimodal OOD Detection, comprising five action
recognition datasets (EPIC-Kitchens [11], HAC [16]. HMDB51 [30], UCF101 [45], and Kinetics-600 [28]) with over 85, 000 video clips. where video, optical flow, and audio are used different types of modali- ties. Fig. 14 shows an example of the Far-OOD setup in MultiOOD. This setup considers an entire dataset as in-disrrbuttin (ID) and further collects datasets, which comprise similar tasks but are disconnected from any ID categories, as OOD datasets. In this scenario, both semantic and domain shifts are present between the ID and OOD samples. We follow the
Dut-of-Distriuution
EPIC-Kitchems
HAC
Figure 14: Multimodal Far-OOD setup in MultiOOD, where Kinetics-600 is the ID dataset and the other four datasets are OOD.
26

---

same setup and framework as proposed in MultiOOD for experiments. More details on the MultiOOD benchmark are [17].
D Implementation Details
For the Multimodal OOD Segmentation task, we follow [17] to adopt the fusion framework from PMF [53], modifying it by adding an additional segmentation head to the combined features from the camera and LiDAR streams. We use ResNet-34 [22] as the backbone for the camera stream and SalsaNext [10] for the LiDAR stream. For optimization, we use SGD with Nesterov [41] for the camera stream and Adam [29] for the LiDAR stream. The networks are trained for 50 epochs with batch size of 4, starting with a learning rate of 0.0005 and with cosine schedule. To prevent overfitting, we apply various data augmentation techniques, including random horizontal flipping. random scaling, color jitter, 2D random rotation, and random cropping. For hyperparameters, set N in Feature Mixing to 10 and "/1 in loss to 3.0. For A2D, we set 2/2 to 1.0. For xMUDA, we 2/2 to 0.5. For the Multimodal OOD Detection task, conduct experiments across video and optical flow modalities using the MultiOOD benchmark [17]. We use the SlowFast network [20] to encode video data and the SlowFast network's 's slow-only pathway for optical flow. The models
pre-trained on each dataset's 's training set using standard cross-entropy loss. The Adam optimizer [29] employed with a learning rate of 0.0001 and batch size of 16. For hyperparameters, we N in Feature Mixing to 512.
E Compatible with Cross-modal Training Techniques
Our proposed framework not only supports the basic late-fusion strategy, but is also compatible with advanced cross-modal training techniques that promote interaction across modalities. To demonstrate its versatility, we show how to integrate A2D [17] and xMUDA [26] into the framework.
(A2D), designed for multimodal OOD detection, aims to amplify the modality prediction discrepancy during training. It assumes additional outputs O and O' from each modality. By removing the e-th value from O and O', A2D derives new prediction probabilities without ground-truth classes, denoted O and O' E (C-1)), A2D then seeks to maximize the discrepancy between O and O', which defined as:
M Σ D(O m. O,))
1
AAD
(31)
M m=1
where D)-) is distance metric quantifying the similarity between two probability distributions. By integrating A2D into the framework, the final loss function becomes:
L = L + Ltow + 7 Lent + 72C
(32)
XMUDA facilitates cross moosl learning by encouraging information exchange between modalities, allowing them to learn from each other. xMUDA also assumes additional outputs Of and Of from each modality and define cross-modal loss as:
LiM = DKL (OL((0110) + Dxz(O'110)
(33)
where Dr.. is the Hollback-Leibbee divergence and the final loss function in this case is:
L = L + Liov + 7 Lent + 726xM-
(34)
F More Ablation Studies
Hyperparameter Sensitivity. We evaluate the sensitivity of 71 in the loss function on the Se- manticKITTI dataset. Our findings, as illustrated in Fig. 15, demonstrate that training with multi- modal outlier generation and optimization consistently outperforms the baseline across all parameter settings. These ablations suggest that our approach is robust and exhibits minimal sensitivity to variations in hyperparameter choices.
27

---

60
95
w/o Outlier Optimasatoo Outler Optimiaacio
w/o Optimizerio::
91
Outhee Optimitation
55
93-
92
FP11085
50
AUROC+
91-
90-
45
89-
40
35
0.1
0.5
1.0
3.0
0.1
0.5
1.0
3.0
Figure 15: Ablation of 71 in the loss function on SemanticKITTI.
OOD Scores
FPR@95
AUROC1 AUPR
MaxLogit [23]
31.76
92.83
61.99
MSP [24]
32.93
91.57
51.68
Energy [36]
32.05
92.87
64.87
Entropy [13]
33.00
92.50
59.11
GEN [37]
32.63
93.00
64.43
Table 7: Ablation of OOD Scores SemanticKITTI.
Impact of Various OOD Scores. We evaluate the impact of different commonly used OOD scores by replacing the OOD detection module in our framework with MaxLogit [23] (our default), MSP [24], Energy [36], Entropy [13], and GEN [37]. As shown in Appendix F, the #95 and AUROC show minimal fluctuation (less than 2%) across different OOD scores, further demonstrating the adaptability of our framework to various design choices.
Multimodal OOD Detection Results on Kinetics-600. Tab. presents the multimodal OOD detection results using Kinetics-600 the ID dataset. Feature Mixing outperforms other outlier generation methods in most cases, achieving the lowest FPR of 54.75% and the highest AUROC of 79.23% on average when using HMDB51 ID. These results further demonstrate the effectiveness of Feature Mixing in improving OOD detection across diverse tasks and modalities, while maintaining negligible impact on ID accuracy.
Feature Mixing in Unimodal Settings for OOD Segmentation. We further extend the unimodal FM variant to the OOD segmentation task using LiDAR-only input. The same strategy used for unimodal classification is directly applied to LiDAR point cloud features. As shown in Tab. 9a, our method achieves competitive performance compared to NP-Mix and baseline methods, demonstrating the scalability of FM to segmentation tasks in unimodal settings with minimal modification.
Sensitivity of Parameter N on SemanticKITTI. We further investigate the sensitivity of FM to the mixing parameter N (i.e., the number of swapped feature dimensions) on the SemanticKITTI dataset. As shown in Tab. 9b, FM achieves stable performance across different values of N, supporting the robustness of the method. We use N=10 by default in our main experiments.
G Further Discussions on Feature Mixing
Challenges of extending existing solutions to multimodal setting. While methods such Mixup and VOS have shown success in unimodal tasks, directly applying them to multimodal data is non-trivial due to the following key challenges:
(1) Modality heterogeneity: Multimodal data often involves inputs with fundamentally different structures and distributions-er. e.g. dense 2D images, sparse 3D point clouds, temporal audio signals, etc. Pixel-level interpolation techniques like Mixup are inherently incompatible across heterogeneous modalities, making it difficult to define meaningful cross-modal augmentations. Morcover, Mixup's
28

---

OOD Dutasels
Methods
HMDBSI
UCFI0I
EPIC-Kitchens
HAC
Averge
ID ACC
FPR
AUROCY FPR@95|
AUROCT FPR@95, AUROCY FPR AUROCY FPR AUROCT
Baseline
72.64
71.75
70.12
71.49
43.66
82,05
61.50
74.99
61.98
75.07
73.14
Mixup [52)
66.53
70.33
68.36
69.53
39.68
86.62
56.35
77.95
57.73
76.11
73.40
VOS [19]
65.23
71.48
68.29
73.97
38.12
86.05
56.11
79.02
56.94
77.63
73.07
NPOS [48]
65.72
70.93
68.29
73.97
35.13
86.78
55.89
80,49
56.26
78.04
73,49
NP.Mix [17]
63.27
74.17
67.20
74.50
34.07
87.49
56.09
80.20
55.31
79.09
73.67
Feature Mixing (owns)
62.86
74.32
67.74
74.38
33.51
87.64
54,89
80.58
54.75
79.23
73.67
Table Multimodal OOD Detection using video and optical flow, with Kinetics-600 as ID, Energy
is used as the OOD score.
Method
FPR @95., AUROCA
AUPR mloUt
N
FPR 9PR
AUROC1
AUPR mloUt
Baseline
47.03
85.82
36.06
59.81
38.58
91.14
53.45
61.59
NP-Mix
48.26
86.79
45.73
59.52
10
38.10
91.47
58.74
61.18
Feature Mixing
46.40
88.81
47.67
60.04
12
37.17
90.90
55.19
60.75
(a) Unimodal feature mixing for OOD segmentation on Se-
(b) Ablation of N for OOD segmentation on SemanticKITTL. Table 9: More ablations on the OOD segmentation task.
ManticKITTI (LiDAR only).
random linear blending can produce ambiguous or noisy samples that may lie near or within the in-distribution (ID) manifold, reducing its effectiveness for OOD detection.
(2) Computational inefficiency: Methods like VOS and NP-Mix rely on distribution estimation, sampling, or searching in high-dimensional feature spaces, which becomes computationally expensive when extended to multimodal settings- for dense prediction tasks such as segmentation, where per-pixel per-point point processing is required.
Our Feature Mixing addresses these challenges by (i) operating entirely in the feature space, where modality-speciiff structures have already been abstracted, (ii) introducing a lightweight, swap-based perturbation strategy to synthesize outliers efficiently, avoiding costly sampling or searching, (iii) being theoretically grounded (Theorems and 2) and empirically validated to maintain ID-OOD separation while improving efficiency and scalability.
Clarification on the properties of Feature Mixing and their benefit to OOD detection. Theoretical results in Theorems and 2 establish two key properties of Feature Mixing: (I) the synthesized outliers have low likelihood under the ID distribution, and (2) their deviation from the ID distribution bounded. These properties are important for OOD detection and segmentation, as they ensure that FM produces challenging but plausible outliers that populate the low-density regions near the boundary of the ID distribution. This supports effective decision boundary regularization and reduces model overconfidence on unseen data. As shown in Fig. 4, the FM-generated outliers are well- separated from the ID) features in the embedding space. This separation helps the model to better distinguish between in-distribution and nut-of-distributio regions. These visualizations provide intuitive, empirical evidence that our method generates meaningful and bounded outliers, as predicted by our theoretical analysis.
More details on outlier optimization. The core idea is that the feature mixing branch generates pseudo-OOD features. We apply entropy maximization loss in Eq. (5) on these features to explicitly encourage the model to produce uncertain (i.e., high-entropy) predictions for them. This discourages the model from making overconfident predictions on ambiguous or unfamiliar that characteristic of OOD samples. contrast, the classification loss encourages low-entropy (i.e., congh-onfidence predictions on in-distribution (ID) samples. Together, these complementary objectives train the model develop confidence- based decision boundary: ID samples are associated with confident predictions, while pseudo- OOD samples samples-introduee introduced through feature mixing are explicitly pushed toward uncertain predictions. As shown Fig. 2, this training objective increases the separation in predictive confidence between ID and OOD inputs, which is crucial for effective OOD detection.
behavior
29


Reviews:
[{"review_id": "dTWvpniv8F", "reviewer": "Reviewer_an3N", "summary": "This paper addresses the problem of outlier synthesis for out-of-distribution (OOD) detection. The key observation is that neural networks often produce high-confidence predictions for OOD samples, and outlier synthesis can help mitigate this overconfidence. The authors point out that most existing outlier synthesis methods are designed for unimodal settings and have limitations when applied to multimodal data. To address this, the paper proposes a new feature mixing strategy tailored for the multimodal setting. Specifically, given in-distribution features from two modalities, the method randomly swaps a subset of N feature dimensions between them to generate synthetic multimodal outliers. Experimental results demonstrate both the efficiency and effectiveness of the proposed approach.", "questions": "1. In line 150, the authors state that Mixup may generate noisy samples that still lie within the ID distribution. Further clarification is needed to support the claim that the proposed method avoids this limitation. While visualizing features directly may be challenging, some form of intuitive or empirical analysis would strengthen the argument. \n2. What are the potential challenges or limitations in directly extending Mixup or VOS to multimodal settings? A brief discussion would help highlight the motivation for the proposed approach.\n3. How does the proposed feature mixing strategy differ from the dynamic mix-and-match perturbation introduced in “A Stochastic Conditioning Scheme for Diverse Human Motion Prediction” (CVPR 2020)? A comparison would help clarify the novelty of the method.\n4. Given that the framework operates in feature space, it would be valuable to include a decoder or reconstruction pipeline that maps the mixed features back to image space. This could provide further interpretability and insight into the nature of the synthesized outliers.", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths:\n1. The proposed multimodal feature mixing strategy for outlier synthesis is intuitive and effective. By operating in the feature space rather than the pixel space, it enables efficient generation of synthetic multimodal outliers.\n\nWeaknesses:\n1. While the feature mixing approach demonstrates good empirical performance, its effectiveness across different modality types is not thoroughly analyzed. A deeper investigation into how the method behaves with varying modality combinations would help better assess its generalizability and contribution.\n2. The use of entropy maximization for OOD calibration is not novel. For example, similar ideas have been explored in \"Entropy Maximization and Meta Classification for Out-of-Distribution Detection in Semantic Segmentation\" (ICCV 2021). A clearer positioning of the proposed method relative to such prior work would strengthen the paper.\n3. The challenges of extending existing unimodal feature mixing solutions to multimodal setting are not clearly explained.", "was_revised": true, "final_justification": "This paper addresses the problem of outlier synthesis for out-of-distribution (OOD) detection. My initial concerns were its relation to existing work and the challenges of extending from the unimodal to the multimodal setting. The authors have provided extensive evidence addressing these points, and I therefore raise my rating to a borderline accept.", "rebuttal": "Thanks for your insightful reviews and valuable comments on our paper! We provide the responses to your questions as follows:\n>**Q1**: how the method behaves with varying modality combinations\n\n**A1**: Thank you for raising this important point. We agree that analyzing the method's behavior across diverse modality combinations is crucial for assessing its generalizability. In our current experiments, **we already evaluated Feature Mixing (FM) on substantially different modality pairs**, such as point cloud + image for multimodal OOD segmentation (Table 1), as well as video + optical flow for multimodal OOD detection (Table 2). To further demonstrate the generality, we conducted an additional experiment on a **three-modality setting involving video, optical flow, and audio** for OOD detection on EPIC-Kitchens dataset. We extend FM to this setting by randomly selecting a subset of N feature dimensions from each modality and swapping them cyclically: video->audio, audio->optical flow, and optical flow->video, thus creating outlier features for all three modalities. As shown below, FM consistently outperforms NP-Mix across all metrics in this three-modality setting. These results further support the robustness and versatility of our method. \n\n||FPR95$\\downarrow$|AUROC$\\uparrow$|ACC$\\uparrow$|\n|-|-|-|-|\n|Baseline |69.22| 72.39| 73.13|\n|NP-Mix|62.69| 74.95 |71.46|\n|Feature Mixing|61.38 |77.23 |73.69|\n\nImportantly, FM is inherently **modality-agnostic**: it operates in the feature space without relying on assumptions about the input structure, enabling straightforward application to diverse and heterogeneous modality combinations. We further extend FM to the **single-modality setting** by randomly selecting N dimensions from the first and second halves of the feature embedding and swapping them within the same modality. This enables outlier synthesis without relying on a second modality. We evaluate this unimodal variant on OOD detection using HMDB51 (with video modality). As shown below, our unimodal FM achieves strong performance, outperforming the best NP-Mix baseline, which validates the applicability of FM beyond multimodal settings.\n||FPR95$\\downarrow$|AUROC$\\uparrow$|ACC$\\uparrow$|\n|-|-|-|-|\n|Baseline |64.05| 83.14 |86.93|\n|NP-Mix|62.53 |83.62| 87.15|\n|Feature Mixing|57.30| 84.41 |88.89|\n___\n>**Q2**: positioning relative to prior work on entropy maximization\n\n**A2**:  Thank you for the helpful comment. We would like to clarify that entropy maximization is not the main contribution of our work—it serves as a lightweight auxiliary component for outlier training. The **core novelty of our approach lies in multimodal outlier synthesis via Feature Mixing**, which addresses key limitations of prior methods, particularly in terms of scalability and computational efficiency in multimodal settings. While entropy-based techniques (e.g., ICCV 2021 work) have been explored in the unimodal segmentation context, they typically rely on external OOD datasets for training and do not generalize well to multimodal scenarios. In contrast, our method performs online outlier synthesis using only in-distribution data, eliminating the need for external data, and integrates entropy regularization as a self-contained, plug-in loss. Additionally, our method is significantly more efficient—achieving up to 370× speed-up over prior approaches. We will revise the paper to clarify this positioning and more clearly distinguish our contributions from prior work.\n___\n>**Q3**: challenges of extending existing solutions to multimodal setting\n\n**A3**: We agree that clarifying the challenges of extending unimodal methods to the multimodal setting is important for highlighting our motivation. While methods such as Mixup and VOS have shown success in unimodal tasks, directly applying them to multimodal data is non-trivial due to the following key challenges:\n\n**(1) Modality heterogeneity:** Multimodal data often involves inputs with fundamentally different structures and distributions—e.g., dense 2D images, sparse 3D point clouds, temporal audio signals, etc. Pixel-level interpolation techniques like Mixup are inherently incompatible across heterogeneous modalities, making it difficult to define meaningful cross-modal augmentations. Moreover, Mixup's random linear blending can produce ambiguous or noisy samples that may lie near or within the in-distribution (ID) manifold, reducing its effectiveness for OOD detection.\n\n**(2) Computational inefficiency:** Methods like VOS and NP-Mix rely on distribution estimation, sampling, or searching in high-dimensional feature spaces, which becomes computationally expensive when extended to multimodal settings—especially for dense prediction tasks such as segmentation, where per-pixel or per-point processing is required. \n\nOur Feature Mixing addresses these challenges by (i) operating entirely in the feature space, where modality-specific structures have already been abstracted, (ii) introducing a lightweight, swap-based perturbation strategy to synthesize outliers efficiently, avoiding costly sampling or searching, (iii) being theoretically grounded (Theorems 1 and 2) and empirically validated to maintain ID-OOD separation while improving efficiency and scalability.\n\nWe will revise the paper to include this discussion and better highlight the motivation and advantages of our approach.\n___\n>**Q4**: intuitive or empirical analysis on Feature Mixing\n\n**A4**: Thank you for the helpful comment. We provide both **theoretical and empirical evidence** to support the claim that our proposed method generates true outliers without injecting noise. \n\nTheoretically, Theorems 1 and 2 in the paper prove that samples generated via Feature Mixing have a low likelihood of residing within the in-distribution (ID) manifold, satisfying the criteria for true outliers. In contrast to interpolation-based methods like Mixup, our approach perturbs features in a structured manner, which helps avoid overlapping with the ID region.\n\nEmpirically, we visualize the synthesized outliers using t-SNE on HMDB51 dataset in Figure 4. As shown, Mixup tends to generate noisy samples that remain close to or within the ID feature cluster. In contrast, Feature Mixing produces outliers that are more clearly separated from the ID distribution, effectively expanding the boundary of the embedding space.\n___\n>**Q5**: comparison with mix-and-match\n\n**A5**: While our Feature Mixing and the Mix-and-Match strategy both involve perturbations in feature space, they differ fundamentally in **objective, mechanism, and application domain**:\n\n**1. Different Objectives**: The primary goal of mix-and-match is to encourage diversity within the data distribution for stochastic human motion prediction. In contrast, our Feature Mixing is designed to synthesize OOD samples for the explicit purpose of improving OOD detection and segmentation. We aim to create features that lie in low-likelihood regions of the in-distribution data, rather than generating plausible variations of it.\n\n**2. Different Mechanisms**: Mix-and-match perturbs a unimodal hidden state with random noise. Our Feature Mixing, however, operates in a multimodal setting by swapping feature dimensions between two different modalities (e.g., images and point clouds). This cross-modal exchange is a core element of our approach and is fundamentally different from adding noise to a single modality's representation.\n\n**3. Different Applications**: Mix-and-match is applied to generative modeling for sequence prediction. Our method is designed for discriminative models in the context of OOD detection.\n\nIn summary, while both methods involve a form of feature perturbation, our Feature Mixing is a novel approach for multimodal OOD synthesis, distinct from the unimodal, diversity-focused mix-and-match perturbation.\n___\n>**Q6**: include a decoder or reconstruction pipeline\n\n**A6**: Thank you for the thoughtful suggestion. We agree that incorporating a decoder or reconstruction pipeline from feature space back to image space could enhance interpretability by providing visual intuition about the synthesized outliers. In this work, our primary focus is on efficient multimodal outlier generation in the feature space, which avoids the complexity and overhead of generative modeling. As shown in Figure 4, we already provide t-SNE visualizations to empirically demonstrate that the outliers generated by Feature Mixing are well-separated from the in-distribution data, supporting their OOD nature.\n\nWe **explored the feasibility of feature-to-image inversion in a controlled setting using a text-to-image diffusion model** (Stable Diffusion 3 medium). Specifically, we used the text prompt \"a photo of a dog\" and extracted embeddings for the text. We then applied Feature Mixing by randomly selecting N dimensions from the first and second halves of the text feature embeddings and swapping them. Finally, we used the resulting text embedding as input to Stable Diffusion 3 to generate an image. We found that the generated images shifted away from the dog concept, indicating that the perturbed embedding moved outside the in-distribution manifold. This serves as initial evidence that Feature Mixing does in fact produce semantically meaningful outliers in embedding space. For multimodal scenarios, however, such as video or 3D point clouds, constructing reliable decoders is substantially more difficult due to data scarcity and the complexity of generative modeling across heterogeneous modalities. We therefore consider feature-to-modality inversion for complex modalities a valuable and promising direction for future work.\n___\nWe sincerely thank the reviewer again for the constructive feedback. We will incorporate all clarifications and updates from this rebuttal into the final version to make the paper clearer. If there are any additional questions or concerns, we would be more than happy to further address them."}, {"review_id": "B3uPAGrVWr", "reviewer": "Reviewer_GQ4N", "summary": "This paper propose Feature Mix, an OOD feature synthesizing method that allows model learns better to distinguish ID and OOD observations. The motivation of this paper is to improve the perception task performance by augment training with synthetic OOD features. Given multimodal input (image, LiDAR, or an agnostic modality), and their encoders, the stochasticity are added to the embeddings, which forms an OOD branch in the  neural network. The output of this OOD branch is being optimized, concretely by maximizing the embedding entropy, to help the network better distinguish the ID and OOD samples. Image segmentation and detection tasks on autonomous tasks based on proposed method are studied to validate the efficiency of proposed method.", "questions": "I thinks I mentioned most of the suggestion above:\n1. Methodology might need some reorganization to help readers better comprehend. Especially the part on page 6\n2. want to see more support around Figure 4 in paper and Figure 1 in appendix other than proofs that show how the proposed simple method is correct.", "limitations": "I tried to find it in appendix as mentioned but it seems missing?", "rating": 4, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strength\n\n* The paper is overall well written and organized. Problem is clearly defined.\n\n* Theorems, methods, proofs and experiments on real application data are in concert with each other, making it a self-containing work.\n\n* I like the fact that the proposed idea is indeed extremely simple and authors went through proofs of that, proving the effectiveness and bound (appendix). I went through the appendix and proofs looks reasonable to me, however I’m not a mathematical guy so I will lower down my confidence rating accordingly. (suggestions below for broader audience in case they are not into proofs) \n\nWeakness\n\n* Given the proposed simple feature mixing strategy, how the mixed branch is optimized is the center of this contribution. However, the explanation around this is not sufficient. It is hard to understand why maximizing the entropy of mixed OOD feature can better differentiate the OOD and ID samples. It is mentioned between line 208 and 213 and I get directed to eq.5.  I guess the narrative flow here might need some revision. \n\n* The 2 theorems proposed in the paper need more direct experimental and visualization support. Figure 4 in the main paper and Figure 1 in the appendix are good, but I would like to know how the data visualized in these figures are generated. Are they 2D or low-dimension toy examples, or are they real world data with reduced dimensions? Also in these figures why the ID data, as shown as purple dots, are not identical? I assumes that the comparison should be conducted on the same ID features as baselines. \n\n* In the table 1 and table 2, (and maybe table 3 and table 4 as well), please use consistent bold text to indicate the best entry in each comparisons. Otherwise they are hard to tell which is better.", "was_revised": true, "final_justification": "I would like to thank authors for rebuttal and my scores are remained.", "rebuttal": "Thanks for your insightful reviews and great support of our paper! We provide the responses to your questions as follows:\n>**Q1**: how the mixed branch is optimized? need some reorganization to help readers better comprehend\n\n**A1**: Thank you for the insightful comment. We agree that a clearer explanation of how the entropy maximization loss optimizes the mixed branch is important.\nThe core idea is that the mixed branch generates pseudo-OOD features through feature mixing. We apply an entropy maximization loss (Eq. 5) on these features to explicitly **encourage the model to produce uncertain (i.e., high-entropy) predictions** for them. This discourages the model from making overconfident predictions on ambiguous or unfamiliar inputs—behavior that is characteristic of OOD samples.\n\nIn contrast, the classification loss encourages low-entropy (i.e., high-confidence) predictions on in-distribution (ID) samples. Together, these complementary objectives train the model to develop a confidence-based decision boundary: **ID samples are associated with confident predictions, while pseudo-OOD samples—introduced through feature mixing—are explicitly pushed toward uncertain predictions.**\n\nAs shown in Figure 2, this training objective **increases the separation in predictive confidence between ID and OOD inputs**, which is crucial for effective OOD detection. \nWe appreciate your feedback and will revise the narrative between lines 208 and 213 to better highlight this motivation and improve the clarity around the optimization of the mixed branch.\n___\n>**Q2**: support around Figure 4 in paper and Figure 1 in appendix\n\n**A2**: Thank you for the thoughtful question. We apologize for the confusion. The visualizations in Figure 4 (main paper) and Figure 1 (appendix) are based on **real-world data from the HMDB51** dataset—not on toy or synthetic examples. Specifically, we apply our feature mixing strategy to generate pseudo-OOD features, and then use t-SNE to project the high-dimensional features into 2D for visualization.\n\nRegarding the ID features (purple dots): they are indeed **identical across all methods**. The apparent differences in their positions are due to the stochastic nature of t-SNE, which may project the same high-dimensional points differently depending on the surrounding context (i.e., the presence of different OOD samples). Since t-SNE emphasizes preserving local neighborhood structures rather than global geometry, the introduction of different OOD features can cause slight shifts in the 2D projections of the ID features as well.\n\nTo further clarify how these visualizations support our theoretical claims: Theorems 1 and 2 establish that feature mixing generates samples that lie outside the ID manifold (i.e., are valid outliers), yet remain bounded in their deviation. This is empirically reflected in Figure 4(d): the pseudo-OOD features are clearly separated from the ID cluster (supporting Theorem 1), while remaining relatively close rather than scattered far away (supporting Theorem 2). For Figure 1 (appendix), we prove in Eq. 6 (appendix) that the shifts of outliers from the ID distribution are proportional to N. Therefore, a large N promotes outliers to lie in low-likelihood regions of the ID feature distribution.\n\nTogether, these visualizations provide intuitive, empirical evidence that our method generates meaningful and bounded outliers, as predicted by our theoretical analysis. We will revise the text to clarify how the figures are generated and better connect the visualizations to Theorems 1 and 2.\n___\n>**Q3**: Consistent bold text in tables\n\n**A3**: Thank you for pointing this out. We acknowledge the inconsistency in highlighting the best-performing entries in Tables 1–4. In the final version, we will ensure that all tables consistently use bold text to indicate the best results in each comparison. This will significantly improve readability and make performance differences easier to interpret.\n___\n>**Q4**: discussion on limitations\n\n**A4**: Sorry for the confusion. We discuss the limitations and future work here and will include them in the final version of the paper. \n\nFeature Mixing uses a random selection of feature dimensions to swap between modalities. While this is highly efficient and theoretically grounded, it may not always target the most informative feature regions for outlier synthesis. Future work could explore **adaptive or learnable selection mechanisms** that dynamically identify features that maximize OOD separability.\n___\nWe sincerely thank the reviewer again for the constructive feedback. We will incorporate all clarifications and updates from this rebuttal into the final version to make the paper clearer. If there are any additional questions or concerns, we would be more than happy to further address them."}, {"review_id": "J2r6HOJcbL", "reviewer": "Reviewer_rcrp", "summary": "The paper introduces a simple but effective outlier synthesis procedure to enable better multimodal out-of-distribution detection (OOD). This procedure works by selecting random N indices for each modality feature map and swapping them. This guarantees that the outliers lie in a low-likelihood region and do not extremely deviate from the original features. Both of these claims are also theoretically proven. These outliers are then used in the entropy maximisation learning objective to prevent overconfidence for OOD samples. The procedure is evaluated for multimodal OOD segmentation and multimodal OOD classification. It achieves solid SOTA results in both scenarios.", "questions": "Besides the questions already included in the weaknesses section, I here list two that will have the most impact on my final opinion:\n- How does the parameter N affect the segmentation performance?\n- How hard is it to extend this to single modality? If not too hard to modify, what’s the performance like for segmentation with just RGB data used for both outlier synthesis and inference?", "limitations": "yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n- The paper is nicely written and easy to follow.\n- I like the simplicity of the proposed method; it addresses the issue without overcomplicating the whole process. This, combined with its applicability to related frameworks, makes the contribution strong, despite its simplicity.\n- The results achieved are solid and convincing.\n\nWeaknesses:\n- You say you set the OOD classes in SemanticKITTI and nuScenes to void and ignore them. Have you considered whether the model learns to implicitly detect these classes due to this decision?\n- While the method is designed for multimodal OOD, I would be interested in seeing how it performs in the monomodal scenario. Would the method still work if you just randomly shuffled N features in a single modality?\n- You claim that you repeat each training run with 3 random seeds and report the mean. Why not include the standard deviation, at least in the appendix?\n- It is not entirely clear how the method works for segmentation. Do you perform this mixing for each spatial location in feature maps? Are the indices distinct for each location? I would also like to see the ablation study of the parameter N for segmentation, not just detection.\n\nMinor:\n- You say that the work has no societal impact, yet I think that OOD detection for self-driving cars has some impact. I think it would be appropriate to include some discussion in the appendix.", "was_revised": true, "final_justification": "I strongly believe this paper **should be accepted**. \nThe authors have addressed all my concerns. Most were resolved in rebuttal, and a final question was resolved in the discussion.\nThe paper is clearly written and easy to read. The problem is nicely stated, and the solution is elegant and well-explained. The approach outperforms related methods on various benchmarks. The authors also introduce a new benchmark, which will be beneficial for the field.", "rebuttal": "Thanks for your insightful reviews and great support of our paper! We provide the responses to your questions as follows:\n>**Q1**: whether the model learns to implicitly detect these classes due to this decision\n\n**A1**: Thank you for your insightful comment. We follow the standard protocol in prior OOD segmentation works (e.g., [1]), where certain classes are assigned the void label and are excluded from the loss computation during training. This setup is commonly used due to the absence of publicly available 3D OOD segmentation datasets.\n\nHowever, as you correctly pointed out, this approach introduces the risk that the model may implicitly learn to detect these void-labeled classes, since their input features are still processed by the network. To address this limitation and eliminate the possibility of such implicit supervision, we **introduce the CARLA-OOD dataset, a new benchmark for multimodal OOD segmentation**. In CARLA-OOD, **OOD classes are strictly held out from the training data and only appear during testing, ensuring a clean evaluation protocol without any data leakage**.\n\nWe believe this new dataset provides a more rigorous and realistic testbed for multimodal OOD segmentation in autonomous driving.\n\n[1] Cen et al., Open-world Semantic Segmentation for LIDAR Point Clouds, ECCV 2022\n___\n>**Q2**: How the method works for segmentation? the ablation study of the parameter N for segmentation?\n\n**A2**: Thank you for the helpful questions, and we apologize for the lack of clarity.\nFor segmentation, we first project the LiDAR point cloud onto the image plane, following the approach in [1], to obtain spatially aligned pairs of LiDAR points and image pixels. Feature mixing is then performed at each valid pixel location—i.e., only where a corresponding LiDAR point exists—by mixing the aligned image and point features. Pixels without projected points are excluded from the mixing.\n\nWe have now also included the ablation study of the parameter \nN for segmentation on SemanticKITTI, as shown in the table below. We use N=10 by default, and the results show that the performance is **robust across different values of N**.\n\n[1] Zhuang et al., Perception-aware Multi-sensor Fusion for 3D LiDAR Semantic Segmentation, ICCV 2021\n\n||FPR95$\\downarrow$|AUROC$\\uparrow$|AUPR$\\uparrow$|mIoU$\\uparrow$|\n|-|-|-|-|-|\n|8| 38.58 |91.14 |53.45| 61.59|\n|10| 38.10| 91.47 |58.74| 61.18|\n|12 |37.17 |90.90 |55.19| 60.75|\n___\n>**Q3**: Feature Mixing on a single modality\n\n**A3**: Thank you for the insightful suggestion. We explored extending Feature Mixing to the single-modality setting by **randomly selecting N dimensions from the first and second halves of the feature embedding and swapping them within the same modality**. This approach enables outlier synthesis without relying on a second modality. We first evaluate this unimodal variant on the OOD detection task using the HMDB51 dataset (video modality). As shown in the table below, our **unimodal FM achieves strong performance**, outperforming the best NP-Mix baseline, which validates the applicability of our method beyond multimodal settings.\n\n||FPR95$\\downarrow$|AUROC$\\uparrow$|ACC$\\uparrow$|\n|-|-|-|-|\n|Baseline |64.05| 83.14 |86.93|\n|NP-Mix|62.53 |83.62| 87.15|\n|Feature Mixing|57.30| 84.41 |88.89|\n\nWe then apply the same unimodal FM strategy to the OOD segmentation task on SemanticKITTI using only LiDAR data. The results, also provided in the table below, demonstrate that our method **scales effectively to segmentation and maintains competitive performance in the single-modality case**. These results suggest that FM is flexible and generalizable, and can be extended to unimodal settings with minimal modifications.\n\n||FPR95$\\downarrow$|AUROC$\\uparrow$|AUPR$\\uparrow$|mIoU$\\uparrow$|\n|-|-|-|-|-|\n|Baseline|47.03 |85.82| 36.06| 59.81|\n|NP-Mix|48.26| 86.79 |45.73| 59.52|\n|Feature Mixing|46.40 |88.81 |47.67 |60.04|\n\nTo further demonstrate the generality of our approach, we conducted an additional experiment on a **three-modality setting involving video, optical flow, and audio** for OOD detection on the EPIC-Kitchens dataset. We extend Feature Mixing to this setting by randomly selecting a subset of N feature dimensions from each modality and swapping them cyclically: video->audio, audio->optical flow, and optical flow->video, thus creating outlier features for all three modalities. As shown in the table below, Feature Mixing consistently outperforms NP-Mix across all metrics in this three-modality setting (evaluated using the Energy score). These results further support the robustness and versatility of our method. \n\n||FPR95$\\downarrow$|AUROC$\\uparrow$|ACC$\\uparrow$|\n|-|-|-|-|\n|Baseline |69.22| 72.39| 73.13|\n|NP-Mix|62.69| 74.95 |71.46|\n|Feature Mixing|61.38 |77.23 |73.69|\n___\n>**Q4**: standard deviations\n\n**A4**: Thank you for pointing this out, and we apologize for the confusion. To maintain clarity and readability in the main paper, especially in longer tables such as Table 1 and Table 2, we chose to report only the mean values, as including standard deviations would have made the tables overly cluttered. That said, we agree that reporting variance is important for assessing the reliability of the results. We confirm that each experiment was run with three different random seeds, and we will include the standard deviations in the appendix in the final version of the paper for completeness.\n___\n>**Q5**: Societal impact\n\n**A5**: Thank you for your comment. We agree that OOD detection for self-driving cars has important societal implications. In our work, Feature Mixing advances multimodal OOD detection and segmentation by enabling more effective and efficient outlier exposure during training. This contributes directly to the reliability and safety of autonomous systems, including self-driving cars, by helping models handle unfamiliar or unexpected scenarios in open-world environments.\n\nBeyond autonomous driving, the method also has broader societal impact in other safety-critical domains such as healthcare and security, where robustness to unknown or anomalous data is crucial. By improving model reliability in the presence of distribution shifts, our approach **supports the deployment of AI systems in dynamic, high-stakes environments where failure could have significant consequences**.\n\nWe will include a discussion of these points in the appendix as suggested.\n___\nWe sincerely thank the reviewer again for the constructive feedback. We will incorporate all clarifications and updates from this rebuttal into the final version to make the paper clearer. If there are any additional questions or concerns, we would be more than happy to further address them."}, {"review_id": "9H2AIpnQ1f", "reviewer": "Reviewer_4Bsi", "summary": "This work proposes a new outlier synthesis method for multimodal OOD detection and segmentation. The idea is to randomly swap the elements at certain dimension of the features between the two modalities. Theoretical proofs are presented to indicate that the synthesized outliers may share similar characteristics to the real ones in deployment environment. Empirical experiments support the two major claims of this work, that the proposed method leads to improved performance while being much more efficient in outlier synthesis (in terms of runtime).", "questions": "Please see Weaknesses/Questions/Suggestions.", "limitations": "It is claimed in the paper checklist that \"In the Appendix, we discussed limitations and future work.\" Yet I didn't see such discussion in the appendix. It would be helpful to discuss whether scaling to more modalities would be an limitation of the proposed method.", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "### Strengths\n\n1. The two major claims (improved performance and efficiency) are supported well.\n\n### Weaknesses/Questions/Suggestions\n\n1. The swapping mechanism seems to be specific for cases where there are strictly two modalities. What if there are more modalities? How are the features going to be swapped?\n2. It's good to have theoretical proofs in section 3.3, yet the two conclusions derived from the proofs (the \"two key properties\" at line 179) are somewhat ambiguous in terms of how they are going to benefit OOD detection. Instead, I think a more straightforward approach is to visualize real outliers together with synthesized outliers in Figure 4 and see if they have large overlap. If so, then it is a strong evidence that the synthesized outliers could be useful in regularizing the model's outputs against real outliers (since they tend to overlap in model feature space).\n3. In appendix line 8 Eq. (1) and (2), the mask is the same for either modality's features (with the same notation $\\mathcal{M}$). This is not consistent with what's being shown in Figure 3 and Algorithm 1, where there are obviously two different masks for each modality.", "was_revised": true, "final_justification": "My questions are well-addressed by the rebuttal, where I find the additional results under three-modality and single-modality scenarios quite helpful, demonstrating the great flexibility of the proposed approach. I'm thus raising my score from 4 (pre-rebuttal) to 5 (final assessment).", "rebuttal": "Thanks for your insightful reviews and great support of our paper! We provide the responses to your questions as follows:\n>**Q1**: What if there are more modalities? How are the features going to be swapped?\n\n**A1**: Thanks for your suggestion! To further demonstrate the generality of our approach, we conducted an additional experiment on a **three-modality setting involving video, optical flow, and audio** for OOD detection on the EPIC-Kitchens dataset. We extend Feature Mixing to this setting by randomly selecting a subset of N feature dimensions from each modality and swapping them cyclically: video->audio, audio->optical flow, and optical flow->video, thus creating outlier features for all three modalities. As shown in the table below, Feature Mixing consistently outperforms NP-Mix across all metrics in this three-modality setting (evaluated using the Energy score). These results further support the robustness and versatility of our method. \n\n||FPR95$\\downarrow$|AUROC$\\uparrow$|ACC$\\uparrow$|\n|-|-|-|-|\n|Baseline |69.22| 72.39| 73.13|\n|NP-Mix|62.69| 74.95 |71.46|\n|Feature Mixing|61.38 |77.23 |73.69|\n\nImportantly, FM is inherently **modality-agnostic**: it operates in the feature space without relying on assumptions about the input structure, enabling straightforward application to diverse and heterogeneous modality combinations. We further extend FM to the **single-modality setting** by randomly selecting N dimensions from the first and second halves of the feature embedding and swapping them within the same modality. This enables outlier synthesis without relying on a second modality. We evaluate this unimodal variant on OOD detection using HMDB51 (video modality). As shown below, our unimodal FM achieves strong performance, outperforming the best NP-Mix baseline, which validates the applicability of FM beyond multimodal settings.\n||FPR95$\\downarrow$|AUROC$\\uparrow$|ACC$\\uparrow$|\n|-|-|-|-|\n|Baseline |64.05| 83.14 |86.93|\n|NP-Mix|62.53 |83.62| 87.15|\n|Feature Mixing|57.30| 84.41 |88.89|\n\n___\n>**Q2**: Clarification of the \"two key properties\" and their benefit to OOD detection\n\n**A2**: Thank you for the thoughtful suggestion. Theoretical results in Theorems 1 and 2 establish two key properties of Feature Mixing (FM): (1) the synthesized outliers have low likelihood under the ID distribution, and (2) their deviation from the ID distribution is bounded. These properties are important for OOD detection and segmentation, as they ensure that **FM produces challenging but plausible outliers that populate the low-density regions near the boundary of the ID distribution**. This supports effective decision boundary regularization and reduces model overconfidence on unseen data. \n\nAs shown in Figure 4, the FM-generated outliers are well-separated from the ID features in the embedding space. This separation helps the model to better distinguish between in-distribution and out-of-distribution regions. These visualizations provide intuitive, empirical evidence that our method generates meaningful and bounded outliers, as predicted by our theoretical analysis.\n\nIn response to your suggestion, we have additionally visualized real outliers alongside FM-synthesized outliers, and we observe a significant degree of overlap between them. This provides empirical evidence that the synthesized outliers are representative of real OOD samples, which supports their effectiveness in regularizing the model. Interestingly, **FM-synthesized outliers also extend into other low-likelihood regions not covered by the real OOD samples, further increasing the diversity and coverage of the synthesized data**.\n___\n>**Q3**: the mask is not consistent\n\n**A3**: Thank you for pointing this out, and we apologize for the confusion.\nIn Appendix, we used the same mask notation for both modalities in Equations (1) and (2) for notational simplicity in the theoretical analysis. However, as correctly illustrated in Figure 3 and Algorithm 1, we do use two distinct masks—one for each modality—in the actual implementation.\n\nThis simplification in notation **does not affect the validity of the theoretical results**. For example, the expectations computed in Equations (3) and (4) in appendix remain the same regardless of whether the two modalities share the same mask or not, since the proofs rely on the marginal distributions rather than the specific mask values.\n\nWe will correct the notation in the final version of the paper to accurately reflect the use of distinct masks and to ensure consistency throughout the manuscript.\n___\n>**Q4**: discussion on limitations\n\n**A4**: Sorry for the confusion. We discuss the limitations and future work here and will include them in the final version of the paper. \n\nFeature Mixing uses a random selection of feature dimensions to swap between modalities. While this is highly efficient and theoretically grounded, it may not always target the most informative feature regions for outlier synthesis. Future work could explore **adaptive or learnable selection mechanisms** that dynamically identify features that maximize OOD separability.\n___\nWe sincerely thank the reviewer again for the constructive feedback. We will incorporate all clarifications and updates from this rebuttal into the final version to make the paper clearer. If there are any additional questions or concerns, we would be more than happy to further address them."}]

Author rebuttal:

Meta-review: This paper proposes an outlier synthesis technique for multi-modal OOD detection based on the simple yet effective idea of feature mixing. The authors successfully addressed the reviewers' concerns during the discussion phase, and as a result, all the reviewers unanimously support acceptance. The strengths highlighted by the reviewers are: strong empirical results, simplicity of the proposed method, flexibility across diverse multi-modal settings, the introduction of a new multi-modal OOD benchmark, and clear presentation.

Therefore, AC recommends acceptance of this submission. The authors are strongly encouraged to incorporate the reviewers' feedback (e.g., additional experimental results, extended discussions, clarifications, and implementation details), as doing so would further strengthen and clarify the contributions of this work.
Decision: Accept (poster)