Title: Extremely Simple Multimodal Outlier Synthesis for Out-of-Distribution Detection and Segmentation
Venue: NeurIPS 2025
Accepted: yes
Keywords: Outlier Synthesis

Multimodal Learning

OOD Detection

OOD Segmentation
Abstract: Out-of-distribution (OOD) detection and segmentation are crucial for deploying machine learning models in safety-critical applications such as autonomous driving and robot-assisted surgery. While prior research has primarily focused on unimodal image data, real-world applications are inherently multimodal, requiring the integration of multiple modalities for improved OOD detection. A key challenge is the lack of supervision signals from unknown data, leading to overconfident predictions on OOD samples. To address this challenge, we propose Feature Mixing, an extremely simple and fast method for synthesizing multimodal outliers with theoretical support, which can be further optimized to help the model better distinguish between in-distribution (ID) and OOD data. Feature Mixing is modality-agnostic and applicable to various modality combinations. Additionally, we introduce CARLA-OOD, a new multimodal dataset for OOD segmentation, featuring synthetic OOD objects across diverse scenes and weather conditions. Extensive experiments on SemanticKITTI, nuScenes, CARLA-OOD datasets, and the MultiOOD benchmark demonstrate that Feature Mixing achieves state-of-the-art performance with a $10 \times$ to $370 \times$ speedup. Our source code and dataset are available at https://github.com/mona4399/FeatureMixing.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis work proposes a new outlier synthesis method for multimodal OOD detection and segmentation. The idea is to randomly swap the elements at certain dimension of the features between the two modalities. Theoretical proofs are presented to indicate that the synthesized outliers may share similar characteristics to the real ones in deployment environment. Empirical experiments support the two major claims of this work, that the proposed method leads to improved performance while being much more efficient in outlier synthesis (in terms of runtime).\n\nSTRENGTHS AND WEAKNESSES:\n### Strengths\n\n1. The two major claims (improved performance and efficiency) are supported well.\n\n### Weaknesses/Questions/Suggestions\n\n1. The swapping mechanism seems to be specific for cases where there are strictly two modalities. What if there are more modalities? How are the features going to be swapped?\n2. It's good to have theoretical proofs in section 3.3, yet the two conclusions derived from the proofs (the \"two key properties\" at line 179) are somewhat ambiguous in terms of how they are going to benefit OOD detection. Instead, I think a more straightforward approach is to visualize real outliers together with synthesized outliers in Figure 4 and see if they have large overlap. If so, then it is a strong evidence that the synthesized outliers could be useful in regularizing the model's outputs against real outliers (since they tend to overlap in model feature space).\n3. In appendix line 8 Eq. (1) and (2), the mask is the same for either modality's features (with the same notation $\\mathcal{M}$). This is not consistent with what's being shown in Figure 3 and Algorithm 1, where there are obviously two different masks for each modality.\n\nQUESTIONS:\nPlease see Weaknesses/Questions/Suggestions.\n\nLIMITATIONS:\nIt is claimed in the paper checklist that \"In the Appendix, we discussed limitations and future work.\" Yet I didn't see such discussion in the appendix. It would be helpful to discuss whether scaling to more modalities would be an limitation of the proposed method.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-06-16"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces a simple but effective outlier synthesis procedure to enable better multimodal out-of-distribution detection (OOD). This procedure works by selecting random N indices for each modality feature map and swapping them. This guarantees that the outliers lie in a low-likelihood region and do not extremely deviate from the original features. Both of these claims are also theoretically proven. These outliers are then used in the entropy maximisation learning objective to prevent overconfidence for OOD samples. The procedure is evaluated for multimodal OOD segmentation and multimodal OOD classification. It achieves solid SOTA results in both scenarios.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n- The paper is nicely written and easy to follow.\n- I like the simplicity of the proposed method; it addresses the issue without overcomplicating the whole process. This, combined with its applicability to related frameworks, makes the contribution strong, despite its simplicity.\n- The results achieved are solid and convincing.\n\nWeaknesses:\n- You say you set the OOD classes in SemanticKITTI and nuScenes to void and ignore them. Have you considered whether the model learns to implicitly detect these classes due to this decision?\n- While the method is designed for multimodal OOD, I would be interested in seeing how it performs in the monomodal scenario. Would the method still work if you just randomly shuffled N features in a single modality?\n- You claim that you repeat each training run with 3 random seeds and report the mean. Why not include the standard deviation, at least in the appendix?\n- It is not entirely clear how the method works for segmentation. Do you perform this mixing for each spatial location in feature maps? Are the indices distinct for each location? I would also like to see the ablation study of the parameter N for segmentation, not just detection.\n\nMinor:\n- You say that the work has no societal impact, yet I think that OOD detection for self-driving cars has some impact. I think it would be appropriate to include some discussion in the appendix.\n\nQUESTIONS:\nBesides the questions already included in the weaknesses section, I here list two that will have the most impact on my final opinion:\n- How does the parameter N affect the segmentation performance?\n- How hard is it to extend this to single modality? If not too hard to modify, what’s the performance like for segmentation with just RGB data used for both outlier synthesis and inference?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-20"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper propose Feature Mix, an OOD feature synthesizing method that allows model learns better to distinguish ID and OOD observations. The motivation of this paper is to improve the perception task performance by augment training with synthetic OOD features. Given multimodal input (image, LiDAR, or an agnostic modality), and their encoders, the stochasticity are added to the embeddings, which forms an OOD branch in the  neural network. The output of this OOD branch is being optimized, concretely by maximizing the embedding entropy, to help the network better distinguish the ID and OOD samples. Image segmentation and detection tasks on autonomous tasks based on proposed method are studied to validate the efficiency of proposed method.\n\nSTRENGTHS AND WEAKNESSES:\nStrength\n\n* The paper is overall well written and organized. Problem is clearly defined.\n\n* Theorems, methods, proofs and experiments on real application data are in concert with each other, making it a self-containing work.\n\n* I like the fact that the proposed idea is indeed extremely simple and authors went through proofs of that, proving the effectiveness and bound (appendix). I went through the appendix and proofs looks reasonable to me, however I’m not a mathematical guy so I will lower down my confidence rating accordingly. (suggestions below for broader audience in case they are not into proofs) \n\nWeakness\n\n* Given the proposed simple feature mixing strategy, how the mixed branch is optimized is the center of this contribution. However, the explanation around this is not sufficient. It is hard to understand why maximizing the entropy of mixed OOD feature can better differentiate the OOD and ID samples. It is mentioned between line 208 and 213 and I get directed to eq.5.  I guess the narrative flow here might need some revision. \n\n* The 2 theorems proposed in the paper need more direct experimental and visualization support. Figure 4 in the main paper and Figure 1 in the appendix are good, but I would like to know how the data visualized in these figures are generated. Are they 2D or low-dimension toy examples, or are they real world data with reduced dimensions? Also in these figures why the ID data, as shown as purple dots, are not identical? I assumes that the comparison should be conducted on the same ID features as baselines. \n\n* In the table 1 and table 2, (and maybe table 3 and table 4 as well), please use consistent bold text to indicate the best entry in each comparisons. Otherwise they are hard to tell which is better.\n\nQUESTIONS:\nI thinks I mentioned most of the suggestion above:\n1. Methodology might need some reorganization to help readers better comprehend. Especially the part on page 6\n2. want to see more support around Figure 4 in paper and Figure 1 in appendix other than proofs that show how the proposed simple method is correct.\n\nLIMITATIONS:\nI tried to find it in appendix as mentioned but it seems missing?", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 2, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper addresses the problem of outlier synthesis for out-of-distribution (OOD) detection. The key observation is that neural networks often produce high-confidence predictions for OOD samples, and outlier synthesis can help mitigate this overconfidence. The authors point out that most existing outlier synthesis methods are designed for unimodal settings and have limitations when applied to multimodal data. To address this, the paper proposes a new feature mixing strategy tailored for the multimodal setting. Specifically, given in-distribution features from two modalities, the method randomly swaps a subset of N feature dimensions between them to generate synthetic multimodal outliers. Experimental results demonstrate both the efficiency and effectiveness of the proposed approach.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. The proposed multimodal feature mixing strategy for outlier synthesis is intuitive and effective. By operating in the feature space rather than the pixel space, it enables efficient generation of synthetic multimodal outliers.\n\nWeaknesses:\n1. While the feature mixing approach demonstrates good empirical performance, its effectiveness across different modality types is not thoroughly analyzed. A deeper investigation into how the method behaves with varying modality combinations would help better assess its generalizability and contribution.\n2. The use of entropy maximization for OOD calibration is not novel. For example, similar ideas have been explored in \"Entropy Maximization and Meta Classification for Out-of-Distribution Detection in Semantic Segmentation\" (ICCV 2021). A clearer positioning of the proposed method relative to such prior work would strengthen the paper.\n3. The challenges of extending existing unimodal feature mixing solutions to multimodal setting are not clearly explained.\n\nQUESTIONS:\n1. In line 150, the authors state that Mixup may generate noisy samples that still lie within the ID distribution. Further clarification is needed to support the claim that the proposed method avoids this limitation. While visualizing features directly may be challenging, some form of intuitive or empirical analysis would strengthen the argument. \n2. What are the potential challenges or limitations in directly extending Mixup or VOS to multimodal settings? A brief discussion would help highlight the motivation for the proposed approach.\n3. How does the proposed feature mixing strategy differ from the dynamic mix-and-match perturbation introduced in “A Stochastic Conditioning Scheme for Diverse Human Motion Prediction” (CVPR 2020)? A comparison would help clarify the novelty of the method.\n4. Given that the framework operates in feature space, it would be valuable to include a decoder or reconstruction pipeline that maps the mixed features back to image space. This could provide further interpretability and insight into the nature of the synthesized outliers.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-03"}