Kati8/ICLR_2023_2025 huggingface dataset:
authors: Zhuo Huang
authors: Gang Niu
authors: Bo Han
authors: Tongliang Liu
reviews: 
Comments: This paper propose a novel out-of-modal generalization problem that focuses on generalizing learned knowledge to unknown modalities. In addition, it considers both semi-supervised and unsupervised scenarios. The proposed connect & explore (COX) scheme tries to explore the shared and unique between modalities via variational information bottleneck. The proposed method can build connections between known and unknown modalities, further generating correspondences to explore more knowledge. The expleriments verify the effectiveness of COX.
Strengths: 1. This paper proposes a novel OOM problem, which is valuable because of the importance on generalizing exsting knowledge to unknown modalities without paired instances. 
2. The proposed COX method sounds reasonable and explores the connections between different modalities without priori knowledge. The motivation and method are clear to me. The results are convincing to me.
Weaknesses: 1. I acknowledge the novelty and value of the proposed OOM scenario. However, since the method lacks paired samples, it only shows limited effectiveness in learning shared knowledge, making it significantly inefficient in exploring unknown modalities. This is evident in the experimental results, which show a large gap compared to "aligned". Although the compared methods contain ERM and SSL, I question whether the proposed approach is better than SOTA unsupervised uni-modal solutions. Only surpassing or enhancing existing uni-modal methods with the proposed approach can make it really valuable. 
2. As I said in 1, the comparison with existing uni-modal methods is necessary.
Recommendation: 6
Reviewer confidence: 4
Soundness/correctness: 3
Clarity: 3
reviews: 
Comments: This paper addresses the problem of out-of-model (OOM) generation and proposes a novel technique, COX. The authors perform experiments on five datasets, comparing their method with several baseline approaches.
Strengths: The problem statement is clear.
Weaknesses: Insufficient and weak baselines: I am especially concerned the proposed approach's performance compared with SSL. 

Some notations should be improved, and a few detailed explanations should be added.
Recommendation: 6
Reviewer confidence: 4
Soundness/correctness: 2
Clarity: 2
reviews: 
Comments: This work focuses on the semi-supervised and  unsupervised scenarios of *Out-of-Model* (OOM) Generalization without given instance-level model correspondence. This work proposes an OOM generalization method based on the interactive relationship between modalities, *connect & explore* (COX), in an attempt to extract the commonality that can help partially comprehend OOM data based on IM data. It introduce a variational information bottleneck framework to connect OOM data and In-Model (IM) data and extract shared information. Finally, several sets of experiments demonstrate the effectiveness and extensibility of the proposed method.
Strengths: This work focuses on the novel and practical issues in the field of *Out-of-Model* (OOM) generalization.

This work proposes an approach that attempts to combine common knowledge based on connections across modalities with unique knowledge of OOM data based on modality disagreement.

This work conducts multiple sets of experiments to verify the effectiveness and extensibility of the proposed method.
Weaknesses: (i) The OOM problem has also been discussed in other works, such as IB for modality missing, cross-modal generalization, etc., ([1-5] for example) but not mentioned in the paper (nor experiments). Please explain the difference between this paper and these works. [1] Unpaired image-to-speech synthesis with multimodal information bottleneck. [2] Visual explanations of image-text representations via multi-modal information bottleneck attribution. [3] Dynamic Multimodal Information Bottleneck for Multimodality Classification. [4] SimMMDG: A simple and effective framework for multi-modal domain generalization

(ii) The work mentions "prediction uncertainty" in section 4.2, but after carefully reviewing all the contents, it is hard to find the definition or description of prediction uncertainty, while without the direct description of this concept in the experiment section for verification. 

(iii)The work mentions "the performance of COX is affected by the quality of IM perceptors", but only contrast the performance between ImageBind and LanguageBind in the experiment section can not fully illustrate this conclusion.

(iv) Has some loopholes, e.g.,

  - Tense and grammatical errors, e.g., "OMM data" -> "OOM data"
  - Confusing notations, e.g.,"given IM data *X<sup> O </sup>*and OOM data *X<sup> I </sup>*"

Please correct the grammatical mistakes and polish them if possible.

(v) In the experimental part, the results in Table 2 and Table 3 can not adequately illustrate this conclusion "leveraging the knowledge from IM perceptors can indeed help OOM generalization compared to using OOM data alone". How do the results in Table 2 and Table 3 show the performance improvement of the proposed method compared to the methods using only OOM data?
Recommendation: 6
Reviewer confidence: 4
Soundness/correctness: 3
Clarity: 3