Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 8: Strong Accept: Technically strong paper, with novel ideas, excellent impact on at least one area, or high-to-excellent impact on multiple areas, with excellent evaluation, resources, and reproducibility, and no unaddressed ethical considerations.
review_content: summary: This paper presents CLUE for cross-modality representation learning from incomplete observations using the cross-encoders. Specifically the focus of CLUE is to learn comprehensive representations from datasets with each having a subset of modalities possibly unimodal.


strengths_and_weaknesses: Strength:
The CLUE make use of the cross-linked VAEs to model the cross-modality encoding.
It leans a comprehensive latent encoding with partially available modalities.
Achieve better performance than the other compared methods.

Weaknesses:
It need K^2 cross-encoders for K modalities.
Scalability may be limited as it requires large-number of cross models to store and compute thus poses the space and computational computational challenges.

questions: Instead of concatenating the latent encoding of different VAEs, what is the effect of learning a shared latent encoding that reconstruct other modalities while using their respective parameters in decoder?
limitations: Yes
review_title: null

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: The paper proposes CLUE (Cross-Linked Unified Embedding) to construct multimodal representations from modality-incomplete datasets and demonstrates its application on multi-modal single-cell data integration.
strengths_and_weaknesses: \+The paper is well written and clearly presented.

\+The paper extends the multi-VAE architecture following a good intuition (preserving modality-specific features rather than simply aligning multimodal data).

\+CLUE yields large improvement over baseline approaches.


-The evaluation is conducted on one dataset for one specific application, which seems a bit limited.

questions: + Can the model be generalized to other applications other than single-cell data integration?

+ For the evaluation, is there other available dataset for the task? Consistent performance improvement over multiple datasets will help demonstrate the generalization ability of CLUE.  
limitations: See questions above
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: This paper proposes a semi-supervised neural network named CLUE (Cross-Linked Unified Embedding) for cross-modality representation learning. The core part is the proposed cross-encoders, which can learn modality-specific representations and combine them to build an effective feature embedding.
strengths_and_weaknesses: This paper uses cross-encoders to learn comprehensive representation. However, the experiments are insufficient.
questions: 1. Some works also try to explore the intersection between different modalities for cross-modal representation learning. What is the main difference between your method and other methods (e.g., “Look, Listen, and Attend: Co-Attention Network for Self-Supervised Audio-Visual Representation Learning” ) ?.
2. In Eq.14, the weights of different losses are different. Please explain the exact reasons. How do you decide the weight values of different losses?
3. Please describe the detailed experimental settings, such as learning rate, etc., so the reader can reproduce.
4. IS the same experiments for all compared methods in Table1 and Table 2? 
5.What is the contribution of $q({{z}_{2}}|{{x}_{1}})$or $q({{z}_{1}}|{{x}_{2}})$,respectively.

limitations: We recommend enriching the experimental section, such as experimental setup and parameter analysis of the method.
review_title: null

Meta-review: metareview: In this paper, the authors propose CLUE (Cross-Linked Unified Embedding) to construct multimodal representations from modality-incomplete datasets and apply CLUE to the single-cell data integration problems. The proposed method is simple yet effective and shows the superior performance over state-of-the-art methods. All reviewers agree to accept the paper; I will also vote for acceptance. In the final version, I encourage the authors to improve the experimental section by addressing the reviewer's concerns.
confidence: Less certain
Decision: Accept