Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: This paper proposes optimal transport (OT) for the multi-modal fusion procedure of multi-modal KGE. It further provides theoretical analysis on target errors in OT fusion and generalization bounds for multi-modal KGE.
strengths_and_weaknesses: Strengths
1. For previous multi-modal KGE methods, the embeddings from different modals are in various heterogeneous spaces, which is hard to avoid by using direct fusion. This paper proposes to mitigate the gap between heterogeneous spaces by using optimal transport (OT). The results of the ablation study clearly show the advantage of using OT for fusion over direction fusion counterparts like average or concatenation for fusion. Moreover, OTKGE can capture a wider range of Inference patterns than previous works.
2. Theoretical analysis of two important questions is given. Theorem 1 shows the target error can be bounded by the Wasserstein distances of different modals. Theorem 2 shows the generalization bound for the latent representation quality, and it further implies that multi-modal KGE can outperform uni-modal KGE analytically. In addition, the guarantee of Theorem 1 can not be easily achieved by other methods due to their fusion approaches. Theorem 2 justifies why multi-modal KGE is better. I think these theoretical results well distinguish the proposed methods from previous methods.
3. The experimental results are comprehensive, and the ablation study aligns with theoretical analysis.

Weaknesses
1. The organization of the theoretical analysis can be improved. It's better to split section 4 into two subsections. In addition, it would be better if the authors could state the relationships between Theorem 1 and Theorem 2 (if there are any). In addition, algorithm 1 and algorithm 2 can be moved into the main text instead of the appendix.

questions: 1. Will the relationship in Eq. 12 always hold? If not, what are the possible cases that violate Eq.12?
2. In Line 245-246, the authors said that "The impact of the intrinsic complexity of function classes will be reduced when the sample size m is larger", I think this result is quite interesting. Could you provide some experimental results regarding this argument? 

limitations: The limitations of this paper are properly discussed.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: The paper studies the multi-modal knowledge graph embedding, which leverages textual, visual, and structure knowledge for learning more accurate representations of entities. The authors propose to utilize the optimal transport to transport different embedding distributions into a unified embedding space, then design a fusion strategy to obtain the final entity embeddings. The authors also provide theoretical analysis to support their claims.
strengths_and_weaknesses: Strengths:
- The work is well motivated. Multi-modal data have heterogeneous spaces. How to fuse their representations is an interesting problem.
- It is novel to leverage the optimal transport to transfer different embedding distributions to a unified space.
- Theoretical analysis is provided to support the central claims.
- The paper is well written and easy to follow.


Weaknesses:
- Experimental results need more discussion and explanations.

questions: - I wonder why OTKGE outperforms other baselines in the uni-modal setting. Is that attributed to the design of relation r? However, we can see from Table 3 that OTKGE w/o trans has minimal performance degradation. It would be better if the authors could provide more detailed ablation studies on uni-model datasets.
limitations: Yes
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: This paper focuses on the multi-modal fusion problem. The authors propose a method to align the different modalities by reducing the Wasserstein distance between different modal distributions. Experimental results show that the proposed method achieves state-of-the-art performance.
strengths_and_weaknesses: Strengths:
1 The paper is well written and easy to follow. 
2 The motivation is clear.
3 The experiments and ablation studies well demonstrate the effectiveness of the proposed method.
Weaknesses：
1 The idea of aligning the different modalities is not novel.
2 Equation 4 is supposed to be further explained. What’s the insight behind the multi-modal fusion based on the Wasserstein barycenter?

questions: 1 In Equation (4), the symbol E represents the structural embedding? The author should clarify.
2 In the experiment, the author should compare the proposed fusion method with other multi-modal fusion methods based on the aligned multi-modal embeddings.

limitations: Yes
review_title: null

Meta-review: metareview: This paper presents a method to learn multi-modal knowledge graph embeddings. To integrate the embeddings from different modalities, which is a difficult task because of the heterogeneity across the different modalities, the paper presents an optimal transport based method to learn multi-modal embeddings.

The paper received positive reviews from all the reviewers. The authors submitted a rebuttal to answer the questions from the reviewers, and the reviewers seem to be satisfied.

Given the unanimously positive reviews and my own reading of the paper, I vote for the acceptance of the paper.
confidence: Certain
Decision: Accept