Paper: EQA-MX: Embodied Question Answering using Multimodal Expression (ICLR-2024)

Reviewer comment (Reproducibility):
There seem to be some details missing on the processing of non-verbal gestures. For example, how is the pointing direction determined? Does the network take in absolute 3D joint positions, relative joint rotations, or some other representation? How is gaze represented?

Author response:
**Reviewer Comment:**
> How is the pointing direction determined? How is gaze represented? Does the network take in absolute 3D joint positions, relative joint rotations, or some other representation?

**Response:** We utilize the implementation of the CAESAR simulator’s pointing [11] to determine the pointing direction. Particularly, inverse kinematics are used to determine the exact joint rotations necessary to point to an object. In addition to inverse kinematics, we utilize physical simulation and real human motion data to guide the movement of an arm towards the intended target.

Importantly, while the data samples include explicit pointing gesture information in the form of 3D skeleton positions, our model primarily processes the image data and does not explicitly use this skeleton information. Our goal is to enable the model to intuitively interpret the pointing gesture signals that are naturally present in the images, thereby mimicking human-like perception in recognizing and understanding nonverbal cues.

We believe this methodology not only enhances the model's capability in interpreting complex scenes but also aligns with real-world scenarios where explicit gesture data might not be available.

[11] Islam, Md Mofijul, Reza Mirzaiee, Alexi Gladstone, Haley Green, and Tariq Iqbal. “CAESAR: An Embodied Simulator for Generating Multimodal Referring Expression Datasets.” Advances in Neural Information Processing Systems 35 (December 6, 2022): 21001–15. https://proceedings.neurips.cc/paper_files/paper/2022/hash/844f722dbbcb27933ff5baf58a1f00c8-Abstract-Datasets_and_Benchmarks.html.

Reviewer comment (Experiments):
Do the authors consider any situations with multiple similar objects? For example, in Fig. 1, what if there are two red lamps next to each other? By extension, what other sources of ambiguity can exist in the dataset (for example, specific pointing with a finger vs. general pointing with the palm facing upwards, pointing to an object that is behind another object in the gaze direction, etc.)?

Author response:
**Reviewer Comment:**
> Do the authors consider any situations with multiple similar objects? For example, in Fig. 1, what if there are two red lamps next to each other?

**Response:** Our dataset generation with the CAESAR simulator involves creating scenes with multiple objects, including those that are similar in terms of attributes such as color, position, shape, and size. The simulator is designed to randomly populate scenes with objects, which naturally leads to the occurrence of similar objects within the same scene.

For instance, in scenarios like the one you mentioned with two red lamps next to each other, our method effectively generates and handles such instances. This capability is crucial for our Object Counting (OC) and Object Attribute Compare (OAC) tasks, where the distribution and characteristics of objects in the scenes are varied and can include multiple similar items.

This approach ensures our dataset reflects a realistic variety of scenarios, enhancing the generalizability and applicability of our model in real-world settings. The random generation of scenes and the varied object attributes contribute significantly to the robustness and effectiveness of our model in comprehending complex visual scenes.

Reviewer comment (Reproducibility):
The proposed method described seems lack of implementation details, I think a section should be devoted to describe each component of the model in very details. I don't understand why multiple codebooks are used in the paper, as read from the figure 4. I understand that each vector representation is divided into different segments along its feature dimension, so the question: is each segment belong to different codebooks? What is the Factor-1, Factor-2, ..., Factor-N in the figure, standing for N codebooks?

Author response:
**Reviewer Comment:**
> The proposed method described seems lack of implementation details, I think a section should be devoted to describe each component of the model in very details. I don't understand why multiple codebooks are used in the paper, as read from the figure 4. I understand that each vector representation is divided into different segments along its feature dimension, so the question: is each segment belong to different codebooks? What is the Factor-1, Factor-2, ..., Factor-N in the figure, standing for N codebooks?

**Response:**
We provide implementation details in the paper and the supplementary document. We also like to refer to our source code for additional implementation details. Thank you for the clarification question regarding the factorial VQ-VAE bottleneck that we use in this paper. 

Clarification of VQ implementation: Our inspiration for using factorial VQ-VAE comes from [Islam et al., 2022, NeurIPS], where we adapt the factorized VQ-VAE model to learn a factorial representation of visual embedding. In the discretization process, we segment the embedding into G components, and each segment is mapped independently to a discretized latent vector where the size of the latent vector depends on the size of the discrete latent space (L-way categorical). In other words, we get discrete factored codebooks for each segmented embedding, and each segment would belong to different codebooks. We emphasize that the codebooks are not shared between the segments, but rather, we use separate codebook spaces for each factor (1 to N). These segments are what we refer to as the factors of the learnt representation, and the segments can then be concatenated to obtain the final embedding (or the corresponding final discretized vector). 

In summary, we tend to use multiple codebooks with the intuition from previous works that the segmented or factorized representations, using this form of segmentation, would enable us to learn a factorial (or compositional) representation from the data. In our work, each of these factors can learn different aspects of the scene that align to the verbal representation. We would like to point out that learning compositional representations is indeed a hard problem, and [Islam et al., 2022] is the closest work from our knowledge that learns this sort of factorial representation for multimodal data. 


**Reference:** Islam, Riashat, Hongyu Zang, Anirudh Goyal, Alex Lamb, Kenji Kawaguchi, Xin Li, Romain Laroche, Yoshua Bengio, and Remi Tachet Des Combes. "Discrete factorial representations as an abstraction for goal conditioned reinforcement learning." arXiv preprint arXiv:2211.00247 (2022).

Reviewer comment (Evaluation):
More qualitative studies regarding the samples answered by VQ-fusion and without VQ-fusion should be included in the paper to help understand why the VQ-fusion works and brings more insight to the proposed baseline approach.

Author response:
**Reviewer Comment:**
> More qualitative studies regarding the samples answered by VQ-fusion and without VQ-fusion should be included in the paper to help understand why the VQ-fusion works and brings more insight to the proposed baseline approach. 

**Response:** The additional qualitative analyses complement the extensive quantitative ablation studies already conducted (Main Paper Sections 7.2 (Table 4) and 7.2 (Table 5); Supplementary document Sections 3.1 (Table 1) and 3.2 (Table 2)). These quantitative studies robustly demonstrate the impact of our proposed approach and the efficacy of our dataset.

We are in the process of conducting targeted qualitative evaluations to further elucidate the benefits of VQ-Fusion over baseline methods. The findings from these evaluations will be included in our revised submission, offering a comprehensive view of VQ-Fusion's effectiveness.

Reviewer comment (Experiments):
Studies of more powerful models like GPT-4V could be added as strong baseline, I am wondering if GPT-4V could solve this problem already?

Author response:
**Reviewer Comment:**
> Studies of more powerful models like GPT-4V could be added as strong baseline, I am wondering if GPT-4V could solve this problem already?

**Response:** Thank you for your suggestion. We appreciate your interest in exploring the capabilities of advanced models and their potential impact on our work. We understand the importance of benchmarking against state-of-the-art models to provide a comprehensive understanding of our proposed approach's performance. However, we faced practical constraints in evaluating our model against GPT-4V, particularly concerning the accessibility and cost associated with using such advanced models.

For research purposes, we focused on open-source models that offer greater accessibility to the research community. This approach aligns with our aim to facilitate reproducibility and wider adoption of our findings. In contrast, GPT-4V, while indeed a powerful model, involves significant costs for evaluation. Based on our estimates, a comprehensive comparison with GPT-4V would require approximately $3840. Such an expense is substantial, especially for research projects with limited funding.

Nevertheless, we recognize the value that such a comparison would add to our work. In response to your suggestion, we are planning to create a smaller, more focused evaluation set that will allow us to conduct a targeted comparison with GPT-4V. This approach will enable us to leverage the capabilities of GPT-4V while managing the associated costs more effectively. We will continually evaluate the very recent models and include them in our benchmark. We plan to release the dataset so that other researchers can also evaluate their model on our dataset and show the impact of those models across diverse tasks and domains.

Reviewer comment (Experiments):
The paper focuses on a synthesized dataset, which, while important for controlled experiments, may not fully capture the complexities and nuances of real-world interactions. This concern is partially due to the way view perspectives are included in the data. "The blender to the right of the book. From which perspective is the object described?" may not be something you'd commonly encounter in everyday (embodied) conversations/questions. Same for the example shown in Fig. 2 for RG.

Author response:
**Reviewer Comment:** 
> The paper focuses on a synthesized dataset, which, while important for controlled experiments, may not fully capture the complexities and nuances of real-world interactions. This concern is partially due to the way view perspectives are included in the data. "The blender to the right of the book. From which perspective is the object described?" may not be something you'd commonly encounter in everyday (embodied) conversations/questions. Same for the example shown in Fig. 2 for RG.

**Related Comment:**
> While the dataset introduced tackles an important embodied task, the execution, motivation, and realism of some of the subtasks could be better explained

**Response:** Our primary objective with this dataset is to systematically examine the impact of nonverbal interaction signals, along with verbal and visual perspectives, on embodied interaction reasoning. Our results, as detailed in the ablation study section of our manuscript (Section 7.2), clearly demonstrate the significance of nonverbal cues in enhancing model performance across various tasks. For instance, in the Object Grounding (OG) task, models trained on data with both gaze and pointing gestures exhibited an improvement in accuracy (68.61%) compared to models trained on data lacking nonverbal gestures (26.65%)​​. This trend was consistent across other tasks, reinforcing the importance of nonverbal cues in embodied question-answering contexts.

Regarding the design of tasks involving explicit instructions, such as "The blender to the right of the book. From which perspective is the object described?" we agree that these might not mimic everyday human interactions precisely. However, these tasks, inspired by prior work in spatial relations grounding (Goyal et al., 2020), are intentionally crafted to effectively evaluate models' capabilities in perspective understanding, a critical aspect in human-AI interaction. Our dataset also includes tasks that more closely resemble real-world interactions, like the Existence Prediction (e.g., “what is that left object?”) and Object Attribute Query (e.g., “What is the color of that back object?”) tasks, which ask about objects' existence or attributes in a scene. These tasks require the model to understand the perspective implicitly, which aligns with real-world interaction. This mix of explicit and implicit perspective-aware tasks aims to balance between effective model evaluation and mimicking real-world interactions.

**We appreciate your feedback and will ensure to clarify these points in the revised manuscript to better convey the rationale behind our task design and the relevance of our dataset in understanding embodied interactions.**

Additionally, we are extending our EQA-MX dataset to collect real-world human interactions to assess models for understanding embodied interactions in real-world settings.

[1] Goyal, Ankit, Kaiyu Yang, Dawei Yang, and Jia Deng. "Rel3d: A minimally contrastive benchmark for grounding spatial relations in 3d." Advances in Neural Information Processing Systems 33 (2020): 10514-10525.

Reviewer comment (Presentation):
Table 1 is a bit unclear, e.g., why is EQA not available for all datasets, when some of the datasets referred to in the table correspond embodied QA benchmarks (EQA, MT-EQA)? Also, to the best of my knowledge, some of the datasets have egocentric views and not exocentric ones.

Author response:
**Reviewer Comment:**
> Table 1 is a bit unclear, e.g., why is EQA not available for all datasets, when some of the datasets referred to in the table correspond embodied QA benchmarks (EQA, MT-EQA)? Also, to the best of my knowledge, some of the datasets have egocentric views and not exocentric ones.

**Related Comment:**
>The paper could benefit from providing a clearer differentiation between embodied and non-embodied works in the field.

**Related Comment:** 
> generalizability of the VQ-fusion approach raise concerns about its applicability to more advanced multimodal models that have been recently more prevalent in the E-AI literature. Additionally, the paper could benefit from providing a clearer differentiation between embodied and non-embodied works in the field, as well as a more comprehensive review and comparison with recent advancements in multimodal language models.

**Response:**
We acknowledge the confusion in Table 1 regarding the categorization of datasets as EQA. To clarify, in the paper (Introduction section (Page 2, para 1) and Figure 1), we distinguish between two types of embodied interactions. EQA can be bifurcated based on embodied interactions: the first centers on an agent, like a virtual robot, navigating to answer verbal questions (Das et al., 2018a), solely incorporating verbal queries. The second encompasses multimodal expressions, where humans interact with the environment using verbal utterances and gestures (Chen et al., 2021; Islam et al., 2022a). Adopting the latter definition, we designed EQA tasks to comprehend questions using multimodal expressions (verbal utterances and nonverbal gestures) in embodied settings, which has been explained in Section 1 on page 2. For instance, an EQA task may involve pointing to an object and asking, “What is that object?” requiring reasoning over multimodal expressions to answer the question.”

Our dataset, EQA-MX, is grounded in the latter, as it involves multimodal expressions for interaction. Buildin on the reviewer’s comment, we will revise the Table 1 to clearly present the distinction of emboedied and non-embodied works. We have revised Table 1 to delineate our work clearly from agent-perspective embodied QA tasks. We have also rectified the representations of egocentric and exocentric views for the relevant datasets, ensuring that our classifications are consistent with current literature.

These updates in the manuscript will explicitly address the distinctions between embodied and non-embodied works, aligning with the definitions provided on Page 2. We believe these changes will resolve the ambiguities and enhance the manuscript's clarity.

Reviewer comment (Novelty):
There is a body of work in pointing gesture recognition - most involving human communication in interactive embodied task-completion environments [2,3,4]. Similarly, there exists work that focuses on view perspectives [5]. It would be good to include and differentiate this work from the existing literature.

Author response:
**Reviewer Comment:**
>There is a body of work in pointing gesture recognition - most involving human communication in interactive embodied task-completion environments [2,3,4]. Similarly, the exists work that focuses on view perspectives [5]. It would be good to include and differentiate this work from the existing literature.

**Response:** 

**Differentiation from pointing gesture recognition Works [2, 3, 4]:** The works by Constantin et al. [2,4] and Wan et al. [3] focus on enhancing human-robot interaction by interpreting and following natural language instructions, considering physical and social contexts. For example, "HandMeThat" by Wan et al. involves understanding and acting upon human instructions with ambiguities, integrating physical states, human actions, and goals​​​​. Our EQA-MX framework, while considering non-verbal cues (like pointing gestures) and multiple-view perspectives, primarily focuses on embodied question-answering tasks in varied simulated environments, emphasizing the interpretation of combined verbal and non-verbal communication in AI models.

**Differentiation from view perspectives work [5]:** Shi and Yang's work on spatial and visual perspective-taking in embodied reference understanding differs significantly from our approach. Their research emphasizes view rotation and relation reasoning for understanding spatial and visual perspectives in embodied AI. In contrast, our EQA-MX dataset and VQ-Fusion model focus on a broader range of embodied interactions and multimodal expressions, including the interpretation of both verbal and non-verbal cues in AI models.

In conclusion, while our work shares some commonalities with these studies, particularly in the context of embodied AI and multimodal interactions, it is distinct in its comprehensive approach to EQA tasks and the specific challenges of fusing multimodal data for question answering.

Reviewer comment (Experiments):
While the reviewer appreciates the performance improvements of VQ-Fusion on 4 baselines, it would be beneficial to include more advanced baselines, such as a Multimodal Transformer model that extracts visual representations across all visual views and the verbal representations together, as well as more recent Multimodal LLMs (Otter, MiniGPT, UniLM, GPT4, InstructBLIP, LLaVA, etc.) that potentially can offer competitive fusion capabilities for these tasks.

Author response:
**Reviewer Comment:** 
> While the reviewer appreciates the performance improvements of VQ-Fusion on 4 baselines, it would be beneficial to include more advanced baselines, such as a Multimodal Transformer model that extracts visual representations across all visual views and the verbal representations together, as well as more recent Multimodal LLMs (Otter, MiniGPT, UniLM, GPT4, InstructBLIP, LLaVA, etc.) that potentially can offer competitive fusion capabilities for these tasks.


**Related Comment:** 
>Based on Table 5 and Table 3, results in Table 3 for VQ-Fusion models are reported with varying codebooks, tuned for each task. This can improve performance, but it also raises questions about the generalizability of the approach. In some cases, the worst-case setting of the number of codebooks (e.g., 4 codebooks for EP or PG) can hurt the performance of the baseline CLIP model, which might impact the model's ability to perform well on new unseen tasks.

**Related Comment:**
>The choice of baselines and the generalizability of the VQ-fusion approach raise concerns about its applicability to more advanced multimodal models that have been recently more prevalent in the E-AI literature

**Response:**

**Generalizability of VQ-Fusion:** Thank you for your insightful comments regarding the generalizability of our VQ-Fusion approach and its comparison with advanced multimodal models. 

Our approach with VQ-Fusion, as shown in Table 3 of our paper, indeed utilizes varying codebooks optimized for each task. This design choice was intended to explore the upper limits of performance enhancements achievable through task-specific tuning. While this approach demonstrates substantial gains in specific tasks, we understand the concerns it raises regarding the broader applicability and generalizability of our model. To address this, it’s important to highlight that the VQ-Fusion model, when applied to the HAMLET framework for human activity recognition tasks (as detailed in section 3.3 of our supplementary document), demonstrated performance improvements in F1-score (87.69%) over the baseline and other state-of-the-art methods. This finding is key in demonstrating the adaptability and effectiveness of VQ-Fusion in enhancing multimodal representation learning across different domains and datasets.

Further addressing generalizability, our exploratory analysis on multitask learning (as detailed in section 3.3 of our supplementary document), sheds light on how VQ-Fusion copes with the challenge of learning multiple tasks simultaneously. While there are trade-offs in performance due to the complex nature of multitask learning, these findings open avenues for future research to refine and enhance multimodal models for learning multiple tasks more effectively.

**Comparison with Advanced Multimodal Models:** The field of multimodal learning is indeed evolving rapidly, with more advanced multimodal models emerging. Our choice of baselines and the VQ-Fusion approach was driven by the necessity to establish a strong foundational understanding of how visual-linguistic representations can be enhanced through our proposed method. Our goal is to show that aligning visual and language representation structures can help to improve the multimodal representations of the existing models. This finding is very crucial and shows an important future direction to improve multimodal representation.

While we acknowledge that our approach might not encapsulate all the recent advancements of multimodal learning, it provides a significant step towards understanding and improving the fusion of multimodal representations. This foundational work is crucial for the development and evaluation of future, more advanced models. As incorporating our proposed VQ-Fusion requires meticulously studying the multimodal model architecture, some of the comparisons with more advanced multi-modal models are beyond the scope of current work. However, following the reviewer's suggestions, we are working to evaluate and benchmark the recent multimodal models on our dataset EQA-MX.

Reviewer comment (Experiments):
Based on Table 5 and Table 3, results in Table 3 for VQ-Fusion models are reported with varying codebooks, tuned for each task. This can improve performance, but it also raises questions about the generalizability of the approach. In some cases, the worst-case setting of the number of codebooks (e.g., 4 codebooks for EP or PG) can hurt the performance of the baseline CLIP model, which might impact the model's ability to perform well on new unseen tasks.

Author response:
**Reviewer Comment:** 
> While the reviewer appreciates the performance improvements of VQ-Fusion on 4 baselines, it would be beneficial to include more advanced baselines, such as a Multimodal Transformer model that extracts visual representations across all visual views and the verbal representations together, as well as more recent Multimodal LLMs (Otter, MiniGPT, UniLM, GPT4, InstructBLIP, LLaVA, etc.) that potentially can offer competitive fusion capabilities for these tasks.


**Related Comment:** 
>Based on Table 5 and Table 3, results in Table 3 for VQ-Fusion models are reported with varying codebooks, tuned for each task. This can improve performance, but it also raises questions about the generalizability of the approach. In some cases, the worst-case setting of the number of codebooks (e.g., 4 codebooks for EP or PG) can hurt the performance of the baseline CLIP model, which might impact the model's ability to perform well on new unseen tasks.

**Related Comment:**
>The choice of baselines and the generalizability of the VQ-fusion approach raise concerns about its applicability to more advanced multimodal models that have been recently more prevalent in the E-AI literature

**Response:**

**Generalizability of VQ-Fusion:** Thank you for your insightful comments regarding the generalizability of our VQ-Fusion approach and its comparison with advanced multimodal models. 

Our approach with VQ-Fusion, as shown in Table 3 of our paper, indeed utilizes varying codebooks optimized for each task. This design choice was intended to explore the upper limits of performance enhancements achievable through task-specific tuning. While this approach demonstrates substantial gains in specific tasks, we understand the concerns it raises regarding the broader applicability and generalizability of our model. To address this, it’s important to highlight that the VQ-Fusion model, when applied to the HAMLET framework for human activity recognition tasks (as detailed in section 3.3 of our supplementary document), demonstrated performance improvements in F1-score (87.69%) over the baseline and other state-of-the-art methods. This finding is key in demonstrating the adaptability and effectiveness of VQ-Fusion in enhancing multimodal representation learning across different domains and datasets.

Further addressing generalizability, our exploratory analysis on multitask learning (as detailed in section 3.3 of our supplementary document), sheds light on how VQ-Fusion copes with the challenge of learning multiple tasks simultaneously. While there are trade-offs in performance due to the complex nature of multitask learning, these findings open avenues for future research to refine and enhance multimodal models for learning multiple tasks more effectively.

**Comparison with Advanced Multimodal Models:** The field of multimodal learning is indeed evolving rapidly, with more advanced multimodal models emerging. Our choice of baselines and the VQ-Fusion approach was driven by the necessity to establish a strong foundational understanding of how visual-linguistic representations can be enhanced through our proposed method. Our goal is to show that aligning visual and language representation structures can help to improve the multimodal representations of the existing models. This finding is very crucial and shows an important future direction to improve multimodal representation.

While we acknowledge that our approach might not encapsulate all the recent advancements of multimodal learning, it provides a significant step towards understanding and improving the fusion of multimodal representations. This foundational work is crucial for the development and evaluation of future, more advanced models. As incorporating our proposed VQ-Fusion requires meticulously studying the multimodal model architecture, some of the comparisons with more advanced multi-modal models are beyond the scope of current work. However, following the reviewer's suggestions, we are working to evaluate and benchmark the recent multimodal models on our dataset EQA-MX.

Reviewer comment (Novelty):
While the dataset introduced tackles an important embodied task, the execution, motivation, and realism of some of the subtasks could be better explained. The choice of baselines and the generalizability of the VQ-fusion approach raise concerns about its applicability to more advanced multimodal models that have been recently more prevalent in the E-AI literature. Additionally, the paper could benefit from providing a clearer differentiation between embodied and non-embodied works in the field, as well as a more comprehensive review and comparison with recent advancements in multimodal language models.

Author response:
We are excited that the reviewer acknowledged our contribution that our proposed vector quantization-based fusion model effectively bridges the gap between discrete verbal and continuous visual representations and enhances the performance of baseline models across EQA tasks. The reviewer also acknowledged our extensive ablation studies, which explore various aspects such as the number of codebooks, the use of multiple views, the generalizability of VQ-fusion, and the impact of learning single versus multiple tasks. The reviewer highlights the paper's strength in presenting eight new EQA tasks that utilize multimodal expressions for understanding questions in embodied settings.

**Summary Response:**
- We presented new experiment results based on the reviewer’s comments, which shows the generalizability of VQ-Fusion across tasks and domains.
- We provided clearer differentiation between embodied and non-embodied works.
- We further clarified the categorization of embodied question-answering works.
- We provided a detailed clarification of the realism of perspective-aware task design
- We discussed the generalizability of VQ-Fusion and the choice of baseline multimodal models.
- We discussed how our work is different from some other works on pointing gesture recognition and view perspective, which are referred to by the reviewer.
- We addressed the resolution issues of Figure 3(a) and clarified the axes
- We clarified that all tasks include non-verbal gestures and fixed a typo
- We confirmed that the dataset split remained the same for all experiments; the only variable we altered was the presence or absence of non-verbal information, such as gaze and gestures.
- We provided references to works on performance degradation in multitask learning settings.

Reviewer comment (Experiments):
While the authors show the value of VQ in Table 3, the value of using multiple views has not been ablated. How much does having visual information from multiple views help in EQA? It seems obvious that it should, especially if some views are likely to have occlusions and ambiguities, while other are not, but it would have been nice to see its effect quantitatively.

Author response:
**Reviewer Comment:**

> While the authors show the value of VQ in Table 3, the value of using multiple views has not been ablated. How much does having visual information from multiple views help in EQA? It seems obvious that it should, especially if some views are likely to have occlusions and ambiguities while other are not, but it would have been nice to see its effect quantitatively.

**Response:** Thank you very much for pointing out the importance of conducting an evaluation by varying the visual views. We conducted the experimentation by ablating the visual views, which are included in the supplementary (Section 3.1, Table 1). We varied the visual perspectives during training and testing by using different camera views (ego, exo, and top) to capture the nonverbal interactions. We used the CLIP model to learn EQA tasks involving verbal utterances and visual views. The results suggest that models trained using multiple visual perspectives perform better than models trained using a single visual perspective (Table 1: Bottom). The reasoning behind this performance improvement is that models using multiple visual views can learn generalized multiview representations, which can improve the performance at inference time when visual views are varied. Additionally, as the reviewer pointed out, multiple visual views provide complementary view representations that aid in extracting comprehensive information to address occlusion and ambiguities in the scene and thus improve the performance across tasks.