Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: 1. The paper attempts to introduce domain generalization to solve query-based video segmentation.
2. The paper proposes QFA and AM-AdaIN modules to process query and video information, respectively.
3. The authors conduct experiments on the A2D Sentences, Refer-Youtube-VOS (RVOS), and J-HMDB Sentences datasets.
strength_and_weaknesses: Strength:
1. The research question is interesting.
2. The author did some experimental comparisons, and the results show that the proposed method has improved on some baselines.

Weaknesses:

1 The paper experiment is not sufficient.

-a. Does Table 1 compare other video segmentation methods? Does Table 1 compare with other domain adaption methods? Why does the title here compare with other generation methods? 

-b. From Table 2, the improvement of each module is minimal. Combining Tables 1 and 2, I doubt whether the motivation for introducing domain adaption is reasonable. I'm concerned that the author is simply doing a combination of methods and ignoring the rationale of the research motivation.

2. Insufficient analysis of the rationality of domain adaption. 

-a. Does the paper have visualization results of domain adaption? I want to know what domain adaption means specifically for video segmentation tasks.

3. The details of the method of the thesis are not clear. 

-a. How is Equation 1 trained? How does it relate to other loss functions (Equations 6 and 11)? 

-b. In Figure 2, what is the output of the augmented feature? Where did he enter the part of a? 

-c. Is the domain adaption of this paper merely from data augmentation?



clarity,_quality,_novelty_and_reproducibility: 1. Clarity:
Some details are not clear.
2. Quality:
It can also be improved.
3. Novelty:
A bit innovative.
4. Reproducibility:
Kind of difficult to reproduce.

summary_of_the_review: The paper lacks analysis and experimentation on research motivation. Some details of the method are unclear. I hope the author answers my question in the response. I'll change the score based on the author's reply.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper studies cross-modal domain gereralization on the query-based video-segmentation task. Two modules, i.e., Query-guided Feature Augmentation (QFA) and attention map adaptive instance normalization (AM-AdaIN) are proposed for the multi-modal generalization task. Experiments are conducted on three domain generalization tasks for query-based video segmentation.
strength_and_weaknesses: Strengths:
1. The studied domain generalization task, i.e., multi-modal generalization for query-based video segmentation, is novel and has not been investigated before. The idea is interesting.
2. Two modules are designed to address this challenging task, i.e. QFA and AM-AdaIN. This method achieves better results than the compared baslines. Ablation studies seem effective and sufficient.
3. The presentation is generally good.

Weaknesses:
1. Motivation is not clear enough. Some source-only (baseline) examples would better show the requirement of domain generalization for query-based video segmentation.
2. The technical contribution is relatively limited. Most components, e.g. both loss functions, are based on existing techniques.
3. I expected there are at leaset two source domains so that domain-invariant representations can be learned. I do not understand why only one source domain is used. This is also domain generalization but not the typical setting.
4. Baselines and some implementation details are not clear enough. For example, the compared baselines are mainly single-modal based methods. How to use them for the multi-modal generalization task?
5. The analysis on why the proposed method performs better in the experiment part is weak.
6. 

clarity,_quality,_novelty_and_reproducibility: The task is novel, but the technical contribution is incremental, resulting limited novelty and originality. The proposed method is clearly presented in general. Some implementation details, especially how the baselines are implemented for multi-modal generalization setting, are not given. The code is not provided. I doubt exact reproducibility.
summary_of_the_review: Novel domain generalization task but limited technical contrbution, good experimental results but insufficient analysis and insights behind the results, and generally fluent presentation.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This work addresses the task of multi-modal DG where each modality has to cope with its own domain shift. Specifically, the specific scenario of query-based video segmentation is studied to better advance the generalization ability of the model in the multi-modal situation. The authors observe that actions belonging to the same type of actors in different domains share similar representations in the query-video latent space, and the main reason for the performance degradation is that visual background and contextual complexity vary widely across different domains. The authors propose a Query-guided Feature Augmentation (QFA) module that uses the attention scores between query and video frames to distinguish query-related foreground regions from unrelated background regions for synthesizing novel visual features. AdapIN is adopted to remove the impact of style in attention map during training, and gradually introduce statistics from other samples to help the model learn robust cross-modal relationships. Extensive experiments validate that the proposed model has better generalization ability.
strength_and_weaknesses: Strength:
1.	This work addresses multi-modal DG problem in query-based video segmentation task, which is still an untouched field. Multi-modal DG is meaningful and has its specific challenge. The authors’ effort on handling the multi-modal DG problem is encouraged.
2.	The authors observe that actions belonging to the same type of actors in different domains share similar representations in the query-video latent space, and the main reason for the performance degradation is that visual background and contextual complexity vary widely across different domains. Such observation is helpful.
3.	The proposed QFA and AM-AdaIN are reasonable.
4.	Extensive experiments validate the effectiveness of the proposed model.

Weakness:
1.	The performances are over-claimed that “From the results we can see that our method outperforms the baseline model by a large margin on all three generalization tasks, demonstrating the effectiveness of our proposed QFA and AM-AdaIN modules.” In 4.3 Main result. The performance of A2R in Table I is just comparable to existing methods.
2.	The proposed AM-AdaIN module is not illustrated in Fig. 2. It is hard to understand how it works in the proposed method.
3.	Can we replace the L_const into triplet loss, rather than Moco. Besides, the \tau in L_const is not addressed.
4.	Some typos:
a)	Bold “c ” in c_{hw}^k of \sigma(C_k)
b)	C’_K -> C’_k in Eqn. 9
c)	Inconsistence between A2R and A2D Sentences to RVOSpart in Table I and Table II.


clarity,_quality,_novelty_and_reproducibility: The paper is well written and easy to follow. The novelty of this work is enough.
summary_of_the_review: Please see the Strength and Weakness.

---------

After reading the author response and other fellow reviewers' comments, I see that some concerns have been addressed, while others still remains unsolved. Based on the technical content and the author response, I decide to revise the score.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: This work aims at improving query-based video segmentation based on domain generalization. Specifically, this work focuses on solving the challenges of cross-modal settings, where domain shift exists at both the text and visual levels. To mitigate these challenges, this work proposes the query-guided feature augmentation method and uses AdaIN to modulate the text query.
strength_and_weaknesses: Strength
+ A good attempt to improve domain generalization of the query-based video segmentation methods.
+ The experimental results show that the proposed method outperforms the competing methods.

Weaknesses
+ This work proposes to augment training data with the proposed QFA. Only the background information is replaced during the augmentation. However, the foreground information also varies significantly for complex environments. For instance, as shown in Figure 1, the targets in the source and target domains have different scales. 
+ I am still confused about the detailed motivation for using AdaIN. It is originally used in style transfer, and this work seems to use it to mitigate the “style” of the query. However, AdaIN requires supervised training to present different styles, while the target domain data is not available during training for this task.
+ The technical novelty of this paper is not up to the standard of ICLR. I believe that model performance is (marginally) improved but combining a data augmentation method and AdaIN and then applying them to the segmentation task is not appealing to me.
+ The presentation should be improved. For instance, in the first sentence of the introduction, it seems that “byGavrilyuk” should be separated into two worlds by spacing. Similarly, “AdapINHuang” in the last paragraph of the introduction.

clarity,_quality,_novelty_and_reproducibility: Clarity: Needs Improvement.
Quality: Needs Improvement.
Novelty: Somewhat novel.
Reproducibility: Reproducible.

summary_of_the_review: This work views query-based segmentation as a cross-domain generalization problem. It proposes a data augmentation scheme and uses AdaIN to improve model performance. The technical novelty of this paper is not up to the standard, and the motivations are not clear enough. In addition, the presentation needs further improvements.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: This paper studies domain generalization in query-based video segmentation, which aims to segment
the queried actors or objects in video based on the given natural language query. The reviewers find the task  novel. At the same time, they also feel that the technical contributions are limited. 

justification_for_why_not_higher_score: technical contributions  limited
justification_for_why_not_lower_score: NA
Decision: Reject