Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: - The paper considers the problem of video visual relation detection under an open-vocabulary scenario.

- The main contribution is on prompt learning to adapt the strong visual-language model towards downstream tasks.

- To enable the open-vocabulary, the prompt learning has exploited two techniques, namely, the compositional prompt representation, and motion-based prompt groups, which makes sense.

- The idea has been evaluated on various datasets, and show clear performance improvements.
strength_and_weaknesses: Strength

- The idea is simple, using compositional prompt and exploiting motion information do generate prompt do make sense in the video visual relation detection task.

- The results have clearly validate the claims, demonstrating state-of-the-art performance under various settings.

Weakness:

- The writing is not satisfactory, it can surely be better.

- Missing many references from ECCV: 
[1] https://arxiv.org/abs/2112.04478
[2] https://arxiv.org/pdf/2208.03550
[3] https://arxiv.org/abs/2208.02816

clarity,_quality,_novelty_and_reproducibility: Clarity : OK

Quality: good

Novelty: good

Reproducibility: the writing is not ideal, which makes it a bit difficult to reproduce, unless the author will release the code.
summary_of_the_review: Overall, I think I like this paper, I like the motivation for solving downstream video tasks with prompt learning, the idea of using composition prompt generation for predicate is good, and exploiting motions makes sense in video understanding tasks.

However, the paper writing needs a lot improvement, now it's very hard to grasp the key ideas, too many notions, and unnecessary equations.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The paper introduces a new task called Open-vocabulary VidVRD, where the aim is to detect not only unseen relationships but also unseen objects and predicates. They propose a novel pipeline (i.e. RePro) to detect tracklets and classify the relations between tracklets. In particular, the proposed approach is a compositional and motion-based relation prompt learning framework. In the experiments, the authors show rich experiments, including various ablation studies. 
strength_and_weaknesses: **Strengths:**

* The paper is well written. 

* The authors have introduced an interesting problem and proposed a novel solution for the problem. Further, they showed rich experimental results. 

**Weaknesses:**

* What is the difference between visual embeddings ($\mathbf{v}_i^O$) and RoI Aligned features ($\mathbf{f}_i$) in Sections 3.1 and 3.2?  

* Figure 3 mix-uses texts (e.g. "dog", "child") for both visual features and categories. 

* The experimental settings in Section 4.4 is not clear. Did the authors use the same tracklets for the comparisons in Tables 3 and 4? This reviewer is unsure if the comparisons are valid. 

* Typos:
    * "we distil the knowledge ..." in Line 172


clarity,_quality,_novelty_and_reproducibility: **Clarity, Quality, Novelty:** 
The paper is well-written, and it shows enough novelty. 

**Reproducibility:**
The authors added a separate section to discuss reproducibility. They also provide the code, but this reviewer hasn't checked the code.

summary_of_the_review: The paper introduces an interesting task and a framework to solve the task.
Some parts are still unclear, and this reviewer left the questions in the Weaknesses section.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper proposes a new approach to video visual relation detection that is based on extension of compositional prompt tuning with motion cues. The Relation Prompt (RePro) is designed to address the technical challenges of Open-vocabulary Video Visual Relation Detection: the prompt tokens should respect the two different semantic roles of subject and object while the tuning should account for the diverse spatiotemporal motion patterns of the subject-object compositions. They achieves a new state-of-the-art performance on two VidVRD benchmarks of not only the base training object and predicate categories, but also the unseen ones.
strength_and_weaknesses: Strength
- They proposes a novel setting for video visual relation detection task, Open-VidVRD.
- They clearly describe interesting idea in this paper. The compositional prompt representation learning method that models the prompt contexts for subject and object separately. the motion-cue-based prompt groups shows good generalization ability on the tasks.

Weakness
- The novelty is limited. The paper claims that the first one to research on the the open vocabulary video visual relation detection, but the open vocabulary VRD has been widely studied in visual domain, which potentially could be compared.
- The motivations in evaluation section, and ablations of the proposed approach is not easy to follow.
It's also hard to follow how the proposed approach improves the performance on the Open-VidVRD benchmark. 
clarity,_quality,_novelty_and_reproducibility: - The main idea is interesting and the work is well-written and easy to follow. 
- The proposed method is clearly described and the results has potential impact.

- They partially shared their code, and hyper parameter settings. The results are reproducible.

summary_of_the_review: This paper proposes a new method for several technical challenges of Open-vocabulary Video Visual Relation Detection, and also provides useful prompt group that can be generalized. The paper shows the video visual relation detection is generalizable with the proposed framework. The proposed video visual relation detection task is very interesting, but I would appreciate if the author can clarify the evaluation and motivation behind the design. Overall, I lean toward acceptance.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper presents an interesting prompt tuning method for video relation detection. It includes compositional and motion related cues. 

The authors also propose a new task, called Open VIdVRD, which further considers unseen relationship combinations and unseen objects/motions.
strength_and_weaknesses: Strengths:
1. The authors propose to add the composition capability and motion cues to the prompt, making it more robust and less biased for video relation detection. The idea is new.
2. The design of multi-mode (multiple different motion patterns) prompt groups is interesting.
3. The paper is well-written and easy to follow. 
4. The authors conducted several experiments to verify the effectiveness of their method. It shows comparable results when comparing to VidVRD SOTA. In their ablation study, their open-vocabulary-based results are better than the baselines.

Weaknesses:
1. The motion pattern design seems to be complex and not easy to handle a variety of motions. While the use of GIoU is nice, the current design considered only toward/away and near/far. It is difficult to extend to generic motions with GIoU. Since the current design of motion group is very limited, the strong performance improvements are less convincing to be attributed to motion cues.
2. In the comparison with traditional VidVRD, one may wonder the performance of the proposed model if adding novel samples.

clarity,_quality,_novelty_and_reproducibility: The proposed learnable compositional prompt is new and interesting.
summary_of_the_review: Overall the proposed method is novel and interesting. However, the design of motion cues seems limited in current stage. It still requires hand-craft effort or expert knowledge.

The introduced task is a good addition to VidVRD dataset. Results on open-vocabulary setting are good compared to baseline variants.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: All four reviewers agreed that the paper makes good contributions with 1) a novel and challenging task of open-vocabulary visual relation detection in videos, 2) a novel prompt learning scheme that leverages compositional and motion-related priors, and 3) strong empirical results supporting the efficacy of the proposed approach. 

The reviewers raised some clarity issues, which were well addressed during the rebuttal. They also expressed some concerns about experiments and provided good suggestions on additional experiments to further improve the quality of the paper. The authors provided additional results in the rebuttal which addressed the initial concerns.

Overall, there is a consensus that this paper has enough contributions to appear at the conference. The proposed task is novel and challenging, the proposed approach is reasonable, and experimental results are convincing to support their claims. Given this, we are happy to recommend acceptance.
justification_for_why_not_higher_score: Although the rebuttal have successfully addressed reviewers' initial concerns, the reviewers did not find the revision and the rebuttal particularly strong enough to warrant higher scores. As a result, although they all provided explicit endorsement for acceptance, they kept their initial ratings (6). This meta reviewer has carefully read the paper, the reviews, the rebuttal, and the discussion threads, and did not find strong reasons to upvote beyond the reviewers' suggestions. 
justification_for_why_not_lower_score: The paper has enough merits to appear at the conference (see above).
Decision: Accept: poster