Venue: ICLR 2023 TinyPapers

Reviews:

final_score: rating: High Potential (HP): a submission which meets the reviewing criteria and has potential to make an impact on the field
review_content: summary_of_contributions: This paper reframes visual referring expressions as a language reasoning task, comparing the abilities of various large pretrained language models (LLMs), particularly Bing Chat and ChatGPT, to perform referring expressions in a zero-shot setting. The evaluation on the RefCOCO/g dataset demonstrates the potential of LLMs for reasoning in vision tasks.
strengths_and_weaknesses: Strengths:
1. The paper introduces a novel approach to referring expression comprehension (ReC) by reframing it as a language reasoning task, utilizing LLMs to address the problem.
2. The experimental results are compelling, as the authors compare the performance of Bing Chat, ChatGPT, and reCLIP on the RefCOCO test set, providing valuable insights into their respective capabilities.

Weaknesses:
1. The paper could benefit from a more detailed analysis of the ChatRef model, particularly exploring whether the performance is constrained by the regionCLIP module, which would help elucidate the model's limitations.
2. While the authors provide prompts in the appendix, there is no mention of whether the code will be made available.
suggested_changes: pleas see the weakness 2
review_title: null

final_score: rating: High Potential (HP): a submission which meets the reviewing criteria and has potential to make an impact on the field
review_content: summary_of_contributions: This paper introduces the idea of using Large Language Models (LLMs) to improve the comprehension of referring expressions. This in turn ca help improve reasoning in vision tasks which is an important aspect for human-robot interaction.
strengths_and_weaknesses: Strengths:
- Clear problem definition and discussion of related work that motivates the new approach.
- The work is reproducible since the details of the approach has been shared in the appendix.
- The examples shared in the paper are motivating examples.

Weakness:
- The practical limitations of the chat limit etc. using fewer examples for comparison makes sense. However, the accuracy numbers should not be established on such a small comparison set.
suggested_changes: Based on the weakness described above the authors should consider running the experiment on a larger eval set (possibly RefCOCO/g full val set) and then report accuracy numbers.
review_title: null

final_score: rating: Clear, Correct, and Reproducible (CCR): a submission which meets the reviewing criteria
review_content: summary_of_contributions: Visual referring expressions comprehension (ReC) is a sub-task of visual grounding that aims to localize an object in a visual scene given a referring expression. ChatRef achieved an accuracy of 75% on sampled datasets, higher than ReCLIP's accuracy of around 50%-60%.
strengths_and_weaknesses: ## Strengths:

1. ChatRef decouples visual perception and language reasoning, offering human-like zero-shot generalization ability and implicit concept hierarchy
2. ChatRef achieved higher accuracy than ReCLIP on sampled RefCOCO/g images.

## Weakness:

1. ChatRef's performance heavily relies on the accuracy of object detection and attribute estimation
2. ChatGPT performed unstably on referring expression comprehension, even with heavy prompt engineering
3. The evaluation was only done through manual input on small datasets, limiting the generalizability of the results.
suggested_changes: There is a scope to improve the quality of the paper by editing English/ removing a few grammatical errors. 

For example,  In "Given a visual scene, ChatRef first recognize the objects and their attributes, including category, location, and color, and then feeds these textual descriptions and referring expressions to an LLM to
locate the target object.", please consider changing "Given a visual scene, ChatRef first recognizes the objects and their attributes, including category, location, and color, and then feed these textual descriptions and refer expressions to an LLM to locate the target object."
review_title: null

Decision: Invite to present