Expanding Reference from Textual to Visual: A New Benchmark for Pixel-Level Visual Entity Linking
Authors: Zhengfei Xu, Sijia Zhao, Qingbin Liu, Xiaolong Liu, Lilili, Bo Li, Xi Chen, Dianbo Sui, Xin Xin


Reviewer paper summaries: In multi-object scenarios, it is very difficult for traditional VEL tasks to accurately refer to the visual object of interest using text queries. In such cases, using visual input to refer to specific objects would be easier. In this paper, A new task called Pixel-Level Visual Entity Linking (PL-VEL) is proposed: the input is image & mask & bbox, and the output is the corresponding Wikipedia entity label; 2. A new benchmark dataset called MaskOVEN-Wiki is introduced, containing about 5 million automatically annotated entries; 3. A visual instruction tuning method based on visual semantic tokenization is proposed. Experiments are conducted on the propsed datasets and the results are reported.
Summary of strengths: 1. The motivation is interesting. The proposed new task PL-VEL can solve the problems of traditional VEL tasks in complex scenarios, which is closer to real-world application scenarios and is an interesting direction. 
2. The newly proposed large-scale dataset could be useful for promoting research in this field. The paper provides a detailed data analysis of the new dataset and conducts experiments to verify the effectiveness of the proposed method.
Summary of weaknesses: 1. In the practical application scenario of PL-VEL, it may be less convenient for users to input a fine-grained mask than to input text. At the same time, why not retain the original text queries in the OV-WIKI dataset and form a multimodal dataset with the newly annotated masks? Retaining the text data may help improve model performance. Just as stated by the authors, the performance would be limited by not providing the textual inputs. This seems contridicts with the original motivations. 
  
2. The method descriped in the Data Filtering section is not clear enough. How can the graphics methods ensure that segmentation errors are definitely excluded, and will there be a manual review?

3. Some experimental details are not provided, such as GPU and training time.
Comments, suggestions and typos: See weakness.
Overall assessment scores: 2.5