Venue: NeurIPS 2021 Track Datasets_and_Benchmarks Round2

Reviews:

final_score: rating: 6: Marginally above acceptance threshold
review_content: summary and contributions: This paper presents a new text spotting dataset, which is the largest in scale to-date. Furthermore, a new end-to-end transformer model is proposed for video text spotting. This dataset contains English and Chinese.
strengths: - large scale dataset
- include multiple languages, which promotes diversity.
- new end-to-end transformer model inspired by DETR as a strong baseline

weaknesses: - only has 2 languages (English, Chinese)

In L58, Arabic is mentioned but I don't find anywhere discussing it so I assume this dataset is Chinese+English. To claim this as a multilingual dataset, I would suggest including more languages, especially those with non-Latin characters


correctness: I don't find severe issues in constructing this dataset.
clarity: Paper is well written.
relation to prior work: Related works are properly discussed.
additional feedback: N/A
review_title: Official Review

final_score: rating: 6: Marginally above acceptance threshold
review_content: summary and contributions: This paper proposes a large-scale open-world video text dataset (MOVText). Compared with previous datasets, it contains more frames and more scenarios. It also contains text types annotation and multilingual text annotation. This dataset can be used for multiple text video tasks including text detection, text recognition, video text tracking, and end-to-end text spotting in Videos. The authors provide detailed information on the data collection and data annotation process, as well as the dataset statistics and comparison with previous datasets. This paper also proposes an end-to-end text spotting framework named TransVTSpotter. Experiments on different tasks are conducted and the results of different methods are reported.
strengths: 1. The open-world large-scale video text dataset has not been proposed before. It could potentially inspire and benefit future work in this direction.
2. The authors provide detailed information and analysis on the dataset, including the data collection and annotation process, data statistics, and evaluation tasks and metrics. Experiments are conducted on different methods and different tasks.
3. This paper is clearly written and easy to follow.
weaknesses: 1. Some of the dataset statistics are missing. For example, how many types of language are there in the dataset? What are the percentages of each language? What are the percentages of each scenario?
2. As the dataset paper, the method part is a bit unnecessary. I would suggest the authors to put less weight on the proposed method and more emphasis on the dataset and benchmarking previous methods and baselines.
correctness: The claims made in the submission are correct.
clarity: The paper is well written.
relation to prior work: It clearly discussed how this work differs from previous datasets in this field.
additional feedback: N/A.
review_title: A large-scale open-world video text dataset

final_score: rating: 5: Marginally below acceptance threshold
review_content: summary and contributions: The authors propose a new large scale dataset (MOVText) for video text spotting. Because of rich annotations the dataset lends itself well to other video text tasks like detection, recognition and tracking as well. It is multi lingual and covers various open-world scenarios. The authors also propose a transformer based method for end to end video text spotting which achieves state of the art performance on MOVText.
strengths: 1. Good coverage of related work
2. Good comparison with other datasets
3. MOVText introduces certain challenges which were not posed by other existing datasets
weaknesses: 1. The language is unclear and grammatically incorrect at way too many places making the paper difficult to understand
2. I is not clear what exactly authors mean by "incidental text". Is it same as the scene text? Or is it a generic term for scene text, titles and captions taken together?
3. What is meant by "open" in the 30+ "open categories" and how is it depicted in Figure 1 a?
4. The task of video text spotting should have been first defined the Introduction
5. It is not clear whether the models are trained using each video as a training example or the sub sequences of frames from the videos as training examples? How are these sequences formed?
6. Line 61-65 is not clear
7. Frequent use of incomplete, bullet-point-like sentences has hampered the readability of the text
8. Line 122-124: Please elaborate how the temporal processing is helping the task of text spotting? 
9. It is not clear from the paper about the license or copy right information of the videos downloaded from YouTube and KuaiShou. Also, what exactly is a "text video class"? Does it mean those videos which have a lot of text in them in addition to the captions?
10. Line 134: what do you mean by "ordinary videos" here?
11. The caption of Fig 3 needs to be re-written to properly highlight the four components
12. Is the dataset actually multi-lingual or bi-lingual with English and Chinese as the two languages for text?

correctness: The details seem to be correct except for my reservations due to the points mentioned under "weaknesses" above. 
clarity: No. The language is unclear and grammatically incorrect at way too many places making the paper difficult to understand. Frequent use of incomplete, bullet-point-like sentences has also hampered the readability of the text. The organization of Section 5 must be improved to better highlight the impact.
relation to prior work: Yes.
additional feedback: Captured in "Weaknesses" above.
review_title: Review

final_score: rating: 7: Good paper, accept
review_content: summary and contributions: This paper presents a video text spotting benchmark MOVText that are multilingual, composed of open-world videos from 30+ categories and annotated with rich text type annotations. A simple yet efficient video text spotting model is also introduced, that achieves state-of-the-art performance on ICDAR2015 (video) dataset, and establishes a strong baseline on MOVText.
strengths: - A large scale video text spotting datasets that are (1) multilingual,  (2) composed of open-world videos from 30+ categories (many are not covered in existing datasets) and  (3) annotated with rich text type annotations. 
- Compared with previous datasets, MOVText is a few magnitude larger and well-annotated. 
- The dataset supports more diverse evaluation tasks.
- Abundant analysis on the dataset are presented.
- The authors also showcased its potential to not only be a decent video text spotting benchmark but also to extend to video-and-language tasks. 
- The proposed baseline model achieve competitive performance on existing dataset and establishes a strong baseline on the proposed dataset.

weaknesses: No major weakness. Please refer to the sections below (Clarity/Documentation/Additional Feedback) for some detailed questions I have. 
correctness: I believe the dataset is constructed in a sound way. The evaluation methods and experiment design also seems appropriate and performed correctly to me. 
clarity: Overall, the paper is clear. Below are some small issues or suggestions.
- TransSpotter or TransVTSpotter or Video TransSpotter? Please unify the model name for better clarity. 
- Section 5.1 mentioned that TransSpotter is first pre-trained on COCOText then finetuned on video-text datasets, but the other baselines are trained in different ways (L255-257). Then wouldn't it be a unfair comparison between TansSpotter and other models?
- L200, it might be more clear to include one sentence describing which errors can MOT help to pinpoint its source. 
- L134, how are the two rounds of screening conducted? with a pre-trained model or by human annotators?
relation to prior work: Clearly discussed. 
- Section 2 and section 3.2 compares MOVText against previous video-text spotting datasets and highlights MOVText's contribution as (1) large-scale (2) multilingual (3) open-world videos (4) rich text type annotations.
- L59-L65 also compares the proposed TransVTSpotter with previous methods. TransVTSpotter requires less steps hence more efficient while achieving better accuracy. 
additional feedback: About text type `caption text`.
- In real-life applications, what is the use case that we will need to use video-text spotting methods to get the caption texts? Wouldn't ASR be easier?
- In some cases, there could be `caption text` in videos without audio. Would it be more useful to annotate caption texts that are not extractable by ASR differently?
review_title: Review

Meta-review: The reviews are mostly positive. The dataset is much larger than existing ones and is likely to be a valuable resource. The authors have addressed the concerns satisfactorily. I am happy to recommend acceptance.
Decision: Accept