Venue: EMNLP 2020 Workshop NLP-COVID

Reviews:

final_score: rating: 5: Marginally below acceptance threshold
review_content: The paper introduces a large-scale dataset with posts related to COVID-19 collected from Weibo, a social media in China. It describes in detail the process to collect the dataset and provides some interesting analysis of the dataset. 

Pros:  
* The dataset  is large and contains rich information such as location and post time. This could be helpful for further COVID-related researches. 
* This paper provides interesting analysis about the dataset such as the daily distribution of COVID-related posts and word clouds at different phases.  

Cons:  
* There are no deduplication steps to deduplicate tweets in the dataset. The keyword-based collection could result in many duplicated tweets, for example, the same announcements from the government reported by lots of different news accounts, and popular tweets retweeted by other accounts. The duplicated tweets could bias the statistics of the dataset and further studies.   
* While the main contribution of this paper is a large-scale dataset, there is no direct evaluation of its quality. It is unknown how many tweets in this dataset are directly related to COVID-19. For example, some keywords (e.g. person and location) could potentially pick up non-COVID-related posts (e.g. searching "Li Wenliang" could pick up tweets that falsely accused him of spreading rumors or tweets that described his profile).  Another example is that a portion of words from Fig.4c and Fig.4d are not directly related to COVID-19, such as "汇算 清缴"(settlement and payment), "梵高 画作"(paintings of Vincent Van Goph) and ”后街男孩“(Backstreet Boys).  The precision of this keyword-based approach to collect COVID-related tweets remains unknown, which makes the dataset less useful to the community. 

Suggestion:  
* Some Chinese keywords do not correctly match their translation in English in the appendix table .  For example, "美国 AND 例" should be translated to "USA AND cases " instead of "USA COV-19 AND cases". 



review_title: A large-scale dataset that may be useful for further studies on COVID-19

final_score: rating: 6: Marginally above acceptance threshold
review_content: This is a short paper providing an overview of the collection of tweets posted on a popular Chinese social media platform called Weibo from December 1, 2019 to April 30, 2020. The authors first create a pool of active users by filtering them based on their activity and the number of tweets, followers, and fans. They then compile a collection of tweets from these users using 179 manually pre-defined keywords.
Several questions and comments:
- Why was the data collected only until the end of April? The authors claim that the pool of active users is dynamically maintained and if so it would be better to provide both the users and the readers with a more recent version of the dataset. 
- In what languages are the tweets collected in Weibo-COV? I assume the majority is in Chinese, this should be stated explicitly somewhere at the beginning of the paper. 
- What does crawling time mean in this context? Please either specify in Table 1 or add the definition to the main text. 
- Not sure what is meant by “Cirque du Soleil in Canada” in section 3.5, please review.
- Typos and grammar mistakes occur throughout the text (e.g. “N95 musk”). It would be nice to correct those to improve the readability of the paper. 
review_title:  A short paper introducing a collection of covid-19-related tweets from Weibo

final_score: rating: 6: Marginally above acceptance threshold
review_content: This paper describes a dataset for COVID-19 built from Weibo, a Chinese social media platform. 
The authors first maintain a Weibo active user pool -- around 20 million active users, and then collect their tweets using a list of specified keywords relating to COVID-19 (179 in total).

Strengths：
- A large scale dataset, containing tweet content, interactive, location and retweet information. Not only text-based but also social network-based analysis can be potentially conducted on the dataset.
- The authors describe in detail how they build the dataset

Weaknesses:
- I am not sure whether collecting only active user's tweets makes sense. It is possible that some users create and start to post tweets because of (after) the pandemic. It may be worth analysing those users as well.

Questions and suggestions to authors:
- Are these keywords manually choosen or decided based on their popularity? Also it looks to me some keywords are repeated. For example, is the result filtered using keyword 'Doctor AND Li Wenliang' a strict subset of result using keyword 'Li Wenliang'?
- The title of (Chen et al, 2020) paper has been updated to 'Tracking Social Media Discourse About the COVID-19 Pandemic: Development of a Public Coronavirus Twitter Data Set'
review_title: A useful resource paper