Starting from its first case, confirmed on December 31 in Wuhan [9] , the novel coronavirus has surged into a world phenomenon rapidly. On January 30, the World Health Organization (WHO) has declared its outbreak as a global emergency [7] . As of October 13, the COVID-19 outbreak has caused over 3.7 million confirmed cases and over 1 million deaths worldwide. 1 To combat the epidemic, maintaining social distance has been considered effective. In turn, working and studying from home has become a new trend. With the Chen Yang, Xinyi Zhou, Reza Zafarani Data Lab, Department of Electrical Engineering and Computer Science, Syracuse University E-mail: cyang03@syr.edu, zhouxinyi@data.syr.edu, reza@data.syr.edu decrease in physical social contacts and the rise of anxiety on the pandemic, the frequency of social media usage has increased. The COVID-19 outbreak as an international public health emergency is closely connected with individuals' health and lives. Any news or information about the COVID-19 or a potential cure highly attracts public attention and influences social media. Therefore, it is of crucial importance to ensure information spread on COVID-19 is credible.
With more information on COVID-19, people gain a deeper understanding. To that end, a number of COVID-related datasets have been released and studied. Existing datasets have contributed to collecting either (i) Chinese COVID data without identifying news credibility (e.g., see [5] and [4] ); or (ii) English COVID data for news credibility (e.g., see [11] and [2] ). Therefore, we are motivated to build a dataset which contains data from Chinese social media and includes ground-truth labels (i.e., true/false).
Weibo (weibo.com), as a platform for information sharing, dissemination, and acquisition based on user relations, is one of the most popular social media in China. According to Weibo's first-quarter earnings report for 2020, 2 Weibo passed 500 million monthly active users and 200 million daily active users in March. Due to the large number of active users, we considered Weibo as one of the most used Chinese social media for people to access information related to COVID-19.
We develop CHECKED, the first Chinese COVID-19 social media dataset with 344 "Fake" microblogs and 1776 "Real" microblogs from December 2019 to August 2020. The main contributions of this work are:
1. We introduce the first fact-checked Chinese COVID-19 social media dataset, which enables more research on tracing the spread of microblogs misinformation and on analyzing content patterns in COVID-19 fake news. 2. We contribute the dataset with a rich set of features on microblogs related to COVID-19. We collect textual, visual, and video information as well as network information, containing spreading patterns of a microblog. 3. We conduct comprehensive experiments to analyze CHECKED data. Data and codes are public (see https://github.com/cyang03/CHECKED), where we hash the information that can identify users for privacy protection.
The rest of the paper is organized as follows. We review related datasets in Section 2. We explain the data collection process in Section 3. We design and analyze the dataset in Section 4. We conclude in Section 5.
Our related work can be organized into (I) social media datasets on COVID-19; and (II) COVID-19 datasets for news credibility.
I. COVID-19 Social Media Datasets Social media can provide a wealth of information, especially during the pandemic. Thus, a number of social media datasets on COVID-19 have emerged. Chen et al. [1] released the first COVID-19 dataset collected from Twitter, tracking the information related to coronavirus from January 2020 till the present with continuous updates. In addition to English, there are a few social media COVID-19 datasets in Chinese (e.g., see [5] and [4] ). Weibo-COV [5] is a large-scale COVID-19 dataset with 40 million microblogs in Chinese from December 2019 to April 2020. The dataset provides textual information, geographical information, and response information. NAIST-COVID [4] is a large-scale multilingual COVID-19 dataset which consists of English and Japanese microblogs from Twitter and Weibo. The dataset consists of 16 million microblogs in English, 9 million microblogs in Japanese, and 180 thousand microblogs in Chinese from January 20 to March 24. The dataset contains the IDs of microblogs and tweets, dates, and keywords.
While these datasets are large, they do not provide visual information or labels on news credibility.
With the spread of COVID-19, related rumors and fake news related also spread [13] . Thus, constructing a COVID-19 dataset with labels on news credibility is invaluable for research. ReCOVery dataset [11] contains over 2000 news articles on COVID-19 and 140 thousand tweets revealing the spreading articles on Twitter from January to May in 2020. The articles in the dataset are labeled as reliable and unreliable by using news publishers' reliability from news rating websites. The dataset includes both textual and visual information, as well as network information. CoAID dataset [2] includes 3252 news articles and 851 microblogs on Twitter about COVID-19 from December 2019 to July 2020. The ground truth labels of news articles are obtained by fact-checking websites and reliable news websites. The dataset collects the URLs of news articles and the ids of tweets and Twitter users. However, both ReCOVery and CoAID are in English.
Comparing with datasets mentioned above, CHECKED is the first COVID-19 dataset with ground truth labels in Chinese. CHECKED includes both textual and visual information, as well as the label of news credibility and propagation information in terms of comments and reposts.
We detail the data collection method which answers the following questions: (I) Where can we obtain the ground-truth labels (real or fake) on news events?; (II) How can we identify whether a news event is relevant to the coronavirus or not?; (III) What time intervals should be considered to achieve an extensive yet efficient search coverage?; and (IV) How can the information on news events (meta-data and its spread on Weibo) be tracked and stored in the dataset.
Center, 3 an official service where users can report either a microblog that (i) contains false information, (ii) releases user privacy without permission, (iii) has evidence of cyberbullying, and (iv) shows plagiarism; or belongs to a user who (v) impersonates as someone else. Experts in charge are then involved in verifying and ultimately, share their detailed evaluation of the reports at the platform publicly. This official service has helped verify over two million microblogs and remove tens of thousands of mis/disinformation. Our fake news collection focuses on these microblogs, which having been reported and labeled as false information in the Weibo Community Management Center.
I-2 Collecting Real News. To collect credible COVID-19 microblogs, we rely on two official reports, the (I) "2019 White Paper on the Social Value of Chinese Online Medium" 45 and the (II) "Research Report on the Public Awareness and Information Dissemination of COVID-19" 6 , provided by the State Information Center in (Administration Center of China E-government Network), the public institution directly affiliated to the National Development and Reform Commission (NDRC). Both reports provide rankings based on different criteria for Chinese online media. Specifically, -In the white paper, domain experts evaluate 24 Chinese major online media based on eight primary criteria and 28 secondary criteria. The evaluation covers aspects of the (i) quality and diversity in platform construction;
(ii) social influence including the platform popularity; (iii) activity; (v) reputation in the field and among online users; and (vi) how the medium contributes to charity. -The COVID-19 research report ranks the performance of online media during the pandemic. The ranking data is obtained through ∼3,000 valid surveys taken by online users, which focuses on the platform content trustworthiness, communication capacity, and social responsibility.
We select the Weibo account, People's Daily, 7 to collect real news. As China's largest newspaper group, People's Daily is ranked first in both reports, with over 120 million followers and over 120,000 microblogs on Weibo.
II. Identifying Microblog Topics. After a broad investigation, we finally select a total of 38 keywords to determine whether a microblog is relevant to COVID-19 or not. All the identified keywords are listed in few, as commonly-used professional terms (e.g., N95), are in English (6 keywords). English keywords are case-insensitive. A microblog containing any of the keywords in our list is regarded as relevant to COVID-19.
III. Collection Interval. For an unbiased data collection, we collected both real and fake microblogs (news events) from December 2019, when the coronavirus was first identified in Wuhan, Heibei, China [6] , to August 2020 (nine months).
Given sources for real and fake news, keywords, and data collection time interval for coronavirus pandemic, we collect microblogs by performing web scraping based BeautifulSoup library (for document parsing and information extraction), 9 . Each microblog is represented by the following components:
id: each microblog is identified by a unique 16 digit ID assigned by Weibo.
label: the label of each microblog is either "real" or "fake".
date: the time microblog is posted in yyyy-mm-dd hh:mm format.
user id: a unique 10 digit ID number assigned to the user by Weibo. Each user can change his or her ID only once to a string formed by 4-20 characters, in which letters are allowed. user name: the display name of the Weibo user that is unique.
text: contains the textual information of a microblog.
pic url: the URL for the visual information of the microblog; Users are allowed to attach no more than 18 images to each microblog. Such visual information has been recently developed for multimodal fake news detection [12, 10, 3] . video url: this URL for the video information of the microblog. Each microblog (i) can only include at most one video; and (ii) cannot contain both a video and an image. comment num, repost num, and like num: the numbers of comments, forwards, and likes for a microblog, respectively. comments: detailed information on user comments for each microblog. The information includes the ID, date, and content of comments (microblogs), and the ID and name of commenters (users). For each comment, no more than one image and no video are allowed. Such comments can provide rich information of user opinions, which has benefited explainable fake news detection [8] .
reposts: detailed information on user forwards for each microblog: the ID, date, content of forwards (microblogs), and the ID and name of forwarders (users). Similar to comments, each forward has at most one image and no video information. If a user forwards a repost with an image, the pic url of the new forward will also include this image along with the original image.
We illustrate a real and fake microblog collected in our dataset in Figure  1 . Note that we hash the IDs of both microblogs and users to protect users' privacy; we do not publicly release the user name and only release these hashed IDs rather than their original ones. Finally, we point out that, due to Weibo's access restriction and editable visibility, it is possible that, for a few microblogs, their number of comments (forwards) in comment (repost), which indicates the number visible on Weibo, is less than that of comments (forwards) in comment num (repost num), which captures the actual number.
The statistics of CHECKED dataset is presented in Table 2 . Next, we analyze the collected data in terms of (I) textual information, (II) visual information, and (III) propagation and responses of collected microblogs.
We first present the distribution of our keywords within the collected microblogs in Figure 2 ; the keywords help iden-tify if a microblog is relevant to COVID-19. We observe that the Chinese keywords are more frequently used than the English ones; this is reasonable as Weibo mostly targets Chinese users who might not be multilingual.
On the other hand, Figure 3 presents the word cloud of collected microblogs for all words in addition to the keywords. The top-ten vocabularies with the highest frequencies are "病例(case)" (#=2919), "确诊(confirmed case)" (#=2158), "新冠((abbr.) COVID-19)" (#=1827), "疫情(pandemic/epidemic)" (#=1701), "新冠肺炎(COVID-19)" (#=1649), "新增(new case)" (#=1331), "美国(United State)" (#=1227), "检测(test)" (#=923), and "中国(China)" (#=866). We observe that most of these vocabularies are included in our keyword list, while a few are not. These are not directly related to COVID-19, e.g., United States and China.
Finally, in Figure 4 , we plot the distribution of the number of words within collected microblogs, which share an average (median) number of 216 (154). We point out that each microblog is restricted to up to 140 words for all Weibo users before version 6.0.0, after which the restriction has been lifted for VIP users, e.g., People's Daily; it explains why some microblogs in our data have over 140 words. Figure 5 presents the distribution of the posting time of our collected microblogs. We observe that a very few microblogs related to COVID-19 were posted in December 2019 as the first case of the coronavirus was confirmed at the end of this month [9] . We notice that the number of COVID-19 microblogs (or say, online discussion) significantly increases since February 2020, which is also the month with the highest number of confirmed cases of the coronavirus. 1 As for images attached within microblogs, we observe from Figure 6 that most collected microblogs contain no more than 9 images. A few microblogs have more than nine images because posting over 9 images within a microblog has been allowed recently, as a new version 9.9.3 of Weibo.
We track the influence of [real and fake] microblogs on the coronavirus, as well as how they propagate on Weibo from three perspectives: (i) comments, (ii) forwards, and (iii) likes; their corresponding distributions for our data are provided in Figures 7-9 , which all follow a power-law-like distribution with a long tail. We observe that (i) in general, ∼80% microblogs have less than 2,000 comments, 2,000 forwards, and/or 20,000 likes; (ii) the most influential microblog can have over 20,000 comments, one million forwards, and/or one million likes; and (iii) all collected microblogs share an average (median) frequencies of being commented, forwarded, and liked of 1,785 (744), 3,482 (586), and 26,851 (6,179).
We present CHECKED, a Chinese COVID-19 dataset containing fact-checked microblogs from Weibo. The dataset includes a total of 2120 microblogs from December 2019 to August 2020. Each microblog contains ground-truth labels, textual, visual, and propagation information. The dataset can be extended (i) in scale, by increasing the number of fact-checked microblogs and (ii) in diversity, by involving the relationship among Weibo users. We hope this dataset can promote research on COVID-19 fake news.