TASK DEFINITION: In this task, you will be presented with a context from an academic paper and a question separated with a 
. You have to answer the question based on the context.
PROBLEM: The second data batch consists of event-related tweets for five natural disasters occurring in the U.S. in 2018. These are: the East Coast Bomb Cyclone (Jan. 2 - 6); the Mendocino, California wildfires (Jul. 27 - Sept. 18); Hurricane Florence (Aug. 31 - Sept. 19); Hurricane Michael (Oct. 7 - 16); and the California Camp Fires (Nov. 8 - 25).  
 Question: Which five natural disasters were examined?

SOLUTION: the East Coast Bomb Cyclone  the Mendocino, California wildfires Hurricane Florence Hurricane Michael the California Camp Fires

PROBLEM: Our dataset is annotated based on Chinese pathology reports provided by the Department of Gastrointestinal Surgery, Ruijin Hospital. It contains 17,833 sentences, 826,987 characters and 2,714 question-answer pairs.  
 Question: How big is QA-CTS task dataset?

SOLUTION: 17,833 sentences, 826,987 characters and 2,714 question-answer pairs

PROBLEM: We identified some limitations during the process, which we describe in this section.

When deciding publisher partisanship, the number of people from whom we computed the score was small. For example, de Stentor is estimated to reach 275K readers each day on its official website. Deciding the audience leaning from 55 samples was subject to sampling bias. Besides, the scores differ very little between publishers. None of the publishers had an absolute score higher than 1, meaning that even the most partisan publisher was only slightly partisan. Deciding which publishers we consider as partisan and which not is thus not very reliable.

The article-level annotation task was not as well-defined as on a crowdsourcing platform. We included the questions as part of an existing survey and didn't want to create much burden to the annotators. Therefore, we did not provide long descriptive text that explained how a person should annotate an article. We thus run under the risk of annotator bias. This is one of the reasons for a low inter-rater agreement. 
 Question: What limitations are mentioned?

SOLUTION:
deciding publisher partisanship, risk annotator bias because of short description text provided to annotators