In this task, you will be presented with a context from an academic paper and a question separated with a
. You have to answer the question based on the context.

[EX Q]: We published a survey on Reddit asking Danish speaking users to suggest offensive, sexist, and racist terms for a lexicon. Language and user behaviour varies between platforms, so the goal is to capture platform-specific terms. This gave 113 offensive and hateful terms which were used to find offensive comments. The remainder of comments in the corpus were shuffled and a subset of this corpus was then used to fill the remainder of the final dataset. The resulting dataset contains 3600 user-generated comments, 800 from Ekstra Bladet on Facebook, 1400 from r/DANMAG and 1400 from r/Denmark.
Question: How large was the dataset of Danish comments?
[EX A]: 3600 user-generated comments

[EX Q]: The dataset includes about 6,000 triples, comprised of videos, questions, and answer spans manually collected from screencast tutorial videos with spoken narratives for a photo-editing software.
Question: What kind of instructional videos are in the dataset?
[EX A]: tutorial videos for a photo-editing software

[EX Q]: A growing body of evidence shows that state-of-the-art models learn to exploit spurious statistical patterns in datasets BIBREF12, BIBREF13, BIBREF14, BIBREF15, BIBREF16, BIBREF17, instead of learning meaning in the flexible and generalizable way that humans do. Given this, human annotatorsbe they seasoned NLP researchers or non-expertsmight easily be able to construct examples that expose model brittleness.
Question: What are the weaknesses found by non-expert annotators of current state-of-the-art NLI models?
[EX A]:
state-of-the-art models learn to exploit spurious statistical patterns in datasets human annotatorsbe they seasoned NLP researchers or non-expertsmight easily be able to construct examples that expose model brittleness