In this task, you will be presented with a context from an academic paper and you have to write an answerable question based on the context. Your questions can be extractive, abstractive, or yes-no questions.

Input: Consider Input: The graphs in figure 1 show the distances between duplicate and non-duplicate questions using different embedding systems. Large achieved a F1 score of 0.71 for this task without any specific training, simply by choosing a threshold below which all sentence pairs are considered duplicates. In order to test whether these results generalised to our domain, we devised a test that would make use of what little data we had to evaluate.

Output: How is the accuracy of the system measured?


Input: Consider Input: The Hausa data used in this paper is part of the LORELEI language pack. It consists of Broad Operational Language Translation (BOLT) data gathered from news sites, forums, weblogs, Wikipedia articles and twitter messages. We use a split of 10k training and 1k test instances. The Yorùbá NER data used in this work is the annotated corpus of Global Voices news articles recently released by BIBREF22. The dataset consists of 1,101 sentences (26k tokens) divided into 709 training sentences, 113 validation sentences and 279 test sentences based on 65%/10%/25% split ratio.

Output: How much labeled data is available for these two languages?


Input: Consider Input: Below, we explore different ways to back off when the ScRNN predicts UNK (a frequent outcome for rare and unseen words):

Pass-through: word-recognizer passes on the (possibly misspelled) word as is.

Backoff to neutral word: Alternatively, noting that passing $\colorbox {gray!20}{\texttt {UNK}}$ -predicted words through unchanged exposes the downstream model to potentially corrupted text, we consider backing off to a neutral word like `a', which has a similar distribution across classes.

Backoff to background model: We also consider falling back upon a more generic word recognition model trained upon a larger, less-specialized corpus whenever the foreground word recognition model predicts UNK. Figure 1 depicts this scenario pictorially.


Output: How do the backoff strategies work?