In this task, you will be presented with a context from an academic paper and a question separated with a 
. You have to answer the question based on the context.

E2E NLG challenge Dataset: The training set of the E2E challenge dataset which consists of 42K samples was partitioned into a 10K paired and 32K unpaired datasets by a random process.  The Wikipedia Company Dataset: The Wikipedia company dataset presented in Section SECREF18 was filtered to contain only companies having abstracts of at least 7 words and at most 105 words. The dataset was then divided into: a training set (35K), a development set (4.3K) and a test set (4.3K). The training set was also partitioned in order to obtain the paired and unpaired datasets.  
 Question: What non-annotated datasets are considered?
E2E NLG challenge Dataset The Wikipedia Company Dataset

Wikilinks can be seen as a large-scale, naturally-occurring, crowd-sourced dataset where thousands of human annotators provide ground truths for mentions of interest. This means that the dataset contains various kinds of noise, especially due to incoherent contexts. We prepare our dataset from the local-context version of Wikilinks, and resolve ground-truth links using a Wikipedia dump from April 2016. We use the page and redirect tables for resolution, and keep the database pageid column as a unique identifier for Wikipedia entities. We discard mentions where the ground-truth could not be resolved (only 3% of mentions). 
 Question: How was a quality control performed so that the text is noisy but the annotations are accurate?
The authors believe that the Wikilinks corpus  contains ground truth annotations while being noisy. They discard mentions that cannot have ground-truth verified by comparison with Wikipedia.

We evaluate the performance of our proposed method using two criteria: i) rank-correlation BIBREF25 to evaluate visual grounding and ii) accuracy to evaluate question answering. Intuitively, rank-correlation measures the similarity between human and model attention maps under a rank-based metric. A high rank-correlation means that the model is `looking at' image areas that agree to the visual information used by a human to answer the same question. In terms of accuracy of a predicted answer INLINEFORM0 is evaluated by: DISPLAYFORM0

 
 Question: How do they measure the correlation between manual groundings and model generated ones?

rank-correlation BIBREF25