TASK DEFINITION: In this task, you will be presented with a context from an academic paper and a question separated with a
. You have to answer the question based on the context.
PROBLEM: We collect utterances from the $\mathbf {C}$hinese $\mathbf {A}$rtificial $\mathbf {I}$ntelligence $\mathbf {S}$peakers (CAIS), and annotate them with slot tags and intent labels. The training, validation and test sets are split by the distribution of intents, where detailed statistics are provided in the supplementary material. Since the utterances are collected from speaker systems in the real world, intent labels are partial to the PlayMusic option. We adopt the BIOES tagging scheme for slots instead of the BIO2 used in the ATIS, since previous studies have highlighted meaningful improvements with this scheme BIBREF30 in the sequence labeling field.
Question: What is the domain of their collected corpus?

SOLUTION: speaker systems in the real world

PROBLEM: E2E NLG challenge Dataset: The training set of the E2E challenge dataset which consists of 42K samples was partitioned into a 10K paired and 32K unpaired datasets by a random process. The Wikipedia Company Dataset: The Wikipedia company dataset presented in Section SECREF18 was filtered to contain only companies having abstracts of at least 7 words and at most 105 words. The dataset was then divided into: a training set (35K), a development set (4.3K) and a test set (4.3K). The training set was also partitioned in order to obtain the paired and unpaired datasets.
Question: What non-annotated datasets are considered?

SOLUTION: E2E NLG challenge Dataset The Wikipedia Company Dataset

PROBLEM: The performances of the NER experiments are reported separately for three different parts of the system proposed. Results shown that, in the first case, in the training set the F1 score is always greater than 97%, with a maximum of 99.65%. With both test sets performances decrease, varying between 94-97%. In the case of UGC, comparing the F1 score we can observe how performances significantly decrease.
Question: What are their results on the entity recognition task?

SOLUTION:
With both test sets performances decrease, varying between 94-97%