TASK DEFINITION: In this task, you will be presented with a context from an academic paper and a question separated with a
. You have to answer the question based on the context.
PROBLEM: We evaluated our models on 3 different datasets:

CSAT dataset for CSAT prediction, consisting of spoken transcripts (automatic via ASR).

20 newsgroups for topic identification task, consisting of written text;

Fisher Phase 1 corpus for topic identification task, consisting of spoken transcripts (manual);
Question: What datasets did they use for evaluation?

SOLUTION: CSAT dataset 20 newsgroups Fisher Phase 1 corpus

PROBLEM: In order to represent individual sentences, we insert external [cls] tokens at the start of each sentence, and each [cls] symbol collects features for the sentence preceding it. We also use interval segment embeddings to distinguish multiple sentences within a document. This way, document representations are learned hierarchically where lower Transformer layers represent adjacent sentences, while higher layers, in combination with self-attention, represent multi-sentence discourse. Position embeddings in the original Bert model have a maximum length of 512; we overcome this limitation by adding more position embeddings that are initialized randomly and fine-tuned with other parameters in the encoder.
Question: What is novel about their document-level encoder?

SOLUTION: Bert model have a maximum length of 512; we overcome this limitation by adding more position embeddings we insert external [cls] tokens at the start of each sentence, and each [cls] symbol collects features for the sentence preceding it document representations are learned hierarchically

PROBLEM: For instance, BIBREF16 and BIBREF17 showed that a hypothesis-only baseline performs better than chance due to cues on their lexical choice and sentence length. Similarly, BIBREF18 showed that NLI models tend to predict entailment for sentence pairs with a high lexical overlap.
Question: What are examples of these artifacts?

SOLUTION:
hypothesis-only baseline performs better than chance due to cues on their lexical choice and sentence length NLI models tend to predict entailment for sentence pairs with a high lexical overlap