In this task, you will be presented with a context from an academic paper and a question separated with a
. You have to answer the question based on the context.
--------
Question: We train language models on two languages: One model is estimated on the German newscrawl distributed by WMT'18 comprising 260M sentences or 6B tokens. Another model is trained on the English newscrawl data comprising 193M sentences or 5B tokens. We learn a joint Byte-Pair-Encoding (BPE; Sennrich et al., 2016) vocabulary of 37K types on the German and English newscrawl and train the language models with this vocabulary. We consider two benchmarks: Most experiments are run on the WMT'18 English-German (en-de) news translation task and we validate our findings on the WMT'18 English-Turkish (en-tr) news task. For WMT'18 English-German, the training corpus consists of all available bitext excluding the ParaCrawl corpus and we remove sentences longer than 250 tokens as well as sentence-pairs with a source/target length ratio exceeding 1.5. This results in 5.18M sentence pairs. We tokenize all data with the Moses tokenizer BIBREF8 and apply the BPE vocabulary learned on the monolingual corpora.
Question: What dataset do they use?

Answer: German newscrawl distributed by WMT'18 English newscrawl data WMT'18 English-German (en-de) news translation task WMT'18 English-Turkish (en-tr) news task

Question: Last, we evaluate our approaches in 9 commonly used text classification datasets. We evaluate our methods on several commonly used datasets whose themes range from sentiment, web-page, science to medical and healthcare.
Question: What NLP tasks do they consider?

Answer: text classification for themes including sentiment, web-page, science, medical and healthcare

Question: We designed 3 evaluation sets: (1) Base Set (1,264 samples) held out from the simulated data. (2) Augmented Set (1,280 samples) built by adding two out-of-distribution symptoms, with corresponding dialogue contents and queries, to the Base Set ("bleeding" and "cold", which never appeared in training data). (3) Real-World Set (944 samples) manually delineated from the the symptom checking portions (approximately 4 hours) of real-world dialogues, and annotated as evaluation samples.
Question: How do they select instances to their hold-out test set?

Answer:
1264 instances from simulated data, 1280 instances by adding two out-of-distribution symptoms and 944 instances manually delineated from the symptom checking portions of real-word dialogues