In this task, you will be presented with a context from an academic paper and a question separated with a 
. You have to answer the question based on the context.

For DQA four participants answered each question, therefore we took the average INLINEFORM0 , INLINEFORM1 , and INLINEFORM2 values over the four evaluators as the result per question. 
 Question: How do they measure performance?
average INLINEFORM0 , INLINEFORM1 , and INLINEFORM2 values

We propose to derive rewards from human-human dialogues by assigning positive values to contextualised responses seen in the data, and negative values to randomly chosen responses due to lacking coherence (also referred to as `non-human-like responses') – see example in Tables TABREF29 and TABREF30 . 
 Question: How do they obtain human generated policies?
derive rewards from human-human dialogues by assigning positive values to contextualised responses seen in the data, and negative values to randomly chosen responses due to lacking coherence

For the benchmarks, we selected five systems. We picked first the langid.py library which is frequently used to compare systems in the literature. Since our work is in neural-network LID, we selected two neural network systems from the literature, specifically the encoder-decoder EquiLID system of BIBREF6 and the GRU neural network LanideNN system of BIBREF10. Finally, we included CLD2 and CLD3, two implementations of the Naïve Bayes LID software used by Google in their Chrome web browser BIBREF4, BIBREF0, BIBREF8 and sometimes used as a comparison system in the LID literature BIBREF7, BIBREF6, BIBREF8, BIBREF2, BIBREF10. 
 Question: Which existing language ID systems are tested?

langid.py library encoder-decoder EquiLID system GRU neural network LanideNN system CLD2 CLD3