In this task, you will be presented with a context from an academic paper and a question separated with a 
. You have to answer the question based on the context.

[EX Q]: For DQA four participants answered each question, therefore we took the average INLINEFORM0 , INLINEFORM1 , and INLINEFORM2 values over the four evaluators as the result per question. The detailed answers by the participants and available online. To assess the correctness of the answers given both by participants in the DQA experiments, and by the QALD system, we use the classic information retrieval metrics of precision (P), recall (R), and F1. INLINEFORM0 measures the fraction of relevant (correct) answer (items) given versus all answers (answer items) given. 
 Question: Do they test performance of their approaches using human judgements?
[EX A]: Yes

[EX Q]: We propose to derive rewards from human-human dialogues by assigning positive values to contextualised responses seen in the data, and negative values to randomly chosen responses due to lacking coherence (also referred to as `non-human-like responses') – see example in Tables TABREF29 and TABREF30 . 
 Question: How do they obtain human generated policies?
[EX A]: derive rewards from human-human dialogues by assigning positive values to contextualised responses seen in the data, and negative values to randomly chosen responses due to lacking coherence

[EX Q]: In the second round, we collected 293 annotations from 12 annotators. After Korektor, there are 4262 unique sentences (including 150 seed sentences) that form the COSTRA 1.0 dataset. 
 Question: How are possible sentence transformations represented in dataset, as new sentences?
[EX A]:
Yes, as new sentences.