In this task, you will be presented with a context from an academic paper and a question separated with a 
. You have to answer the question based on the context.

Q: Despite these shortcomings in using WER as a metric of success, the WER can reflect our model's effectiveness in generating questions that are similar to those of SQuAD based on the given reading passage and answer. WER can be used for initial analyses that can lead to deeper insights as discussed further below. 
 Question: Why did they choose WER as evaluation metric?

A: WER can reflect our model's effectiveness in generating questions that are similar to those of SQuAD WER can be used for initial analyses
****
Q:  To support this claim, we measure our system's performance for datasets across various domains. The evaluations are done by managed judges who understands the knowledge base and then judge user queries relevance to the QA pairs (binary labels). Each query-QA pair is judged by two judges. We filter out data for which judges do not agree on the label. Chit-chat in itself can be considered as a domain. Thus, we evaluate performance on given KB both with and without chit-chat data (last two rows in Table TABREF19), as well as performance on just chit-chat data (2nd row in Table TABREF19). 
 Question: What experiments do the authors present to validate their system?

A:  we measure our system's performance for datasets across various domains evaluations are done by managed judges who understands the knowledge base and then judge user queries relevance to the QA pairs
****
Q: In particular, we aggregate documents from the CommonCrawl dataset that has the most overlapping n-grams with the questions. We name this dataset STORIES since most of the constituent documents take the form of a story with long chain of coherent events. Figure 5 -left and middle show that STORIES always yield the highest accuracy for both types of input processing. 
 Question: Which of their training domains improves performance the most?

A:
documents from the CommonCrawl dataset that has the most overlapping n-grams with the question
****