TASK DEFINITION: In this task, you will be presented with a context from an academic paper and a question separated with a
. You have to answer the question based on the context.
PROBLEM: To study the lexical and semantic diversities of responses, we performed three analyses. First, we aggregated all worker responses to a particular question into a single list corresponding to that question. Second, we compared the diversity of individual responses between Control and AUI for each question. To measure diversity for a question, we computed the number of responses divided by the number of unique responses to that question. We call this the response density.
Question: How was lexical diversity measured?

SOLUTION: By computing number of unique responses and number of responses divided by the number of unique responses to that question for each of the questions

PROBLEM: After removing 135 questions during the validation, the final dataset comprises 13,939 questions, 3,827 of which require commonsense knowledge (i.e. 27.4%).
Question: what is the size of their dataset?

SOLUTION: 13,939

PROBLEM: Both systems were optimized on the tst2014 using Minimum error rate training BIBREF20 . A detailed description of the systems can be found in BIBREF21 .
Question: How is the PBMT system trained?

SOLUTION:
systems were optimized on the tst2014 using Minimum error rate training BIBREF20