In this task, you will be presented with a context from an academic paper and a question separated with a
. You have to answer the question based on the context.
One example: We evaluate the proposed approach on the Chinese social media text summarization task, based on the sequence-to-sequence model. Large-Scale Chinese Short Text Summarization Dataset (LCSTS) is constructed by BIBREF1 . The dataset consists of more than 2.4 million text-summary pairs in total, constructed from a famous Chinese social media microblogging service Weibo.
Question: Are results reported only for English data?
Solution is here: No
Explanation: Based on the context, the dataset is constructed from a famous Chinese social media microblogging service Weibo.

Now, solve this: The selection model selects the best answer from the set $\lbrace a_i
brace _{i=1}^N$ observed during the interaction by predicting the difference of the F1 score to the average F1 of all variants. We train another neural network to pick the best answer from the candidates. We frame the task as binary classification, distinguishing between above and below average performance. In training, we compute the F1 score of the answer for every instance. If the rewrite produces an answer with an F1 score greater than the average score of the other rewrites the instance is assigned a positive label.
Question: how are multiple answers from multiple reformulated questions aggregated?
Solution:
The selection model selects the best answer from the set $\lbrace a_i
brace _{i=1}^N$ observed during the interaction by predicting the difference of the F1 score to the average F1 of all variants.