Teacher:In this task, you will be presented with a context from an academic paper and a question separated with a
. You have to answer the question based on the context.
Teacher: Now, understand the problem? Solve this instance: The selection model selects the best answer from the set $\lbrace a_i
brace _{i=1}^N$ observed during the interaction by predicting the difference of the F1 score to the average F1 of all variants. We train another neural network to pick the best answer from the candidates. We frame the task as binary classification, distinguishing between above and below average performance. In training, we compute the F1 score of the answer for every instance. If the rewrite produces an answer with an F1 score greater than the average score of the other rewrites the instance is assigned a positive label.
Question: how are multiple answers from multiple reformulated questions aggregated?
Student:
The selection model selects the best answer from the set $\lbrace a_i
brace _{i=1}^N$ observed during the interaction by predicting the difference of the F1 score to the average F1 of all variants.