TASK DEFINITION: In this task, you will be presented with a context from an academic paper and a question separated with a 
. You have to answer the question based on the context.
PROBLEM: Table TABREF46 shows that our Open model achieves more than 3 points of f1-score than the state-of-the-art result, and RelAwe with DepPath&RelPath achieves the best in both Closed and Open settings. 
 Question: How big is improvement over the old  state-of-the-art performance on CoNLL-2009 dataset?

SOLUTION: our Open model achieves more than 3 points of f1-score than the state-of-the-art result

PROBLEM: To model an answer which is a collection of spans, the multi-span head uses the $\mathtt {BIO}$ tagging format BIBREF8: $\mathtt {B}$ is used to mark the beginning of a span, $\mathtt {I}$ is used to mark the inside of a span and $\mathtt {O}$ is used to mark tokens not included in a span. In this way, we get a sequence of chunks that can be decoded to a final answer - a collection of spans. 
 Question: How they use sequence tagging to answer multi-span questions?

SOLUTION: To model an answer which is a collection of spans, the multi-span head uses the $\mathtt {BIO}$ tagging format BIBREF8: $\mathtt {B}$ is used to mark the beginning of a span, $\mathtt {I}$ is used to mark the inside of a span and $\mathtt {O}$ is used to mark tokens not included in a span

PROBLEM: We carried out two human evaluations using Mechanical Turk to compare the performance of our model and the baseline. 
 Question: Is there any human evaluation involved in evaluating this famework?

SOLUTION:
Yes