In this task, you will be presented with a context from an academic paper and you have to write an answerable question based on the context. Your questions can be extractive, abstractive, or yes-no questions.

Furthermore, we employ two different parsers for comparison: 1) the PCFGLA-based parser, viz. Berkeley parser BIBREF5 , and 2) a minimal span-based neural parser BIBREF6 .
What is the baseline model for the agreement-based mode?

Dataset Quality Analysis ::: Inter-Annotator Agreement (IAA)
To estimate dataset consistency across different annotations, we measure F1 using our UA metric with 5 generators per predicate. Dataset Quality Analysis ::: Dataset Assessment and Comparison
We assess both our gold standard set and the recent Dense set against an integrated expert annotated sample of 100 predicates. Dataset Quality Analysis ::: Agreement with PropBank Data
It is illuminating to observe the agreement between QA-SRL and PropBank (CoNLL-2009) annotations BIBREF7.
How was quality measured?

To better exploit such existing data sources, we propose an end-to-end (E2E) model based on pointer networks with attention, which can be trained end-to-end on the input/output pairs of human IE tasks, without requiring token-level annotations. Since our model does not need token-level labels, we create an E2E version of each data set without token-level labels by chunking the BIO-labeled words and using the labels as fields to extract.
Do they assume sentence-level supervision?