instruction:
In this task, you will be presented with a context from an academic paper and you have to write an answerable question based on the context. Your questions can be extractive, abstractive, or yes-no questions.
question:
Both systems were optimized on the tst2014 using Minimum error rate training BIBREF20 . A detailed description of the systems can be found in BIBREF21 .
answer:
How is the PBMT system trained?

question:
Paraphrase Identification (PI) is the task of determining whether two sentences are paraphrase or not. We have implemented the cross-lingual variant of kernel functions for PI and RE tasks as described in section SECREF3 and measured the accuracy of models by testing them on the parallel data set.
answer:
What classification task was used to evaluate the cross-lingual adaptation method described in this work?

question:
To control the quality, we ensured that a single annotator annotates maximum 120 headlines (this protects the annotators from reading too many news headlines and from dominating the annotations). Secondly, we let only annotators who geographically reside in the U.S. contribute to the task.

We test the annotators on a set of $1,100$ test questions for the first phase (about 10% of the data) and 500 for the second phase. Annotators were required to pass 95%.
answer:
How is quality of annotation measured?