TASK DEFINITION: In this task, you will be presented with a context from an academic paper and you have to write an answerable question based on the context. Your questions can be extractive, abstractive, or yes-no questions.
PROBLEM: The training objective is to minimize the negative likelihood of the aligned unanswerable question $ ilde{q}$ given the answerable question $q$ and its corresponding paragraph $p$ that contains the answer $a$ : L=-(q,q,p,a)DP(q|q,p,a;) where $\mathcal {D}$ is the training corpus and $ heta $ denotes all the parameters. Sequence-to-sequence and pair-to-sequence models are trained with the same objective.

SOLUTION: What is the training objective of their pair-to-sequence model?

PROBLEM: We introduce a new model for difficulty prediction combining learned representations induced via a pre-trained 'universal' sentence encoder BIBREF6 , and a sentence encoder learned from scratch for this task. In total, this resulted in 57,505 and 2,428 sentences in the train and test set abstracts, respectively.

SOLUTION: How much data is needed to train the task-specific encoder?

PROBLEM: We compare the performance of translation approaches based on four metrics:

[align=left,leftmargin=0em,labelsep=0.4em,font=]

As in BIBREF20 , EM is 1 if a predicted plan matches exactly the ground truth; otherwise it is 0.

The harmonic average of the precision and recall over all the test set BIBREF26 .

The minimum number of insertions, deletions or swap operations required to transform a predicted sequence of behaviors into the ground truth sequence BIBREF27 .

GM is 1 if a predicted plan reaches the ground truth destination (even if the full sequence of behaviors does not match exactly the ground truth). Otherwise, GM is 0.

SOLUTION:
What evaluation metrics are used?