In this task, you will be presented with a context from an academic paper and a question separated with a
. You have to answer the question based on the context.

Input: Consider Input: Various multilingual extensions of NMT have already been proposed in the literature. The authors of BIBREF18 , BIBREF19 apply multitask learning to train models for multiple languages. Zoph and Knight BIBREF20 propose a multi-source model and BIBREF21 introduces a character-level encoder that is shared across several source languages. In our setup, we will follow the main idea proposed by Johnson et al. BIBREF22 . The authors of that paper suggest a simple addition by means of a language flag on the source language side (see Figure 2 ) to indicate the target language that needs to be produced by the decoder. This flag will be mapped on a dense vector representation and can be used to trigger the generation of the selected language. The authors of the paper argue that the model enables transfer learning and supports the translation between languages that are not explicitly available in training.
Question: What neural machine translation models can learn in terms of transfer learning?

Output: Multilingual Neural Machine Translation Models

Input: Consider Input: It is interesting to observe that the baseline model amplifies the bias in the training data set as measured by INLINEFORM0 and INLINEFORM1 . From measurements using the described bias metrics, our method effectively mitigates bias in language modelling without a significant increase in perplexity.
Question: how is mitigation of gender bias evaluated?

Output: Using INLINEFORM0 and INLINEFORM1

Input: Consider Input: Evaluation is done using the development set, consisting of 22 documents and 1112 PIE candidates, and the test set, which consists of 23 documents and 1127 PIE candidates. For each method the best set of parameters and/or options is determined using the development set, after which the best variant by F1-score of each method is evaluated on the test set.

Since these documents in the corpus are exhaustively annotated for PIEs (see Section SECREF40), we can calculate true and false positives, and false negatives, and thus precision, recall and F1-score.
Question: Are PIEs extracted automatically subjected to human evaluation?
Output: Yes