In this task, you will be presented with a context from an academic paper and a question separated with a
. You have to answer the question based on the context.

We took inspiration from these works to design our experiments to solve the CSKS task.
Question: What problem do they apply transfer learning to?
CSKS task

Various multilingual extensions of NMT have already been proposed in the literature. The authors of BIBREF18 , BIBREF19 apply multitask learning to train models for multiple languages. Zoph and Knight BIBREF20 propose a multi-source model and BIBREF21 introduces a character-level encoder that is shared across several source languages. In our setup, we will follow the main idea proposed by Johnson et al. BIBREF22 . The authors of that paper suggest a simple addition by means of a language flag on the source language side (see Figure 2 ) to indicate the target language that needs to be produced by the decoder. This flag will be mapped on a dense vector representation and can be used to trigger the generation of the selected language. The authors of the paper argue that the model enables transfer learning and supports the translation between languages that are not explicitly available in training.
Question: What neural machine translation models can learn in terms of transfer learning?
Multilingual Neural Machine Translation Models

To deal with negative values, we propose clipped $\mathit {PMI}$,

which is equivalent to $\mathit {PPMI}$ when $z = 0$. In practice we find this works poorly if done symmetrically, so we introduce a variant called $\mathit {NNEGPMI}$ which only normalizes $\mathit { exttt {-}PMI}$:
Question: What novel PMI variants are introduced?
clipped PMI; NNEGPMI