Question: What language model architectures are used?
Answer: uni-directional model to augment the decoder
Alternative answer: bi-directional language model to augment the sequence to sequence encoder ,  uni-directional model to augment the decoder