Task: Question Generation
Input (English): In our final system, after pre-training the forward and backward LMs separately, we remove the top layer softmax and concatenate the forward and backward LM embeddings to form bidirectional LM embeddings, i.e., INLINEFORM0 . 
Output (English): how are the bidirectional lms obtained?