Question: What is the attention module pretrained on?
Answer: the model is pre-trained on CTC-based ASR task and MT task in the pre-training stage.