Before revision:  In addition, we introduce leaky attention to alleviate the problem of unexpected high attention weights on special tokens  . Experiments on four language pairs show that our model significantly outperforms all existing unsupervised neural baselines and obtains new state-of-the-art results.
After revision:  In addition, we introduce leaky attention to alleviate the problem of unexpected high cross-attention weights on special tokens . Experiments on four language pairs show that our model significantly outperforms all existing unsupervised neural baselines and obtains new state-of-the-art results.