Before revision:  We propose that the lead bias can be leveraged in  a simple and effective way in our favor to pretrain abstractive news summarization models on large-scale unlabeled corpus : predicting the leading sentences using the rest of an article.
After revision:  We propose that the lead bias can be leveraged in a simple and effective way to pre-train abstractive news summarization models on large-scale unlabeled corpus : predicting the leading sentences using the rest of an article.