Before revision:  It requires no supervision or  parallel corpora, only two pretrained language models : forward and backward.
After revision:  It requires no supervision or even parallel corpora, only two pretrained language models : forward and backward.