Before revision:  It requires no supervision or  parallel corpora, only two pretrained language models : forward and backward.
After revision:  It requires no supervision or parallel corpora, only two off-the-shelf pretrained LMs in opposite directions : forward and backward.