Before revision: To obtain high-quality sentence embeddings from pretrained language models  , they must either be augmented with additional pretraining objectives or finetuned on large amounts of labeled text pairs.
After revision: To obtain high-quality sentence embeddings from pretrained language models , they must either be augmented with additional pretraining objectives or finetuned on a large set of labeled text pairs.