Before revision: To obtain high-quality sentence embeddings from pretrained language models  , they must either be augmented with additional pretraining objectives or finetuned on large amounts of labeled text pairs.
After revision: To obtain high-quality sentence embeddings from pretrained language models (PLMs) , they must either be augmented with additional pretraining objectives or finetuned on large amounts of labeled text pairs.