Self-Guided Contrastive Learning for BERT Sentence Representations (2021)

Abstract: Although BERT and its variants have reshaped the NLP landscape, it still remains unclear how best to derive sentence embeddings from such pre-trained Transformers. In this work, we propose a contrastive learning method that utilizes self-guidance for improving the quality of BERT sentence representations. Our method fine-tunes BERT in a self-supervised fashion, does not rely on data augmentation, and enables the usual [CLS] token embeddings to function as sentence vectors. Moreover, we redesign the contrastive learning objective (NT-Xent) and apply it to sentence representation learning. We demonstrate with extensive experiments that our approach is more effective than competitive baselines on diverse sentence-related tasks. We also show it is efficient at inference and robust to domain shifts.

## introduction pre-trained transformer @xcite language models such as bert @xcite and roberta @xcite have been integral to achieving recent improvements in natural language understanding. however, it is not straightforward to directly utilize these models for sentencelevel tasks, as they are basically pre-trained to focus on predicting (sub)word tokens given context. the most typical way of converting the models into sentence encoders is to fine-tune them with supervision from a downstream task. in the process, as initially proposed by @xcite , a pre-defined token's (a.k.a. @xcite ) embedding from the last layer of the encoder is deemed as the representation of an input sequence. this simple but effective method is possible because, during supervised fine-tuning, the [cls] embedding functions as the only communication gate between the pre-trained encoder we observe that the performance can be dramatically varied according to the selected layer and pooling strategy. our self-guided training (sg / sg-opt) assures much improved results compared to those of the baselines. and a task-specific layer, encouraging the @xcite vector to capture the holistic information. on the other hand, in cases where labeled datasets are unavailable, it is unclear what the best strategy is for deriving sentence embeddings from bert. 1 in practice, previous studies @xcite @xcite reported that naïvely (i.e., without any processing) leveraging the [cls] embedding as a sentence representation, as is the case of supervised finetuning, results in disappointing outcomes. currently, the most common rule of thumb for building bert sentence embeddings without supervision is to apply mean pooling on the last layer(s) of bert. yet, this approach can be still sub-optimal. in a preliminary experiment, we constructed sentence embeddings by employing various combinations of different bert layers and pooling methods, and tested them on the semantic textual similarity (sts) benchmark dataset @xcite . 2 we discovered that bert(-base)'s performance, measured in spearman correlation (× 100), can range from as low as 16.71 @xcite , the 10 th layer) to 63.19 (max pooling, the 2 nd layer) depending on the selected layer and pooling method (see figure 1 ). this result suggests that the current practice of building bert sentence vectors is not solid enough, and that there is room to bring out more of bert's expressiveness. in this work, we propose a contrastive learning method that makes use of a newly proposed selfguidance mechanism to tackle the aforementioned problem. the core idea is to recycle intermediate bert hidden representations as positive samples to which the final sentence embedding should be close. as our method does not require data augmentation, which is essential in most recent contrastive learning frameworks, it is much simpler and easier to use than existing methods @xcite . moreover, we customize the nt-xent loss @xcite , a contrastive learning objective widely used in computer vision, for better sentence representation learning with bert. we demonstrate that our approach outperforms competitive baselines designed for building bert sentence vectors @xcite in various environments. with comprehensive analyses, we also show that our method is more computationally efficient than the baselines at inference in addition to being more robust to domain shifts. ## related work contrastive representation learning. contrastive learning has been long considered as effective in constructing meaningful representations. for instance, @xcite propose to learn word embeddings by framing words nearby a target word as positive samples while others as negative. @xcite generalize the approach of @xcite for sentence representation learning. more recently, several studies @xcite @xcite suggest to utilize contrastive learning for training transformer models, similar to our approach. however, they generally require data augmentation techniques, e.g., back-translation @xcite , or prior knowledge on training data such as order information, while our method does not. furthermore, we focus on revising bert for computing better sentence embeddings rather than training a language model from scratch. on the other hand, contrastive learning has been also receiving much attention from the computer vision community @xcite al. (2020); chen and he (2020); he et al. (2020) , inter alia). we improve the framework of @xcite by optimizing its learning objective for pre-trained transformerbased sentence representation learning. for extensive surveys on contrastive learning, refer to @xcite and @xcite . fine-tuning bert with supervision. it is not always trivial to fine-tune pre-trained transformer models of gigantic size with success, especially when the number of target domain data is limited @xcite . to mitigate this training instability problem, several approaches @xcite @xcite have been recently proposed. in particular, @xcite propose to exploit contrastive learning as an auxiliary training objective during fine-tuning bert with supervision from target tasks. in contrast, we deal with the problem of adjusting bert when such supervision is not available. sentence embeddings from bert. since bert and its variants are originally designed to be fine-tuned on each downstream task to attain their optimal performance, it remains ambiguous how best to extract general sentence representations from them, which are broadly applicable across diverse sentence-related tasks. following conneau et al. (2017), reimers and gurevych (2019) (sbert) propose to compute sentence embeddings by conducting mean pooling on the last layer of bert and then fine-tuning the pooled vectors on the natural language inference (nli) datasets @xcite . meanwhile, some other studies concentrate on more effectively leveraging the knowledge embedded in bert to construct sentence embeddings without supervision. specifically, @xcite propose a pooling method based on linear algebraic algorithms to draw sentence vectors from bert's intermediate layers. @xcite suggest to learn a mapping from the average of the embeddings obtained from the last two layers of bert to a spherical gaussian distribution using a flow model, and to leverage the redistributed embeddings in place of the original bert representations. we follow the setting of @xcite in that we only utilize plain text during training, however, unlike all the others that rely on a certain pooling method even after training, we directly refine bert so that the typical [cls] vector can function as a sentence embedding. note also that there exists concurrent work @xcite @xcite whose motivation is analogous to ours, attempting to improve bert sentence embeddings in an unsupervised fashion. ## method as bert mostly requires some type of adaptation to be properly applied to a task of interest, it might not be desirable to derive sentence embeddings directly from bert without fine-tuning. while @xcite attempt to alleviate this problem with typical supervised fine-tuning, we restrict ourselves to revising bert in an unsupervised manner, meaning that our method only demands a bunch of raw sentences for training. among possible unsupervised learning strategies, we concentrate on contrastive learning which can inherently motivate bert to be aware of similarities between different sentence embeddings. considering that sentence vectors are widely used in computing the similarity of two sentences, the inductive bias introduced by contrastive learning can be helpful for bert to work well on such tasks. the problem is that sentence-level contrastive learning usually requires data augmentation @xcite or prior knowledge on training data, e.g., order information @xcite , to make plausible positive/negative samples. we attempt to circumvent these constraints by utilizing the hidden representations of bert, which are readily accessible, as samples in the embedding space. ## analysis we here further investigate the working mechanism of our method with supplementary experiments. all the experiments conducted in this section follow the configurations stipulated in section 4.1 and 4.2. models sts tasks (avg.) bert-base + sg-opt (l opt3 ) 74.62 + l opt2 73.14 (-1.48) + l opt1 72.61 (-2.01) + sg (l base ) 72.17 (-2.45) bert-base + sg-opt (τ = 0.01, λ = 0.1) 74.62 + τ = 0.1 70.39 (-4.23) + τ = 0.001 74.16 (-0.46) + λ = 0.0 73.76 (-0.86) + λ = 1.0 73.18 (-1.44) -projection head (f ) 72.78 (-1.84) ## discussion in this section, we discuss a few weaknesses of our method in its current form and look into some possible avenues for future work. first, while defining the proposed method in section 3, we have made decisions on some parts without much consideration about their optimality, prioritizing simplicity instead. for instance, although we proposed utilizing all the intermediate layers of bert and max pooling in a normal set-ting (indeed, it worked pretty well for most cases), a specific subset of the layers or another pooling method might bring better performance in a particular environment, as we observed in section 4.4 that we could achieve higher numbers by employing mean pooling and excluding lower layers in the case of senteval (refer to appendix a.3 for details). therefore, in future work, it is encouraged to develop a systematic way of making more optimized design choices in specifying our method by considering the characteristics of target tasks. second, we expect that the effectiveness of contrastive learning in revising bert can be improved further by properly combining different techniques developed for it. as an initial attempt towards this direction, we conduct an extra experiment where we test the ensemble of back-translation and our self-guidance algorithm by inserting the original sentence into bert t and its back-translation into bert f when running our framework. in ## conclusion in this paper, we have proposed a contrastive learning method with self-guidance for improving bert sentence embeddings. through extensive experiments, we have demonstrated that our method can enjoy the benefit of contrastive learning without relying on external procedures such as data augmentation or back-translation, succeeding in generating higher-quality sentence representations compared to competitive baselines. furthermore, our method is efficient at inference because it does not require any post-processing once its training is completed, and is relatively robust to domain shifts.