Semantic Re-tuning with Contrastive Tension

1 INTRODUCTION . Representation learning concerns the pursuit of automatically learning representations of data that are useful for future extraction of information ( Bengio et al. , 2013 ) . Recent work has predominantly been focused on training and extracting such representations from various deep neural architectures . However , as these deep models are mostly trained via error minimization of an objective function applied to the final layers ( Rumelhart et al. , 1988 ) , features residing in layers close to the objective function will be task-specific Yosinski et al . ( 2014 ) . Therefore , to reduce the representation ’ s bias towards the objective function it is common to discard one or several of the final layers , or alternatively consider features of other intermediate layers , as with AutoEncoders ( Rumelhart et al. , 1986 ) . One domain where this issue is particularly striking is learning semantic sentence embeddings with deep Transformer networks ( Vaswani et al. , 2017 ) pre-trained towards some language modeling task . Although utilizing pre-trained Transformer models such as BERT , XLnet , ELECTRA and GPT-2 ( Devlin et al. , 2019 ; Yang et al. , 2019 ; Clark et al. , 2020 ; Brown et al. , 2020 ) has become the dominant approach within the field of Natural Language Processing ( NLP ) , with current State Of The Art ( SOTA ) results in basically all NLP tasks belonging to fine-tuned versions of such models , it has been shown that simply extracting features from the layers of such models does not produce competitive sentence embeddings ( Reimers & Gurevych , 2019 ; Liu et al. , 2019a ) . Our interpretation of this phenomenon , which we will demonstrate in this paper , is that the currently used language modeling objectives enforce a task-bias at the final layers of the Transformer , and that this bias is not beneficial for the learning of semantic sentence representations . Reimers & Gurevych ( 2019 ) propose to solve this by pooling a fixed size sentence embedding from the final Transformer layer and fine-tune towards a Natural Language Inference ( NLI ) task , an approach that when applied to Transformers is known as Sentence-BERT ( or S-BERT in short ) . While Hill et al . ( 2016a ) empirically show that fine-tuning language models towards NLI data yields good results on Semantic Textual Similarity ( STS ) , there exists no convincing argument for why NLI is preferred over other tasks . Hence , it is unclear whether the impressive improvements of S-BERT are to be mainly attributed to the NLI task itself , or if this merely trains the model to output sentence embeddings , in turn exposing the semantics learned during pre-training . Since NLI requires labeled data , it would be highly valuable if an alternative method that requires no such labels was possible . ∗Main contribution . We therefore propose a fully self-supervised training objective that aims to remove the bias posed by the pre-training objective and to encourage the model to output semantically useful sentence representations . Our method trains two separate language models on the task of maximizing the dot product between the two models ’ representations for identical sentences , and minimizing the dot product between the models ’ representations for different sentences . When applied to pre-trained BERT models , our method achieves SOTA results for multiple unsupervised STS tasks , and when applied to the S-BERT model it outperforms previous SOTA by a clear margin . To further bolster the robustness of our method , we demonstrate that CT drastically improves STS scores for various models , across multiple languages . Additionally , we contribute with a layer-wise STS survey for the most common Transformer-based language models , in which we find great variability in performance between different architectures and pre-training objectives . Finally , by introducing an alteration to the supervised regression task of S-BERT , we are able to improve upon the supervised STS embedding results for all tested models . In summary , the main contributions of our paper are as follows : 1 . A novel self-supervised approach for learning sentence embeddings from pre-trained language models . 2 . Analytical results of the layer-wise STS performance for commonly used language models . 3 . An improvement to the supervised regression task of S-BERT that yields a higher performance for all tested models . Code and models is available at Github.com/FreddeFrallan/Contrastive-Tension 2 RELATED WORK . Where earlier work for learning sentence embeddings focused on the composition of pre-trained word embeddings ( Le & Mikolov ( 2014 ) ; Wieting et al . ( 2015 ) ; Arora et al . ( 2016 ) ) , recent work has instead favored extracting features from deep neural networks . The training methods of such networks can be divided into supervised and self-supervised . A systematic comparison of preTransformer sentence embedding methods is available in the works of Hill et al . ( 2016b ) . Self-supervised methods typically rely on the assumption that sentences sharing similar adjacent sentences , have similar meaning . Utilizing this assumption , Kiros et al . ( 2015 ) introduced SkipThoughts that trains an encoder-decoder to reconstruct surrounding sentences from an encoded passage . Logeswaran & Lee ( 2018 ) proposed QuickThoughts that instead frames the training objective as a sentence context classification task . Recently , and still under peer-review , Giorgi et al . ( 2020 ) proposed DeCLUTR that uses a setup similar to QuickThoughts , but allow positive sentences to be overlapping or subsuming ( one being a subsequence of the other ) , which further improves results . Supervised methods utilize labeled datasets to introduce a semantic learning signal . As the amount of explicitly labeled STS data is very limited , supervised methods often rely on various proxy tasks where more labeled data is available . Conneau et al . ( 2017 ) introduced InferSent that learns sentence embeddings via a siamese BiLSTM trained on NLI data . The Universal Sentence Encoder ( USE ) of Cer et al . ( 2018 ) is a Transformer encoder trained with both unlabeled data and labeled NLI data . S-BERT by Reimers & Gurevych ( 2019 ) adopts the training objective of InferSent but instead applies pre-trained BERT models . Finally , Wang & Kuo ( 2020 ) recently proposed S-BERT-WK , an extension to S-BERT that further increases the performance by subspace analysis of the model ’ s layer-wise word features . Recently , Grill et al . ( 2020 ) introduced the self-supervised BYOL framework that attain useful image representations , comparable with previous supervised methods . Although their method also utilizes two untied dual networks , the main training objective and the underlying motivation for this differ greatly . Where BYOL train using solely positive samples generated via data augmentation , our method mainly aims to dissipate negative examples and relies on two networks in order to stabilize the training process . To the best of our knowledge , our work is the first that suggests learning sentence representations by removing the bias imposed from the pre-training objective . 3 LAYER-WISE STUDY OF TRANSFORMER MODELS . Previous work analyzing the downstream applicability of layer-wise features in Transformer model reports similar trends of performance increasing until the middle layers before decreasing towards the final layers . Merchant et al . ( 2020 ) found the best suited features for linguistic tasks such as entity typing and relation classification reside in the intermediate layers of BERT , and Chen et al . ( 2020 ) found the most useful representations for image classification in the intermediate layers of Image-GPT . We contribute with a layer-wise study of the semantic quality of the sentence representations found in a selected number of common Transformer architectures . Following the approach of S-BERT , we generate sentence embeddings by mean pooling over the word-piece features of a given layer . These sentence embeddings are directly evaluated towards the STS-b test ( Cer et al. , 2017 ) , without any additional training , from which we report the Spearman correlation between the cosine similarity of the embeddings and the manually collected similarity scores . The test partition of the dataset contains 1,379 sentence pairs , with decimal human similarity scores ranging from 0.0 ( two sentences having completely different meanings ) to 5.0 ( two sentences have identical meaning ) . Figure 1 shows the results for BERT , Electra , XLNet and GPT-2 , with results for additional models in appendix B.4 . Although the different models display different layer-wise patterns , a common theme is that it is not obvious where to extract features for semantic sentence embeddings ; the worst-performing representations are often found in the layers close to the objective function , with the exception of RoBerta base ( Liu et al. , 2019b ) . Considering the discrepancy between BERT and Electra which share an almost identical architecture but differ drastically in their pre-training objectives , it is clear that the semantic quality of a model ’ s sentence representations is heavily impacted by the choice of pre-training objective . 4 METHOD . To counter the negative trend found in Section 3 , where the lacking STS performance of the sentence representations in the final layers became apparent , we define a training objective meant to encourage the model to retain a semantically distinguishable sentence representation until the final layer . We name this method Contrastive Tension ( CT ) , where two independent models , with identically initialized weights , are set to maximise the dot product between their sentence representations for identical sentences , and minimize the dot product for their sentence representations of differing sentences . Hence , the CT objective is defined as : z = f1 ( s1 ) T · f2 ( s2 ) L ( z , s1 , s2 ) = { −log σ ( z ) if s1 = s2 −log σ ( 1− z ) if s1 6= s2 ( 1 ) Where f1 and f2 are two independently parameterized models that given a sentence s produces a fixed size vector representation and where σ refers to the Logistic function . Following the works of Reimers & Gurevych ( 2019 ) , we generate fixed size sentence representations by mean pooling over the features in the final layer of pre-trained transformer models . Training data is randomly generated from a given corpus , where for each randomly selected sentence s , K negative sentences are sampled to generate K + 1 training samples by pairing s with the negative sentences and copying s into an identical sentence pair . This yields one positive training sample and K negative training samples . We include the K + 1 training samples in the same batch and always use f2 to embed the K negative sentences ( See Appendix A.1 for a visual example ) . Our approach for generating negative samples is based on the assumption that two randomly selected sentences are very likely to be semantically dissimilar . As the models are initialized with identical weights , the CT objective creates a tension between having the two models retain similar representations for identical sentences , at the same time as the two models are encouraged to distinguish their representations for differing sentences . Our intuition is that this creates a training dynamic where the two models acts as smooth anchors to each other , where the tension to remain synchronized mitigates the downsides of simply distancing the embeddings of differing sentences . This makes CT a nondestructive method for distinguishing the sentence embeddings of non semantically similar sentences . 5 EXPERIMENTS . Unless stated otherwise , the following set of hyperparameters is applied when using CT throughout all experiments : Training data is randomly sampled from English Wikipedia ( See Appendix C.2 ) , where we collectK = 7 negative sentence pairs for each positive sentence pair . The batch size is set to 16 , which results in every batch having 2 positive sentence pairs and 14 negative sentence pairs . We apply an RMSProp optimizer ( Hinton , 2012 ) with a fixed learning rate schedule that decreases from 1e−5 to 2e−6 ( Appendix A.3 ) . To showcase the robustness and unsupervised applicability of CT , we strictly perform 50,000 update steps before evaluating , and for all unsupervised tasks we report results for the worst-performing of the two models used in the CT setup . The experiment section follows the model naming convention elaborated upon in A.2 , which describes the order and what training objectives that has been applied to a model . There exists a clear discrepancy between previously reported STS scores for various methods and models . To improve upon this state of confusion we perform all evaluation with the SentEval package ( Conneau & Kiela , 2018 ) , to which we provide code and models for full reproducability of all tested methods . A Discussion regarding our experience with trying to reproduce previous work is available in Appendix A.4 . A comprehensive list of all used model checkpoints is available in Appendix C.1

Summary: The paper studies the problem of finding effective representation for semantic text similarity (STS). The paper first investigates the effectiveness of pre-trained masked language models (e.g. BERT) in the STS task. They found different layers of BERT have different performance when employed in STS task -- in particular, the popular method of using the last layer does not usually lead to  good performance. In fact, the last layers are worse than those preceding layers in STS. This is universal in several models including BERT, Electra, XLNet, and GPT-2, but not RoBerta. The paper then propose to use dual contrastive training method (basically use two BERT branches) to further fine-tune BERT, where the additional objective is defined by bringing two models' output closer for the same input sentence and further for randomly sampled different sentences.