ConvFiT: C onversational Fine-Tuning of Pretrained Language Models (2021)

Abstract: Transformer-based language models (LMs) pretrained on large text collections are proven to store a wealth of semantic knowledge. However, 1) they are not effective as sentence encoders when used off-the-shelf, and 2) thus typically lag behind conversationally pretrained (e.g., via response selection) encoders on conversational tasks such as intent detection (ID). In this work, we propose ConvFiT, a simple and efficient two-stage procedure which turns any pretrained LM into a universal conversational encoder (after Stage 1 ConvFiT-ing) and task-specialised sentence encoder (after Stage 2). We demonstrate that 1) full-blown conversational pretraining is not required, and that LMs can be quickly transformed into effective conversational encoders with much smaller amounts of unannotated data; 2) pretrained LMs can be fine-tuned into task-specialised sentence encoders, optimised for the fine-grained semantics of a particular task. Consequently, such specialised sentence encoders allow for treating ID as a simple semantic similarity task based on interpretable nearest neighbours retrieval. We validate the robustness and versatility of the ConvFiT framework with such similarity-based inference on the standard ID evaluation sets: ConvFiT-ed LMs achieve state-of-the-art ID performance across the board, with particular gains in the most challenging, few-shot setups.

## introduction and motivation pretrained transformer-based (masked) language models (lms) such as bert @xcite or roberta @xcite , coupled with task-specific fine-tuning, offer unmatched state-ofthe-art performance in a wide array of standard language understanding and conversational tasks @xcite . however, pretrained lms do not produce coherent and effective sentence encodings off-the-shelf; their further adaptation is required, akin to standard task fine-tuning. for instance, @xcite transform monolingual english bert with supervised natural language inference and paraphrasing data @xcite into a sentence encoder which excels at sentence similarity and retrieval tasks @xcite . this transformation process supports the creation of other similar universal sentence encoders in monolingual and multilingual settings @xcite @xcite , and is typically based on dual-encoder architectures. another parallel research thread aims at learning conversational encoders: it validates the benefits of masked language modeling (mlm) pretraining on naturally conversational data @xcite , as well as the benefits of transfer learning for conversational tasks which goes beyond mlm as the pretraining objective @xcite @xcite . in particular, response selection as a suitable pretraining task @xcite @xcite learns representations that organically capture conversational cues from conversational text data such as reddit @xcite , again via dual-encoder architectures. inspired by these two research threads, we pose the following two crucial questions: in order to address these two questions, we propose convfit, a two-stage conversational fine-tuning procedure that turns general-purpose mlm-pretrained encoders into sentence encoders specialised for a particular conversational domain and task. casting the end-task (e.g., intent detection) as a pure sentence similarity problem then allows us to recast task-tailored fine-tuning of a pretrained lm as gradual sentence encoder specialisation, as illustrated in figures 1 and 2 . our hypothesis is that the pretrained lms, which already store a wealth of semantic knowledge, can be gradually turned into conversational taskadapted sentence encoders without expensive full pretraining. (s1) stage 1 transforms pretrained lms into universal conversational encoders via adaptive fine-tuning @xcite on (a fraction of) reddit data (see figure 2b ), relying on a standard dual-encoder architecture with a conversational response ranking loss @xcite ; cf. q1. (s2) stage 2 further specializes the sentence encoder via contrastive learning with intask data, that is, it learns meaningful task-related semantic clusters/subspaces. we then show that the s2 task-tailored specialisation effectively enables a simple and interpretable similarity-based classification based on nearest neighbours (nns) in the specialised encoder space (see q2 and figure 2c ). the two-stage convfit transformation offers new insights and contributions to representation learning for conversational tasks. unlike prior work which conducted large-scale conversational pretraining from scratch using large datasets, we demonstrate that full pretraining is not needed to obtain universal conversational encoders. by leveraging the general semantic knowledge already stored in pretrained lms, we can expose (i.e., 'rewire') that knowledge @xcite @xcite via much cheaper and quicker adaptive fine-tuning on a tiny fraction of the full reddit data (e.g., even @xmath0.01% of the reddit corpus). further, the task-oriented s2 convfit-ing transforms pretrained lms into task-specialised sentence encoders. our results with similarity-based classification, targeting the crucial conversational nlu task of intent detection (id), reach state-of-the-art (sota) across all standard id datasets, with particular gains in the most challenging, few-shot setups. importantly, we show that the gradual application of s1 and then s2 yields a synergistic effect, that is, it attains the highest id results across the board. finally, convfit is highly versatile: it can be used with a range of pretrained lms and on a spectrum of text classification problems; it also allows for the simple usage of diverse fine-tuning objectives in both stage 1 and stage 2, beyond the ones proposed and evaluated in this work. ## methodology preliminaries. for any input text t, we obtain its encoding @xmath1, where enc is a sentence encoder at any convfit stage (i.e., before any finetuning, after s1, or after s2), or any other sentence encoder. the text t is tokenized into subwords @xcite relying on each encoder's dedicated tokeniser. the final encoding t is created via a pooling operation such as (a) using the [cls] token, (b) or mean-pooling the output subword vectors. following prior work @xcite , we always use mean-pooling. ## experimental setup input lms. we experiment with several popular transformer-based @xcite lms as input (see figure 1 ), aiming to validate the robustness of convfit, as well as to analyse the impact of lm pretraining on the final task performance: (i) bert @xcite (labeled bert henceforth); (ii) roberta (rob), as an improved variant of bert, lm-pretrained with more data @xcite ; (iii) distilroberta (drob), a distilled more compact version of roberta, lm-pretrained with around 4 times fewer data than the teacher roberta model @xcite . the cased base variants are used for all input lms: 768dimensional transformer layers with 12 (bert, rob) or 6 (drob) attention layers. in addition, to isolate the effects of lm-pretraining and con-vfit-ing from the mere "parameter capacity", we also experiment with a bert/rob architecture with randomly initialised parameters using the xavier initialisation @xcite . unless noted otherwise, convfit stage 1 always proceeds with a sample comprising 2% of the full reddit corpus from henderson et al. (2019a). 7 intent detection datasets. as discussed in §2, the main evaluation task is intent detection (id), with a particular focus on low-data (i.e., few-shot) scenarios. our stage 2 fine-tuning and the final task evaluation are based on three standard id datasets in english, also available as part of the recently published dialoglue benchmark @xcite ): banking77 @xcite , hwu64 @xcite , and clinc150 @xcite . 8 the key statistics of all three datasets are provided in few-shot and full data setups. prior work has recognised the importance of building intent detec-tors in low-data regimes @xcite . therefore, following this initiative, we evaluate the models in two n-shot scenarios, where we assume that only @xmath2; figure 1 . 9 the models are also evaluated in the full setup, where all annotated training examples per intent are used. note that we always report the scores on the same test set for each setup. for the few-shot scenarios, we report the scores as averages over 3 independent experimental runs. hyperparameters and optimisation. convfit is implemented via the sentence-transformers (sbert) repository @xcite , which is in turn built on top of the huggingface repository @xcite . similar to casanueva et al. (2020), we do not rely on any development data, and follow the general suggestions from prior work @xcite for the hyperparameter setup, which is adopted across all intent id datasets. 10 for s1 with mneg, we always train for 2 epochs in batches of 256 with default hparams from sbert. 11 in stage 2, with all three evaluated objective functions the batch size is 32, the maximum sequence length is 48, the output layer's dimensionality is set to d @xmath3. unless stated otherwise, we always fine-tune for 10, 5, and 2 epochs for the 10-shot, 30-shot, and full setups, respectively. for the cos and ocl variants, unless noted otherwise, we report the results with @xmath4, and with @xmath5. an analysis of the impact of n on the final id performance is presented later in §4. following the suggested settings of @xcite ; @xcite , in both convfit stages we use the adamw optimiser @xcite ; the learning rate is 2e -5 with the warmup rate of 0.1 and linear decay afterwards, and the weight decay rate is set to 0.01. similarity-based classification. the intent class is chosen according to the @xmath6, based on the cosine distance in the fine-tuned space. 12 importantly, in few-shot setups we use only the few-shot data as the nn pool for classification. ## results and discussion the main results are summarised in mlp versus similarity-based id. first, we note that convfit-ed lms achieve peak id scores across all three id datasets, and in all data setups, with rob+s1+s2-ocl being the highestperforming model variant overall. running stage 1 does transform input lms into effective (universal) conversational encoders already: for mlpbased id, we observe competitive or even improved performance (cf., the results on banking77 and hwu64 as two more challenging evaluation sets) with the rob+s1 and bert+s1 variants against current state-of-the-art (conversational) sentence encoders such as convert, use, and labse. importantly, the results after stage 2 'unanimously' suggest the effectiveness of treating id as a semantic similarity task, and additional taskspecific specialisation of the sentence encoders with in-task data. put simply, it seems more effective to use the in-task training data to 'taskspecialise' the sentence encoder space than to learn a standard (mlp) classifier, which directly maps from the feature space to intent labels @xcite . the gains are especially pronounced in fewshot setups (e.g., see 10-shot banking77). we speculate that dual-encoder contrastive learning surpasses mlp-based approaches especially in few-data scenarios because it learns from finergrained and more abundant information in such low-data scenarios: i.e., we learn to contrast between pairs of instances rather than simply learning an mlp-based mapping from an instance to its underlying class intent/class. this formulation can also capture some subtle cross-instance (dis)similarities which cannot be captured by mlp. extending beyond pure absolute performance, decisions based on k-nn similarity-based id in the specialised space are also easy to interpret @xcite . stage 1 + stage 2? the scores in in sum, this finding is line with prior work in other domains and nlp tasks @xcite @xcite : both domainadaptive (our s1) and task-adaptive additional tuning (our s2) of general-purpose lms have a synergistic positive impact on the final task performance. the impact of the gradual two-stage sentence encoder transformation is also clearly visible from the t-sne visualisation in figure 2 . besides this, a standard quantitative measure of cluster coherence, the silhouette coefficient σ (rousseeuw, 1987) also points in the same direction: σ = 0.067 for the test examples and model variant from figure 2a , σ = 10 30 full training set size (# of examples per intent) 60 65 70 75 80 85 90 95 accuracy rand+s2 rand+s1+s2 rob+s1+s2 0.188 (figure 2b ), and σ = 0.698 (figure 2c ). 16 17 impact of input lms. while the results suggest that the convfit framework is applicable and effective with any pretrained lm, the choice of the input lm naturally impacts the absolute id performance. as expected, the convfit variants with roberta achieve the highest scores across the board. a comparison between drob and bert reveals that the pretraining data size and regime seem to play a more critical role than the parameter capacity: the more compact drob lm is competitive with or even outscores bert-based variants. 18 16 higher σ scores are desirable as they imply more coherent and compact clusters, and a stronger inter-cluster separation. importance of lm pretraining is illustrated by figure 3 . the trend is quite straightforward: semantic knowledge acquired by lm-pretraining is particularly important in the fewest-shot (i.e., 10shot) setups, and the gap gets reduced with more in-task data available for s2 tuning. however, the gap remains substantial even in the full setups. figure 3 also reveals that the strength of con-vfit stage 1 is in adapting the knowledge acquired at lm pretraining: s1 fine-tuning of rand with smaller amounts of reddit data cannot match rob as the input lm, although the gap does become smaller with more in-task data for s2. stage 2: fine-tuning losses. ## conclusion and future work we proposed convfit, a two-stage conversational fine-tuning procedure that transforms pretrained lms (e.g., bert, roberta) into universal (after stage 1) and task-specialised conversational sentence encoders (after stage 2) through dualencoder architectures. the semantic knowledge already stored in the pretrained lms gets 'rewired' for a particular domain and task. we demonstrated that such task-specialised sentence encoders enable casting intent detection (id) as simple sentence similarity; convfit-ed encoders yield strong id results across diverse id datasets and setups. the convfit framework is very versatile and opens up many future research paths and further extensions and experimentation beyond the scope of this paper. for instance, it is possible to replace the current contrastive loss functions with other recent effective contrastive losses @xcite , or mine hard (instead of using random) negative examples @xcite @xcite . we will also extend con-vfit to other pretrained models, experiment with automatic paraphrasers for data augmentation, and port the framework to other conversational tasks (e.g., slot labelling for dialogue), as well as to other, non-dialogue text classification tasks.