Contrastive Representation Learning for Exemplar-Guided Paraphrase Generation (2021)

Abstract: Exemplar-Guided Paraphrase Generation (EGPG) aims to generate a target sentence which conforms to the style of the given exemplar while encapsulating the content information of the source sentence. In this paper, we propose a new method with the goal of learning a better representation of the style and the content. This method is mainly motivated by the recent success of contrastive learning which has demonstrated its power in unsupervised feature extraction tasks. The idea is to design two contrastive losses with respect to the content and the style by considering two problem characteristics during training. One characteristic is that the target sentence shares the same content with the source sentence, and the second characteristic is that the target sentence shares the same style with the exemplar. These two contrastive losses are incorporated into the general encoder-decoder paradigm. Experiments on two datasets, namely QQP-Pos and ParaNMT, demonstrate the effectiveness of our proposed constrastive losses.

## introduction paraphrase generation @xcite , aiming to generate a sentence with the same semantic meaning of the source sentence, has achieved a great success in recent years. to obtain a paraphrase sentence with a particular style, exemplar-guided paraphrase generation (egpg) @xcite has attracted considerable attention. different from other controllable text generation tasks whose constraints are taken from a finite set, e.g., binary sentiment or political slant @xcite , multiple personas @xcite , over * the work described in this paper is substantially supported by a grant from the research grant council of the hong kong special administrative region, china (project code: 14200620). ## proposed model given a source sentence x i and an exemplar sentence z i , our goal is to generate a sentence y i that has the same style (syntax) with z i and retains the content (semantics) of x i . as shown in figure 1 , we design the encoders e s and e c for style and content respectively. the decoder d generates the output. our model is trained by optimizing three losses simultaneously: (1) generation loss; (2) content contrastive loss; (3) style contrastive loss. generation task for x i and z i , we firstly obtain their corresponding content features c x i and style features s z i : content contrastive learning (ccl) considering that x i and y i share the same content, their content features should be close with each other in the content feature space. contrastive learning which is designed to minimize the distance between positive pairs and maximize the distance between negative pairs can help model this relationship. formally, during training, given a batch {(x i , y i , z i )} n @xmath0, we firstly obtain the corresponding content features of x i and and c x i with the other remaining features in this batch form 2n -2 negative pairs. for c y i , the definition of positive/negative pairs is the same as c x i . then the contrastive loss is employed, giving style contrastive learning (scl) aims to help e s learn a better style representation by considering that z i and y i share the same style. similar to ccl, we firstly obtain the style features {(s z i , s y i )} n @xmath1 ## conclusion we introduce the content contrastive loss and the style contrastive loss into egpg to design a multilosses scheme without requiring additional labeled data. this scheme can obtain better results compared with the baseline and ablative models, which demonstrates the effectiveness of contrastive learning for learning better representations. moreover, the proposed framework is general and may benefit other similar nlp tasks.