Parallel Data Augmentation for Formality Style Transfer (2020)

Abstract: The main barrier to progress in the task of Formality Style Transfer is the inadequacy of training data. In this paper, we study how to augment parallel data and propose novel and simple data augmentation methods for this task to obtain useful sentence pairs with easily accessible models and systems. Experiments demonstrate that our augmented parallel data largely helps improve formality style transfer when it is used to pre-train the model, leading to the state-of-the-art results in the GYAFC benchmark dataset.

## introduction formality style transfer (fst) is defined as the task of automatically transforming a piece of text in one particular formality style into another @xcite . for example, given an informal sentence, fst aims to preserve the styleindependent content and output a formal sentence. previous work tends to leverage neural networks @xcite @xcite such as seq2seq models to address this challenge due to their powerful capability and large improvement over the traditional rule-based approaches @xcite . however, the performance of the neural network approaches is still limited by the inadequacy of training data: the public parallel corpus for fst training -gyafc @xcite ) -contains only approximately 100k sentence pairs, which can hardly satiate the neural models with millions of parameters. to tackle the data sparsity problem for fst, we propose to augment parallel data with three specific data augmentation methods to help improve the model's generalization ability and reduce the overfitting risk. besides applying the widely used back fst (test instance) input (informal) i dunno, even if she like you, and then she 'll prob. reference (formal) i don't know. she probably will if she likes you. f-dis source i dunno... good luck. french je ne sais pas... bonne chance. target i don't know ... good luck. m-task source i think she like cat too. target i think she likes cat too. mt mt figure 1 : an example that formality style transfer (fst) benefits from data augmented via formality discrimination (f-dis) and multi-task transfer (m-task). the mapping knowledge indicated by the color (@xmath0. f-dis identifies useful sentence pairs from paraphrased sentence pairs generated by cross-lingual mt, while m-task utilizes training data from gec to help formality improvement. ## experiments in this section, we present the experimental settings and related experimental results. we focus on @xmath1. ## conclusion in this paper, we propose novel data augmentation methods for formality style transfer. our proposed data augmentation methods can effectively generate diverse augmented data with various formality style transfer knowledge. the augmented data can significantly help improve the performance when it is used for pre-training the model and leads to the state-of-the-art results in the formality style transfer benchmark dataset. fluency given the outputs, the annotators are asked to evaluate the fluency of each sentence in isolation. a sentence is considered to be fluent if it makes sense and is grammatically correct. the sentences satisfying the requirements will be rated 2 points. the sentences with minor errors will be rated 1 point. if the errors lead to confusing meaning, we give it 0 point.