Multi-Pair Text Style Transfer for Unbalanced Data via Task-Adaptive Meta-Learning (2021)

Abstract: Text-style transfer aims to convert text given in one domain into another by paraphrasing the sentence or substituting the keywords without altering the content. By necessity, state-of-the-art methods have evolved to accommodate nonparallel training data, as it is frequently the case there are multiple data sources of unequal size, with a mixture of labeled and unlabeled sentences. Moreover, the inherent style defined within each source might be distinct. A generic bidirectional (e.g., formal \Leftrightarrow informal) style transfer regardless of different groups may not generalize well to different applications. In this work, we developed a task adaptive meta-learning framework that can simultaneously perform a multi-pair text-style transfer using a single model. The proposed method can adaptively balance the difference of meta-knowledge across multiple tasks. Results show that our method leads to better quantitative performance as well as coherent style variations. Common challenges of unbalanced data and mismatched domains are handled well by this method.

## introduction text-style transfer is a fundamental challenge in natural language processing. applications include non-native speaker assistants, child education, personalization and generative design @xcite @xcite @xcite . figure 1 shows a prominent example on applying style transfer into a hypothetical online shopping platform, where the generated style variations can be used for personalized recommendations. however, compared with other domains, the lack of parallel corpus and quality training data is currently an obstacle for text-style transfer research. for example, assume one supports a multitenant service platform including tenant-specific text data, but there is no guarantee that each tenant will provide sufficient amount of data for model training. to build a multi-task language model that matches the text-style of each tenant is more practical and efficient than training individual models. this single-model approach might also have relatively favorable empirical performance. existing works on text style transfer have addressed different applications such as sentiment transfer @xcite , word decipherment @xcite , and author imitation @xcite . if parallel training data is available, a wide range of supervised techniques in machine translation (e.g., seq2seq models @xcite and transformers @xcite ) can also be applied to style transfer problems. for non-parallel data, @xcite proposed a probabilistic formulation that models non-parallel data from two domains as a partially observed parallel corpus, and learn the style transfer model in a completely unsupervised fashion. unsupervised machine translation method has also been adapted to this setting @xcite . in recent research focused on learning disentangled content and style representations using adversarial training @xcite @xcite , models are designed for non-parallel data while preserving content. @xcite argued that the adversarial models are not really doing disentanglement, and proposed a denoising auto-encoding approach instead. another way to approach this problem is through identifying and substituting style-related sub-sentences @xcite , where the unchanged part guarantees consistency over content. additionally, state-of-theart language models (bert @xcite , gpt-2 @xcite , ctrl @xcite , etc.) and text-to-text models @xcite achieve good performance generating text meta-learning @xcite is a method to enable generalization ability to a model over a distribution of tasks. we focus on optimizationbased meta-learning for our applications. maml @xcite learns a common initialization parameter for each task using a few gradient steps. this standard maml approach has been applied to text style transfer problems with low resources @xcite and achieved better performance in this situation. however, this method did not take into account the internal variations between tasks. a similar algorithm called reptile @xcite achieves better performance by maximizing the inner product between gradient of different mini-batches from the same task in its update. recent works @xcite ) improved a single meta-learner to task-adaptive meta-learning models, which in-cludes task-specific parameters to help generalize better between tasks. bayesian meta-learning is another active area of research @xcite proposed a probabilistic version of maml, where the variational inference framework utilizes a taskspecific gradient update. more recently, @xcite incorporated a bayesian framework into task-adaptive meta-learning. specifically, they introduce balancing variables for task and classspecific learning and leverage the uncertainties of these parameters derived from training data statistics. in this paper, we will adapt the bayesian task adaptive meta-learning (taml) for our application shown in figure 2 overview. ## balancing variations between tasks a common challenge in aforementioned real application is that data from multiple sources may suffer from different problems, such as insufficient training samples, unbalanced class labels, or domain mismatch. however, simply ignore these differences and concatenate all tenants' data for model training will not lead to ideal results. meta-learning is one of the most relevant approaches for generalized learning from few samples of different tasks. assume a task distribution p(τ ) that randomly generates task τ consisting of training set d τ = {x τ , x τ } and a test set an overview of our multi-pair style transfer method: assume learning from each tenant's data is a task, and the training data available for each task varies. the style transfer model can adaptively learn tasks using our method and the resulting model performs style transfer across multiple domains. if parallel training data is not available, then we only have d τ = x τ and d τ @xmath0. the maml algorithm initialize task-specific parameter θ τ using a few gradient steps on a small amount of data. in this case, the optimized parameters can generalize to new tasks. specifically, we have the loss minimization eq (1) is effective in numerous applications, yet insufficient in addressing our data problems, as it treats the initialization and learning parameters with equal importance for each task. inspired by @xcite , we now introduce three balancing variables: z τ , γ τ , ω τ for every task τ . let ω τ = (ω τ 1 , ..., ω τ c ) ∈ [0, 1] c be the multiplier of each of the class specific gradients to vary the learning rate for each class. in real applications, we often have a style transfer problem with unbalanced training data. for instance, when training formality style transfer models, the number of formal/positive sentences is normally much larger than the number of informal/exclusive sentences. also, denote γ τ = (γ τ 1 , ..., γ τ l ) ∈ [0, ∞) l to be the multipliers of the original learningrate α, where the new learning rate becomes γ τ 1 α, γ τ 2 α, ..., γ τ l α. note that the value of γ is task-dependent (e.g., sample size of the training data from each task), and is meant to deal with the small data problem in multi-pair text style transfer. moreover, since the text data collected from every source or tenant is very hard to be aligned, it is common to have training data with significantly different context. we can treat this as an out of dis-tribution problem and this can be reflected on the value of initial parameters. we use z τ to modulate the initial parameter θ for each task. specifically, z τ relocates the initial θ to a task-dependent starting point prior to the learning process. we unify these properties as the learning framework below: θ 0 = θ * z τ , and for @xmath1, ..., k : ## learning the balancing variables through variational inference we now discuss how to find the most suitable value of each balancing variable. we employ the variational inference framework from probabilistic maml @xcite and taml @xcite to extract the task-specific information. the variational inference framework is used to compute posterior distributions for the balancing variables z τ , γ τ , ω τ . assume the training data to solve the optimization problem requires determining the true posterior p(φ τ |d τ , d τ test ), which is intractable. we resort to variational inference with a tractable figure 3 : the inference network for generating posterior distribution of balancing variables γ, z, and ω for task i. given that each balancing variable is independent, q(φ τ |d τ ; ψ) can therefore be fully factorized we assume each single dimension of q(φ τ |d τ ; ψ) follows a uni-variate gaussian distribution with trainable mean and variance. given φ τ s ∼ q(φ τ |d τ ; ψ), we then use the monte-carlo approximation on eq (3) as a new objective: to better model the variational distribution q(φ τ |d τ ; ψ), an informative representation encoded from the training dataset d τ is necessary. in this case, the inference network can capture all useful statistical information in d τ to recognize its imbalances. we use a two-stage hierarchical set encoder, for a given text style transfer task, we first encodes each class, and then encodes the whole set of classes. define the encoder statisticspooling(•) that generates concatenation of the class statistics such as mean, variance and cardinality. the twostage encoder first encodes all text sentences of each class into s c , followed by encoding representations of the whole set of classes: ## task-adaptive style transfer we discuss formulation of multi-pair text-style transfer problem using the taml framework. an overview of our method is shown in figure 2 . we assume training data in each task could either be parallel (task 1 and 4) or non-parallel (task 2 and 3). the number of training samples in task i is represented by n i , which is not necessarily equal for each task. in addition, the class distribution in non-parallel training data is heavily skewed. we now formulate our problem as follows. given a distribution of similar tasks p(τ ), each task represents performing text style transfer on a certain dataset d τ . define a generic loss function l and shared parameters θ within tasks, the algorithm 1 multi-pair text style transfer via taml 1: input: style pair for each task τ , {(s τ , s τ )} t τ =1 , parameters α, β, 2: meta-training procedure: 3: while not done do 4: train inference network q(φ τ |d τ ; ψ) by minimizing objective (4) 6: obtain balancing variables {z τ , γ τ , ω τ } ∼ q(φ τ |d τ ; ψ) 7: initialize sub-learner with θ τ 0 = θ * z τ 8: sample batch data from d τ s 10: update parameters for task τ using sample batch data from d τ t 13: evaluate l(θ τ k , d τ t ) 14: ## experiments we conduct experiments on multiple style-transfer datasets: shakespeare @xcite , yelp reviews @xcite and an internal dataset from a company that contains formal/informal text sentences. performing style transfer on each of the above dataset defines a unique task. the shakespeare dataset contains 21k parallel sentences, which includes original text style and shakespeare's style. the maximum length of the sentences is 20. the yelp dataset contains around 252k sentences of positive and negative restaurant reviews, where we use a maximum length of 15 to conduct the experiment. we evaluate our method using state-of-the-art transformers including bert, gpt-2, t5, and vae @xcite designed for style transfer by learning disentangled representations. our baseline method in-cludes regular model training without distinguishing the difference between tasks, and the maml method in eq (1) to fine-tune the style transfer models on multiple distinct tasks, which has also been proposed by @xcite . we then employ algorithm 1 to adaptively fine-tune the style transfer models for each task. the unbalanced training data is created by sampling from each class at different rate (75% positive class, 25% negative class). we use the pretrained transformers in huggingface library @xcite as our initial style transfer models. specifically, we build a two-head model (figure 4 ) on top of the decoders where each head is composed of multiple dense layers. we do not perform end-to-end training for the entire transformer but only train the two-head model. the model input is the sentence and style pairs (x τ , s τ ) while the forward propagation of transformer's output to each model head is dependent on the style labels. the resulting output sentences are style dependent, and one can perform text-style transfer by flipping the style labels during the inference phase. similarly, we use both baseline and taml to train vae and obtain disentangled style and content representations, and replace the style embedding during the inference stage to get style transferred sentences. note that we focus on improving the fine-tuning part of text style transfer models, while we do not modify the model structure themselves. in terms of content preservation, the objective function of the vae model proposed by @xcite ) contains a content-oriented loss, while for other transformer-based models, we designed the loss l to be the cross entropy loss between y τ and x τ , or between y τ and x τ in non-parallel situations. for bert, gpt-2, and t5, we use the built-in vocabulary within the transformers library. adam optimizer is used with learning of 5 × 10 -4 to train the model. the batch size is set to 16 and the model is trained for 100 epochs. we build the two-head model by using 6 fully connected layers with hidden size of 256 and relu activation function. the parameters are chosen empirically with the best performance. for vae approach, we use the same parameter settings as reported in @xcite . as for nn 1 in inference network, we used two consecutive blocks of 3 × 3 convolution layer followed by 2 × 2 max pooling layer, the output is then fed into one fully connected layer for statistics pooling. we then use two fully connected layers for nn 2 . all the activation functions are relu. we evaluate competing methods on quality and accuracy of style transfer. the adopted metrics are common choices among recent works. bleu: we use bleu @xcite ) score to evaluate the content preservation, the scores are calculated using scarebleu @xcite . when parallel sentences are available, we compute the bleu score between the style transferred sentences y τ and the ground truth sentences x τ . otherwise, we use the original sentences x τ instead. ppl: we implemented a bigram language model @xcite to quantitatively evaluate the fluency of a sentence. the language model is trained on the target-style domain, and we report the ppl of the generated sentences. accuracy: we also trained a textcnn classifier @xcite simultaneously while training style transfer models. the trained classifier is then used to evaluate the classification accuracy on the generated sentences. ## conclusion in this paper, we investigated meta-learning approaches applied to text-style transfer, for situations with multiple data sources. given the distinct context and total amount of data, we propose a task-adaptive meta-learning approach to fine-tune style-transfer models. the proposed method introduces three balancing variables with probabilistic distributions, which can be encoded from training data. these balancing variables are then used to solve class and task imbalance problems. empirically, we found that taml improves the styletransfer performance on multiple models. in the future, we wish to explore generating style variations in more fine-grained levels (for @xmath2.