Unsupervised Aspect-Level Sentiment Controllable Style Transfer (2020)

Abstract: Unsupervised style transfer in text has previously been explored through the sentiment transfer task. The task entails inverting the overall sentiment polarity in a given input sentence, while preserving its content. From the Aspect-Based Sentiment Analysis (ABSA) task, we know that multiple sentiment polarities can often be present together in a sentence with multiple aspects. In this paper, the task of aspect-level sentiment controllable style transfer is introduced, where each of the aspect-level sentiments can individually be controlled at the output. To achieve this goal, a BERT-based encoder-decoder architecture with saliency weighted polarity injection is proposed, with unsupervised training strategies, such as ABSA masked-language-modelling. Through both automatic and manual evaluation, we show that the system is successful in controlling aspect-level sentiments.

## introduction with a rapid increase in the quality of generated text, due to the rise of neural text generation models @xcite @xcite , controllable text generation is quickly becoming the next frontier in the field of text generation. controllable text generation is the task of generating realistic sentences whose attributes can be controlled. the attributes to control can be: (i). stylistic: like politeness, sentiment, formality etc, (ii). content: like information, entities, keywords etc. or (iii). ordering: like ordering of information, events, plots etc. controlling sentence level polarity has been well explored as a style transfer task. @xcite used unsupervised machine translation techniques for polarity transfer in sentences. yang et al. (2018) § equal contribution the service was speedy and the salads were great, but the chicken was bland and stale. the service was slow, but the salads were great and the chicken was tasty and fresh. used language models as discriminators to achieve style (polarity) transfer in sentences. @xcite proposed a simpler method where they deleted the attribute markers and devise a method to replace or generate the target attribute-key phrases in the sentence. in this paper we explore a more fine-grained style transfer task, where each aspect's polarities can be changed individually. recent interest in aspect-based sentiment analysis (absa) @xcite has shown that sentiment information can vary within a sentence, with differing sentiments expressed towards different aspect terms of target entities (e.g. 'food', 'service' in a restaurant domain). we introduce the task of aspect-level sentiment transfer -the task of rewriting sentences to transfer them from a given set of aspect-term polarities (such as 'positive sentiment' towards the service of a restaurant and a 'positive sentiment' towards the taste of the food) to a different set of aspect-term polarities (such as 'negative sentiment' towards the service of a restaurant and a 'positive' sentiment towards the taste of the food). this is a more challenging task than regular style transfer as the style attributes here are not the overall attributes for the whole sentence, but are localized to specific parts of the sentence, and multiple opposing at-tributes could be present within the same sentence. the target of the transformation made needs to be localized and the other content expressed in the rest of the sentence need to be preserved at the output. an example of the task is shown in figure 1 . for successful manipulation of the generated sentences, a few challenges need to be addressed: (i). the model should learn to associate the right polarities with the right aspects. (ii). the model needs to be able to correctly process the aspectpolarity query and accordingly delete, replace and generate text sequence to satisfy the query. (iii). the polarities of the aspects not in the query should not be affected. (iv). the non-attribute content and fluency of the text should be preserved. we explore this task in an unsupervised setting (as is common with most style-transfer tasks due to the lack of an aligned parallel corpus) using only monolingual unaligned corpora. in this work, a novel encoder-decoder architecture is proposed to perform unsupervised aspect-level sentiment transfer. a bert @xcite based encoder is used that is trained to understand aspect-specific polarity information. we also propose using a 'polarity injection' method, where saliency-weighted aspect-specific polarity information is added to the hidden representations from the encoder to complete the query for the decoder. ## related work in this section we present an overview of the related literature. ## experiments in this section we report the datasets used for the experiments and the implementation details. ## conclusion and future work in this paper, the task of aspect-level sentiment style transfer has been introduced, where stylistic attributes can be localized to different parts of a sentence. we have proposed a bert-based encoder-decoder architecture with saliency-based polarity injection and show that it can be successful at the task when trained in an unsupervised setting. the experiments have been conducted on an aspect level polarity tagged benchmark dataset related to the restaurant domain. this work is hopefully an important initial step in developing a fine-grained controllable style transfer system. in the future, we would like to explore the ability to transfer such systems to data-sparse domains, and explore injecting attributes such as emotions to targets attributes in larger pieces of text.