Unsupervised style transfer in text has previously been explored through the sentiment transfer task . The task entails inverting the overall sentiment polarity in a given input sentence , while preserving its content . From the Aspect-Based Sentiment Analysis ( ABSA ) task , we know that multiple sentiment polarities can often be present together in a sentence with multiple aspects . In this paper , the task of aspect-level sentiment controllable style transfer is introduced , where each of the aspect-level sentiments can individually be controlled at the output . To achieve this goal , a BERT-based encoder-decoder architecture with saliency weighted polarity injection is proposed , with unsupervised training strategies , such as ABSA masked-languagemodelling . Through both automatic and manual evaluation , we show that the system is successful in controlling aspect-level sentiments . With a rapid increase in the quality of generated text , due to the rise of neural text generation models ( Kalchbrenner and Blunsom , 2013 ; Cho et al . , 2014 ; Sutskever et al . , 2014 ; Vaswani et al . , 2017 ) , controllable text generation is quickly becoming the next frontier in the field of text generation . Controllable text generation is the task of generating realistic sentences whose attributes can be controlled . The attributes to control can be : ( i ) . Stylistic : Like politeness , sentiment , formality etc , ( ii ) . Content : Like information , entities , keywords etc . or ( iii ) . Ordering : Like ordering of information , events , plots etc . Controlling sentence level polarity has been well explored as a style transfer task . Zhang et al . ( 2018 ) used unsupervised machine translation techniques for polarity transfer in sentences . Yang et al . ( 2018 ) § equal contribution The service was speedy and the salads were great , but the chicken was bland and stale . The service was slow , but the salads were great and the chicken was tasty and fresh . used language models as discriminators to achieve style ( polarity ) transfer in sentences . Li et al . ( 2018a ) proposed a simpler method where they deleted the attribute markers and devise a method to replace or generate the target attribute-key phrases in the sentence . In this paper we explore a more fine-grained style transfer task , where each aspect 's polarities can be changed individually . Recent interest in Aspect-Based Sentiment Analysis ( ABSA ) ( Pontiki et al . , 2014 ) has shown that sentiment information can vary within a sentence , with differing sentiments expressed towards different aspect terms of target entities ( e.g. ' food ' , ' service ' in a restaurant domain ) . We introduce the task of aspect-level sentiment transfer -the task of rewriting sentences to transfer them from a given set of aspect-term polarities ( such as ' positive sentiment ' towards the service of a restaurant and a ' positive sentiment ' towards the taste of the food ) to a different set of aspect-term polarities ( such as ' negative sentiment ' towards the service of a restaurant and a ' positive ' sentiment towards the taste of the food ) . This is a more challenging task than regular style transfer as the style attributes here are not the overall attributes for the whole sentence , but are localized to specific parts of the sentence , and multiple opposing at-tributes could be present within the same sentence . The target of the transformation made needs to be localized and the other content expressed in the rest of the sentence need to be preserved at the output . An example of the task is shown in Figure 1 . For successful manipulation of the generated sentences , a few challenges need to be addressed : ( i ) . The model should learn to associate the right polarities with the right aspects . ( ii ) . The model needs to be able to correctly process the aspectpolarity query and accordingly delete , replace and generate text sequence to satisfy the query . ( iii ) . The polarities of the aspects not in the query should not be affected . ( iv ) . The non-attribute content and fluency of the text should be preserved . We explore this task in an unsupervised setting ( as is common with most style-transfer tasks due to the lack of an aligned parallel corpus ) using only monolingual unaligned corpora . In this work , a novel encoder-decoder architecture is proposed to perform unsupervised aspect-level sentiment transfer . A BERT ( Devlin et al . , 2019 ) based encoder is used that is trained to understand aspect-specific polarity information . We also propose using a ' polarity injection ' method , where saliency-weighted aspect-specific polarity information is added to the hidden representations from the encoder to complete the query for the decoder . In this paper , the task of aspect-level sentiment style transfer has been introduced , where stylistic attributes can be localized to different parts of a sentence . We have proposed a BERT-based encoder-decoder architecture with saliency-based polarity injection and show that it can be successful at the task when trained in an unsupervised setting . The experiments have been conducted on an aspect level polarity tagged benchmark dataset related to the restaurant domain . This work is hopefully an important initial step in developing a fine-grained controllable style transfer system . In the future , we would like to explore the ability to transfer such systems to data-sparse domains , and explore injecting attributes such as emotions to targets attributes in larger pieces of text .

Challenge: Controllable generation such as the sentiment transfer task where one sentence with multiple aspects can have multiple sentiment polarities is the next frontier.
Approach: They first propose the aspect-level sentiment controllable style transfer task where each sentiment can be indivisually controlled and a BERT-based encoder-decoder with unsupervised training strategies.
Outcome: Automatic and manual evaluation with a restaurant dataset shows that the proposed model can successfully perform aspect-level sentiment control when trained in an unsupervised setting.