Mask and Regenerate: A Classifier-based Approach for Unpaired Sentiment Transformation of Reviews for Electronic Commerce Websites. (2022)

Abstract: Style transfer is the task of transferring a sentence into the target style while keeping its content. The major challenge is that parallel corpora are not available for various domains. In this paper, we propose a Mask-And-Regenerate approach (MAR). It learns from unpaired sentences by modifying the word-level style attributes. We cautiously integrate the deletion, insertion and substitution operations into our model. This enables our model to automatically apply different edit operations for different sentences. Specifically, we train a multilayer perceptron (MLP) as a style classifier to find out and mask style-characteristic words in the source inputs. Then we learn a language model on non-parallel data sets to score sentences and remove unnecessary masks. Finally, the masked source sentences are input to a Transformer to perform style transfer. The final results show that our proposed model exceeds baselines by about 2 per cent of accuracy for both sentiment and style transfer tasks with comparable or better content retention.

## introduction a text style is a feature that specifies text. the objective of style transfer is to rewrite a given sentence into a target-style domain with the preservation of semantic content. in this paper, we follow the opinion @xcite that textual sentiment should also be treated as styles and conduct experiments to transfer sentiments of sentences collected from three electronic commerce websites. e.g. "the food here is delicious." (positive) → "the food here is gross." (negative) a key issue is that the lack of available parallel data has a considerable impact on the use of supervised learning. it results in the majority of recent studies concentrating on unpaired text transfer approaches @xcite @xcite . compare with related work, in this example, we transfer a negative sentence to a positive one. the [mask] of the word 'not' has been removed by a language model. methods based on word-level operations @xcite have become one of the most frequently used approaches because they ensure high content preservation. the approach we introduce in this paper mainly follows two works, the delete-retrieval-generate (drg) model @xcite and the tag-and-generate model (tag) @xcite . the motivation behind the drg model is to delete stylecharacteristic words by computing the frequency of occurrence of words, retrieve one similar sentence in the target style corpus and generate a new sentence which is the result of crossing the two sentences. by following the idea of drg, the tag model is proposed. the tag model calculates tf • idf scores @xcite to determine style-characteristic words and it includes a tagger to insert a special symbol ' @xcite ' into the input sentences, that will be filled by target-stylecharacteristic phrases. we identify the following weak points in these models: to tackle the above problems, we suggest that: our main contributions are as follows: ## methodology for an overview, we train two independent modules called the masker and the generator respectively. the masker consists of a text mlp and an lm. for an input sentence x (i) a , the masker masks or deletes style-characteristic words to generate a content representation sequence z a . the generator is a standard transformer which is used to insert style-characteristic words into the sequence z a and replace masks with attribute words of style b. ## conclusion we proposed a novel word substitution based approach called mask-and-regenerate for sentiment and style transfer. it can be regarded as a generator in a generative adversarial network to facilitate the training of a detector which can better identify fake comments on electronic commerce platforms. due to the lack of available parallel corpora, the original sentences were edited to delete, insert, or substitute words. we carried out a study on the neural-based style-characteristic word recognition and the automatic application of edit operations in the domain of style transfer. for sentiment and formality transfer, the results showed that our proposed model generally outperforms baselines by about 2 per cent in terms of accuracy with comparable or better bleu scores.