## 2 Related Work  
A common approach to tackle the style transfer task involves using a supervised encoder-decoder model to \"translate\" the source sentence into the desired target style [13]. When the source and target belong to the same language, one can leverage pre-trained Language Models (LMs) such as Generative Pre-trained Transformer (GPT) [22] to accomplish this task. This technique can be enhanced by fine-tuning the LMs on relatively small parallel corpora, leading to favourable outputs [23]. Nevertheless, the application of this method is somewhat limited due to the scarcity of sufficiently large parallel data. In contrast, most previous research on text style transfer has been carried out using unsupervised algorithms. Four approaches are considered, which are described below: explicit style-content disentanglement, implicit style-content disentanglement, without style-content disentanglement, and toxic style transfer. The first approach is known as \"explicit style-content disentanglement\" [18]. This style transfer method is relatively straightforward, but highly effective, and involves leaving the sentence unchanged and modifying only the individual words linked to the desired style. The first attempt to achieve such a transfer was introduced by the Delete-Retrieve-Generate [18] framework, where the authors suggest DRG-TemplateBased. First, it removes the style-related words from the input sentence, for example \"nasty\", and retains only the content-specific information such as \"food\". Next, it replaces source-style words with target-style words with the help of a retrieval mechanism based on sentences similar in content and a Seq2 Seq model to alleviate fluency. Ultimately, the performance depends on the robustness of the replacement mechanism. [PERSON] et al. [18] also propose two models which are focused on two phases of DRG, known as DeleteOnly and DeleteAndRetrieve (D&R). Typically, words associated with a particular style are identified based on their frequency in the corpus. Conversely, [PERSON] et al. [19] propose improving the DRG model by using a Delete Transformer (DT) and a Generative Style Transformer (GST). They propose two variants that train the Generative Style Transfomer (GST) in two different ways: BlindedGST (B-GST) and GuidedGST (G-GST). The first variant, B-GST, consists of training the model so that it generates an output sentence given only the content and target style, being blind to specific desired attributes. It is relevant that based on [PERSON]'s work [18], this variant uses a weak retrieval component, so it can be considered a D&G model. In the second variant, G-GST, the model is guided towards generating a target sentence with target attributes. In our work, we extend the DRG framework following the previously presented extensions [18; 19] so that the retrieval component is not used, and we focus our research on improving two components: Delete and Generate. The second approach that can be followed is \"implicit style-content disentanglement\". Rather than performing a straightforward replacement of specific parts in the text, some style transfer models attempt to learn the latent representations of content and style from the input corpus [17]. The model then combines the source text content's representation with the target style's representation to generate the desired text. [PERSON] et al. [17] presented a representative study in the field in which they used adversarial learning to train the encoder of an encoder-decoder network where the input text is encoded without style information (\(\mathbf{z}\)). The decoder generates texts in different target styles when fed with \(\mathbf{z}\). A major concern is whether the reconstruction loss is enough to keep the semantics of the input. To improve this approach, other works have proposed a new loss of cycle consistency [24] that also alleviates the problem of model collapse. [PERSON] et al.