## 3 Proposed Approach  
This section offers a formalization of the problem to solve and describes how the proposed models approach it. Firstly, the formalization of the problem considers a dataset \(\mathbf{D}\) formed by sentences in natural language of different styles. Each example in the dataset is \((\mathbf{x_{i}},\mathbf{s_{i}})\), in which \(\mathbf{x_{i}}\) is a sentence and \(\mathbf{s_{i}}\in\mathbf{S}\) is the style of the sentence. We refer to the text in the same style as the text that shares specific attributes, that is, \(\mathbf{S}=\{positive,formal,toxic,neutral,\dots\}\). We denote \(\mathbf{c}\) as the set of words that represent the content or information that the sentence contains, such that \(Style(\mathbf{c})\
otin\mathbf{S}\), and \(\mathbf{a}\) as the collection of words with stylistic information. Therefore, the goal of text style transfer is to, given a source sentence \(\mathbf{x}\) with source style \(\mathbf{s^{src}}\) and a desired target style \(\mathbf{s^{tgt}}\), output a new sentence \(\mathbf{y}\) that maintains as much content \(\mathbf{c}\) as possible from \(\mathbf{x}\) but whose style is \(\mathbf{s^{tgt}}\). In particular, in the detoxification task, we consider \(\mathbf{S}=\{toxic,neutral\}\) where \(\mathbf{s^{src}}=toxic\) and \(\mathbf{s^{tgt}}=neutral\). In order to generate this transferred style sentence \(\mathbf{y}\) where \(Style(\mathbf{y})=\mathbf{s^{tgt}}\) given an input sentence \(\mathbf{x}\) with style \(\mathbf{s^{src}}\), our LexiconGST model tries to learn \(P(\mathbf{y}|\mathbf{x},\mathbf{s^{tgt}})\). We base our work on the BlindedGST [19] method; hence, the LexiconGST model is a two-step process: style deletion and style-conditioned generation. First, we describe the style deletion component, which aims to separate style and content from the input sentence by modeling the distribution \(P(\mathbf{c},\mathbf{a}|\mathbf{x})\). The original sentence \(\mathbf{x}\) can be fully reconstructed from \(\mathbf{c}\) and \(\mathbf{a}\). For example, given \"your stupid contributions are shift\", then the toxic-related words, \"stupid\" and \"shit\", should be removed. This approach is founded on the \"input reduction\" method proposed in the work of [PERSON] et al. [35], which suggests that the style of a sentence is heavily affected by the contribution of specific words or sub-phrases associated with such style. Consequently, a style classifier would be confused about the style of a sentence \(\mathbf{x}\) whose style is \(\mathbf{s}\) if the style-related attributes \(\mathbf{a}\) are removed from the sentence. To detect these style attributes, we use a weighting algorithm to measure the contribution to style that each token in \(\mathbf{x}\) makes. In the case of neutral style, we measure the contribution of a token using the method of [PERSON] et al. [19]. A BERT classifier is fine-tuned for toxic sentence classification, and then it is used to filter out neutral tokens based on the attention weights of a selected attention head. However, when detecting and removing toxic tokens, we propose an algorithm (see Algorithm 1) that exploits token embedding similarities with a toxic lexicon. First, we use the embedding layer from a RoBERTa [36] toxicity classifier because these embeddings encode some style information. Since we are interested in the style similarity of tokens rather than semantic similarity, we follow the intuition that this embedding representation is better for our task than other semantic embeddings. Then, we employ a similarity function between the sentence's token embeddings and the lexicon word embeddings and remove tokens whose similarity is above a predefined threshold. Unfortunately, due to how word embeddings are generated, some words like \"betwrayed\" (word inside one of the toxic lexicons used) and \"portrayed\" have high embedding similarity, which will result in the deletion of \"portrayed\" from the source sentence as if it were a toxic style attribute, but in reality it should remain as part of the source content.