## 3 Proposed Methodology
### Baselines
In the keyword-matching baseline, the lexicon of vulgar words provided to the method is considered as a list of features, while the matching procedure is treated as classification in the sense that if at least one vulgar word from the lexicon is matched in the input sentence, the sentence is considered vulgar. Since the baseline method for classification is only based on simple keyword matching, we compared the three above-mentioned methods for keyword extraction to evaluate which of the keyword extraction methods would be the most effective and efficient and if we could find a method with sufficiently high efficiency and efficacy. For efficiency, we consider the amount of human effort put into preparing the lexicon, while for efficacy, we consider the method's accuracy in reclassifying the sentences into either vulgar or non-vulgar. The whole process of keyword extraction and comparison of all three methods is shown in Figure 7.  
Firstly, in the purely manual extraction method, the human annotators read all sentences and manually extracted relevant vulgar expressions. This resulted in a total of 1010 vulgar words. The total accuracy was 0.648. Next, we aimed to propose a method capable of fully automatic (no human effort required) or semi-automatic (only a limited human effort required comparing to fully manual method) extraction of vulgar keywords. Firstly, to initially extract vulgar keyword candidates, we applied _TF-IDF_ and probability of occurrence. Here, _TF_, or term frequency, is calculated by dividing the occurrences of a specific word (or \"term\") in a document by the number of all terms in that document [57], as in Equation (1). \[TF=\frac{\text{Frequency of a certain word in the document}}{\text{Word count in the entire document}} \tag{1}\]  
Next, _IDF_, or inverse document frequency, determines the importance of keywords in a text and is calculated as a logarithmically scaled inverted division of the number of documents containing the term and total number of documents, as shown in Equation (2). \[IDF=\text{log}_{2}\bigg{(}\frac{\text{Total documents}}{\text{Documents with a particular term}}\bigg{)} \tag{2}\]  
Finally, in order to generate the _TF-IDF_ measure, the _TF_ and _IDF_ are multiplied, as in Equation (3). \[\textit{TF-IDF}=\textit{TF}\times\textit{IDF} \tag{3}\]  
By calculating _TF-IDF_ for all words in the two groups of sentences (vulgar and non-vulgar) we obtain the list of words, where the higher _TF-IDF_ score for the vulgar group represents a higher probability of the word being the most representative of the vulgar group, which by assumption should be equivalent to the word being potentially vulgar. In the evaluation, we test the efficacy of this purely TF-IDF-based method without any additional filtering. However, in reality it is not always true, and many non-vulgar words also become included in the list. Specifically, one can observe how many actual vulgar words are included in the first ten, twenty, etc., words on the list, as represented in the result and discussion section. To solve this problem, we added an additional method to delete the words that have the highest probability of being non-vulgar from the list of top TF-IDF vulgar word candidate terms. The method is explained as follows. The idea for this step was borrowed from [PERSON] and [PERSON]'s (2021) method for the automatic extraction of technical terms from larger corpora [58]. To perform this, firstly, we calculate the probability of occurrence (PoO) for each word in either vulgar or non-vulgar class according to [PERSON] et al. 's (2019) pattern extortion method [54], represented in Equation (4), which is a simplified sigmoid function normalizing the weighted score between 1 (completely vulgar) and \(-1\) (completely non-vulgar). \[PoO=\left(\frac{\text{Occurrence of Vulgar word}}{\text{(Occurrence of Vulgar word + Occurrence of Non-vulgar word)}}-0.5\right)*2 \tag{4}\]  
From that list, we then take all words for which the weight was \(-1\) (appeared only in the non-vulgar group).