## 4 Method
### Text Standardization  
Training a high-quality model requires operating with clean data. The better the data are pre-processed and standardized, the more accurate model predictions will be, especially considering that this study follows the transfer learning approach using the pre-trained BERT model. Authors of BERT used BooksCorpus and English Wikipedia datasets [6] for pre-training; thus, it is important to make text messages have a standard language form. Text pre-processing is widely addressed in natural language processing and text classification studies. Each problem and data context requires an individual approach. A relevant example of data pre-processing in the healthcare domain is presented in the study \"An Analysis on Large Language Models in Healthcare: A Case Study of BioBERT\" [19]. Authors describe general data cleansing, as well as standardizing medical terms and applying custom tokenization \"to accommodate the unique vocabulary and structure of biomedical and clinical texts... specialized tokenizers may be needed to handle medical terminology, abbreviations, and symbols\" [19]. A similar problem is faced in the current research and presented later: the wide use of chat-style language with slang abbreviations, different word spelling, etc. This lexis appears as different tokens for the machine learning model. Hence, the model should be either initially trained on this data or adapted to processing such data. The first approach is not applicable since this study employs the transfer learning method, while the second one can be achieved by implementing custom tokenizers or additional pre-processing of the data. Another relevant study is \"CARE: Contextualized Affect Representations for Emotion Recognition\" [20], in which authors perform emotion recognition on the dataset extracted from Twitter. Despite differences in audiences and texting styles, both Twitter and Reddit are discussion platforms on the Internet and present similar language. Authors describe an identical problem of slang and coded words having the same meaning, such as \"tnx\" and \"thanks\" or \"waaaaking me\" instead of \"waking me\". That research provides its solution using \"graph-based pattern representations\" to extract emotion-relevant information. This sub-section continues with the authors' research of the language used in the GoEmotions dataset and the implementation of the cleansing process step by step. This text pre-processing can be simmilarly applied to other datasets sourced from Internet discussion platforms, such as Reddit, Twitter, etc. However, additional processing and analysis might also be required, especially if texts are inclined towards some specific domain, as in the BioBERT study example mentioned before. To begin with, the authors apply the unidecode module [12] to the text to remove non-ASCII characters. The main advantage of this tool is that it will not only remove any wrong characters, such as unreadable spaces and emojs but also replace them with correct interpretation if possible. For example, the \"Latin small letter a with diaeresis\" character (a) becomes the \"Latin small letter a\", while the \"single comma quotation mark\" character becomes a usual \"single quotation mark\". This resolves any encoding issues, as well as removes different spellings of the same word. The dataset is depersonalized and contains placeholders instead of name or religion references. For example, \"I'm not talking about [RELIGION] anymore though...\", \"[NAME]...I'm sorry. This is just wrong. I, can't.\" These placeholders were removed from the text so that they do not affect the model and the model will be less biased. The dataset contains text commentaries written in English using an informal chat-style language. To give it a standard look, the authors replace English contractions [21] with their full spelling where it is unambiguous. For instance, \"can't\" is simply replaced with \"can not\".