# GreenRu: A Russian Dataset for Detecting Mentions of Green Practices in Social Media Posts
## 3 Experiments and Evaluation
### Handling Class Imbalance
#### 3.3.1 Data Augmentation  
Since green practices have varying degrees of prevalence, the impact of different data augmentation methods to reduce class imbalance was explored. The impact of data augmentation was investigated using one-versus-rest models. For all augmentation methods, one synthetic example was generated for every positive example in the training subset. Thus, the number of entries of the target practice in the training subset was two times larger than in the original subset.  
The following data augmentation methods were compared.  
* Simple duplication, a method that consists of duplicating the texts from the original dataset.
* Backtranslation, a method using backtranslating phrases between any two languages. The BackTranslation library ([[https://pypi.org/project/BackTranslation](https://pypi.org/project/BackTranslation)]([https://pypi.org/project/BackTranslation](https://pypi.org/project/BackTranslation)) (accessed on 20 May 2024)) based on googletrans and English as a target language was utilized.
* Text generation (RuGPT3). RuGPT3 ([[https://huggingface.co/ai-forever/rugt3](https://huggingface.co/ai-forever/rugt3) medium_based_on_gpt2]([https://huggingface.co/ai-forever/rugt3](https://huggingface.co/ai-forever/rugt3) medium_based_on_gpt2) (accessed on 20 May 2024)) [37] was fine-tuned for predicting the next word using the following parameters: the number of epochs--3, the maximum sequence length--256 tokens, and the learning rate--\(4\times 10^{-5}\). Next, each text mentioning minority green waste practices was transformed in the following way. If the text was more than five tokens long, it was truncated to five tokens. If the text length was shorter, the entire text was used. For each transformed text, a continuation was generated using the fine-tuned RuGPT3.
* Decoding masked sentences (RuT5). Word-level masking was applied, inspired by [45], by replacing a continuous chunk of \(k\) words \(w_{i},w_{i+1}...w_{i+k}\) with a single mask token \(<mask>\). Masking was applied to 50% of the random words in each text. With different random seeds, 10,000 training examples were produced and RuT5-base ([[https://huggingface.co/ai-forever/ruT5-base](https://huggingface.co/ai-forever/ruT5-base)]([https://huggingface.co/ai-forever/ruT5-base](https://huggingface.co/ai-forever/ruT5-base)) (accessed on 20 May 2024)) [37] was fine-tuned to decode the original sequence given a masked sequence. To better distinguish between texts containing mentions of different green waste practices, the types of the corresponding practices were added at the beginning of each masked text as control codes (for example, _waste sorting: text_). No control codes were added to target texts. Thus, the augmentation scheme was as follows. 1. _Creating data for fine-tuning RuT5_. Forming a dataset of 10,000 examples for the fine-tuning of RuT5-base: adding control codes and masking to the original texts, using original texts as a target. 2. _Fine-tuning on masked texts_. RuT5-base was fine-tuned over three epochs with a maximum sequence length of 256 tokens, the learning rate of 4e-5, and the AdamW optimizer. 3. _Masking and generating_. For each text from the minority classes, 50% of tokens were randomly masked according to the procedure described before, the corresponding control code was added, and a synthetic example was generated.
* ChatGPT, a cloud-based artificial intelligence chatbot that utilizes OpenAI's GPT-3.5-turbo model. As a large language model, ChatGPT can create novel and contextually relevant responses to a given prompt, making it an ideal tool for data augmentation [46; 47; 48]. To obtain generated texts, ChatGPT was provided with the original text from the data-set using a prompt. The model's response was used as a synthetic text, which was subsequently added to the original data during the training of the classifier. The following prompt translated into Russian was used, inspired by [46]: _Please rephrase the following sentence: {text} (\"Ioxak.nytica, neepprasnyi cheylytomene prediokeme: {text})_.