From Data to Grassroots Initiatives: Leveraging Transformer-Based Models for Detecting Green Practices in Social Media (2025)

Abstract: Green practices are everyday activities that support a sustainable relationship between people and the environment. Detecting these practices in social media helps track their prevalence and develop recommendations to promote eco-friendly actions. This study compares machine learning methods for identifying mentions of green waste practices as a multi-label text classification task. We focus on transformer-based models, which currently achieve state-of-the-art performance across various text classification tasks. Along with encoder-only models, we evaluate encoder-decoder and decoder-only architectures, including instruction-based large language models. Experiments on the GreenRu dataset, which consists of Russian social media texts, show the prevalence of the mBART encoder-decoder model. The findings of this study contribute to the advancement of natural language processing tools for ecological and environmental research, as well as the broader development of multi-label text classification methods in other domains.

## introduction growing environmental challenges and climate change have led governments to develop adaptation and mitigation policies. these policies are expected to influence people's behavior, shaping what are known as social practices @xcite . however, it is unclear whether these practices are becoming more eco-friendly or how they can be improved to better address the environmental crisis. green practices are social actions aimed at harmonizing the relationship between people and the environment by reducing resource consumption, waste, pollution, and emissions @xcite . studying the prevalence of green waste practices is crucial to give people new ideas for promoting and expanding these actions @xcite . despite this, awareness of these practices in society remains limited. to fill this gap, researchers need to collect and analyze large amounts of data on green waste practices. social media provides a rich repository of environmental information, but manually reviewing posts is time-consuming and inefficient. automated approaches, such as deep learning and content analysis, can contribute to solving this problem. however, to date, only a limited number of studies have used big data tools to investigate green waste practices @xcite @xcite . in this work, we explore the possibilities of natural language processing (nlp) tools for detecting mentions of green waste practices in social media. this task is framed as a multilabel text classification problem. since large language models (llms) demonstrate superior performance across various nlp tasks, the focus of our research is on applying pre-trained language models (plms) to detect mentions of green waste practices. we seek to answer the following research questions (rqs): the contributions of this paper can be summarized as follows. to address rqs, we present the first large-scale comparison of encoder-only, encoder-decoder, and decoder-only transformerbased models for the task of detecting mentions 1 of green waste practices in social media. several label descriptors to represent data for generative models were evaluated. the presented evaluation has revealed that encoder-decoder models, namely mbart, can outperform both encoderonly and decoder-only models for detecting mentions of green waste practices. the obtained results provide insights into the potential of nlp to address environmental challenges. our findings can also be used in other similar nlp tasks related to multi-label text classification. 2020 ); @xcite . @xcite were the first to use the t5 model @xcite for multi-label classification. however, their approach utilized only the encoder component of the model, omitting the use of the model's decoder. to date, there are several studies that used the encoder-decoder models fine-tuned for multilabel text classification in a generative manner. @xcite analyzed four methods for multi-label classification based on t5 and evaluated several types of label descriptors. savci and das (2024) compared multi-label bart @xcite and bert; however, the results of bart were lower. up to now, there are only several approaches to perform multi-label text classification using decoder-only models. @xcite ; siddiqui et al. (2024) performed fine-tuning of a pre-trained gpt2-model @xcite with different prompt formats. @xcite analyzed the performance of gpt-3 @xcite for fine-grained multi-label tweet classification using zero-shot labeling. @xcite revealed that few-shot learning consistently outperforms zero-shot learning. ## data this study uses the greenru foot_0 dataset @xcite for detecting mentions of green waste practices in russian social media texts. greenru consists of 1,326 posts in the russian language with an average length of 880 symbols collected from online green communities. the posts have a sentence-level multi-label markup indicating green waste practices mentioned in them. the average length of a sentence is 110 symbols. nine types of green waste practices @xcite were used for the annotation of greenru: 1) waste sorting, i.e. separating waste by its type; 2) studying the product labeling to indicate product packaging as a type of waste; 3) waste recycling, i.e. transforming waste materials into reusable resources for future production.; 4) signing petitions to influence the authorities; 5) refusing purchases to reduce consumption and environmental footprint; 6) exchanging an unnecessary item or service for a desired one; 7) sharing things with other people for a fee or free of charge; 8) participating in actions to promote responsible consumption, including workshops, festivals, lessons, etc.; 9) repairing things as an alternative to throwing them away. the distribution of mentions of green waste practices in the dataset is presented in figure 1 . greenru is pre-split into training and test sets, with their characteristics presented in ## models in this study, we compared several approaches to multi-label text classification to detect mentions of green waste practices in social media. alongside encoder-only plms, which are traditionally used for multi-label text classification, we also employed encoder-decoder and decoder-only models. all transformer-based plms were implemented using the simple transformers foot_1 and transformers @xcite libraries. the overview of models is shown in ## results and discussion all four plms outperformed baselines. the highest result of 67.88% in terms of the multi-label f1-score was shown by rubertlarge. encoder-decoder plms mostly achieved the results above baselines. the best scores for rut5 were obtained using simplified and original label descriptors (62.51% and 60.54% respectively). the use of the numbers, special tokens, and onehot label descriptors did not increase the mlp results. the one-hot label descriptors did not even surpass knn (34. 95%), indicating that the rut5 model struggles to interpret this method of label representation. mbart demonstrated the highest score using simplified label descriptors (69.76%). the second and third highest scores were obtained with the original and simplified-eng labels descriptors (69.49% and 69%). the use of the numbers and original-eng label descriptors also improved the results of encoder-only plms (68.91% and 68.53%). the one-hot and special token label descriptors demonstrated the multilabel f1-score of 67.12% and 65.71% respectively which did not surpass rubert-large but outperformed baselines. in the lowest results in comparison to encoder-only and encoder-decoder plms. the highest result of rugpt was obtained using the simplified label descriptors (51.08%). the use of the numbers, special tokens, and one-hot label descriptors showed the results below the knn baseline. the instruction-based t-lite model also did not demonstrate high results. the use of prompt-based learning obtained 42.04% and 47.77% in terms of the multi-label f1-score. despite the fact that incorporating explanations of green waste practices led to a more than 5% improvement in performance, t-lite failed to outperform the mlp baseline. figure 2 shows the performance growth using different label descriptors in comparison to the mlp baseline. the figure reveals that the labels descriptors based on text representation (original, simplified, original-eng, and simplified-eng) show higher results than the labels descriptors based on numerical and special token representation. for all three models (rut5, mbart, rugpt) the best results were achieved using the simplified label descriptors. the rqs were aimed to evaluate the effectiveness of plms in detecting mentions of green waste practices on social media and to determine which transformer-based model architectures are the most effective for this task. our experiments demonstrated that the performance of plms varies depending on their architecture and model type. encoder-only models achieved the multilabel f1 score values between 65.28% and 67.88%, showing consistent and relatively strong performance. this supports their common use in multilabel classification tasks. however, the best result in our experiments was achieved by the mbart model (69.76%), highlighting the strong potential of encoder-decoder models for multi-label classification. label descriptors greatly affect encoderdecoder models; for example, rut5 results vary from 34.95% to 62.51%. decoder-only models, including instruction-based ones, showed the poorest performance in our experiments. however, the results indicate that incorporating explanations into the prompt can enhance the performance of instruction-based models. ## conclusion in this work, we explored the efficiency of plms for detecting mentions of green waste practices in social media. to address rqs, we compared encoder-only, encoder-decoder, and decoder-only plms. our findings showed that encoder-only and encoder-decoder models generally outperformed decoder-only models. mbart achieved the best performance and revealed the most suitable label descriptors for generative plms in the multi-label text classification task. this current study is limited by the use of only one data set to detect green waste practices. this is due to the fact that, to the best of the authors' knowledge, greenru is currently the only freely available dataset specifically annotated for this task. a potential future direction for this research could involve applying transfer learning techniques and generating texts to train models for other languages. another possible limitation of this study is the use of general-domain models. further research can investigate the role of indomain pre-training for this task. future research directions can additionally include exploring additional multilingual models beyond mbart and the mlp baseline to expand comparative insights. investigating models with billion-scale parameters while incorporating peft (parameter-efficient fine-tuning) approaches could also enhance performance and efficiency. the results obtained in this study allowed us to identify the most effective models for searching for green waste practices on social networks. using these models, the following management tasks can be solved: the information obtained through plms can be used by authorities, eco-businesses, and activists to promote behavioral change, support green innovation and promote sustainable social practices. the models for automatically detecting mentions of green waste practices make researching these practices easier and cheaper as they replace experts in dealing with textual information. additionally, these methods allow processing large amounts of textual data that are not accessible to expert analysis.