Towards Fine-grained Classification of Climate Change related Social Media Text (2022)

Abstract: With climate change becoming a cause of concern worldwide, it becomes essential to gauge people’s reactions. This can help educate and spread awareness about it and help leaders improve decision-making. This work explores the fine-grained classification and Stance detection of climate change-related social media text. Firstly, we create two datasets, ClimateStance and ClimateEng, consisting of 3777 tweets each, posted during the 2019 United Nations Framework Convention on Climate Change and comprehensively outline the dataset collection, annotation methodology, and dataset composition. Secondly, we propose the task of Climate Change stance detection based on our proposed ClimateStance dataset. Thirdly, we propose a fine-grained classification based on the ClimateEng dataset, classifying social media text into five categories: Disaster, Ocean/Water, Agriculture/Forestry, Politics, and General. We benchmark both the datasets for climate change stance detection and fine-grained classification using state-of-the-art methods in text classification. We also create a Reddit-based dataset for both the tasks, ClimateReddit, consisting of 6262 pseudo-labeled comments along with 329 manually annotated comments for the label. We then perform semi-supervised experiments for both the tasks and benchmark their results using the best-performing model for the supervised experiments. Lastly, we provide insights into the ClimateStance and ClimateReddit using part-of-speech tagging and named-entity recognition.

## introduction the effects of climate change are becoming increasingly apparent, with various natural disasters, including floods, droughts, storms, and fires, increasing in intensity and frequency. the biosphere is changing, endangering the natural resources and agriculture that are essential for our survival. ac-cording to the fifth assessment report of the intergovernmental panel on climate change (ipcc) foot_0 , continued climate change will have severe and irreversible impacts on people and ecosystems worldwide. according to the report, climate change is predicted, with high confidence, to lead to increased intensity and frequency of daily temperature extremes, sea-level rise, ocean acidification, and reduced crop yields. climate change and its effects have become major causes of concern globally, leading to increased participation in public discourse. they have been the subject of various newspaper articles, scientific papers, blogs, and social media threads. although many steps can help control the intensity and effects of climate change, integrating these steps with public policy depends on the vox populi of climate change. there are multiple ways to study and quantify public opinion on climate change. however, traditional methods, including polling, do not take advantage of the growing prevalence and the abundance of public discourse in social media. twitter is one of the most popular social media and serves as a vital data source for determining public opinion and perception of climate change. globally, it has more than 200 million daily active users, with an average annual growth of around 20% in the number of active users. linden (2017) discusses the impact social networks have on developing climate change risk perception, which suggests the importance of understanding discourses in social media for this domain. one of the primary concerns around climate change is polarization, with social media being one of the key influencers of the same. as has been observed in previous works, climate change skepticism has achieved a higher level of visibility in media than scientific literature @xcite . nevertheless, it may help mit-igate differences and spread awareness of information related to climate change. social media also creates an open space for organizations, climate activists, and scientists to reach more people worldwide. ukcop26 and greenpeace are a few examples that use social media platforms to share knowledge about current climate conditions and collaborate with artists, activists, politicians, and academic institutions. hence, the importance of social media outreach in climate change awareness campaigns is immense for effectively reaching a large global audience. this work proposes climate change prevention stance detection and fine-grained classification of climate-change-related social media text. we release two twitter-based english datasets 2 , climat-estance and climateeng, consisting of 3777 tweets, manually annotated for both the tasks. thirdly, we benchmark state-of-the-art text classification models including bert, roberta and distilbert on both the tasks. fourthly, we create a reddit-based pseudo-labeled dataset climatereddit from the best performing model for climatestance and clima-teeng and benchmark its performance based on a smaller manually annotated test dataset. finally, we perform a linguistic feature-based analysis for both the datasets based on part-of-speech tagging and named entity recognition. ## related works early work in analyzing climate-change-related text in the social media setting is primarily focused on statistical analysis @xcite @xcite . @xcite collected tweets on climate change and global warming in five languages and studied the effect of geography, time, major news events that inspired central topics of discussion over climate change. @xcite presented the tweet authors and topics associated with the publication of the ipcc's ar5 on the physical science basis for climate change based on the tweet's hashtags. moreover, @xcite performed the analysis on tweets during 2012-2013 to conclude that users are establishing a relationship between temperature anomalies and climate change. on the other hand, @xcite used hedonometer to determine how collective sentiment differs in response to climate change-2 https://github.com/serendipity5497/ finegrained-climate-change-social-media related events, news, natural disasters, oil drillings. they conclude that natural disasters and other phenomena related to climate change contributed to a decrease in overall happiness. although the works mentioned above are immensely helpful in understanding climate change-related discourses in social media, recent advances in natural language processing enable the fine-grained detection of climatechange-related social media text. the advent of contextualized word representation for improving natural language representation for various downstream tasks, including text classification, has been particularly significant. recent work in the area employs the techniques of topic modeling @xcite , and lexiconbased sentiment analysis @xcite ). dahal et al. (2019) provided an overview of highimpact areas where machine learning and ai can assist the fight against climate change and highlighted climate mitigation and adaptation, as well as metalevel tools that enable other strategies. loureiro and alló (2020) analyzed twitter conversations related to climate change in uk and spain and employed nlp tools to access the sentiment associated and various emotions evoked by these tweets. they used the lexicon developed by the national research center canada (nrc), denoted as emolex (mohammad and turney, 2013). luo et al. (2020) released the global warming stance detection dataset, specifically focused on identifying stance on global-warming-related sentences from news articles. sobhani et al. (2016) released a twitter dataset for stance detection and further concluded that sentiment features assist in stance classification but are not sufficient on their own. moreover, maynard and bontcheva (2015) release an open-source toolkit for enabling researchers to use twitter to analyze and understand the engagement of the society regarding climate change. ## dataset this section outlines the dataset creation process for both fine-grained classification of climatechange-related tweets and climate change prevention stance detection. first, we detail the data collection process, which entails scraping, filtering, and preparing the text for annotation. secondly, we outline the data annotation schema for the finegrained climate-change-related tweet classification task, along with examples of each of the five categories. we then detail the data annotation schema stance example favor "it's time for the american electorate to make #climate change a political do-or-die, up and down the ticket." #climatecrisis it's not too late, but it's late to start reversing climate change. ## methodology this section briefly describes the various state-ofthe-art models that we used for our benchmarking experiments. ## conclusion in this work, we proposed the task of predicting stance in social media texts related to climate change. we further proposed the task of categorizing these texts into five categories. we benchmarked the datasets using state-of-the-art contextualized word embeddings and provided baselines for both the proposed tasks. we observed that roberta-large outperforms all other models in three of the four evaluation metrics for the finegrained classification task, obtaining an f1 of 0.735. moreover, we also observed that roberta-base obtained the best f1 score in the stance detection task with a 0.510 f1 score. we further extend this work to the semi-supervised setting and use pseudo-labeling to predict for the stance detection and fine-grained classification tasks in a redditbased dataset. this work can be further expanded to analyze people's reactions to climate change in multi-modal and multilingual settings to get a broader understanding.