Automatic Classification of Neutralization Techniques in the Narrative of Climate Change Scepticism (2021)

Abstract: Neutralisation techniques, e.g. denial of responsibility and denial of victim, are used in the narrative of climate change scepticism to justify lack of action or to promote an alternative view. We first draw on social science to introduce the problem to the community of nlp, present the granularity of the coding schema and then collect manual annotations of neutralised techniques in text relating to climate change, and experiment with supervised and semi- supervised BERT-based models.

## introduction there is strong consensus in the scientific community on human-induced climate change @xcite . despite this, action on climate change has become an increasingly partisan issue with strong opposition voices discrediting scientists, and spreading scepticism and misinformation. one such source is climate change counter movement organizations, which are an amalgam of lobbyists, big corporations, conservative think tanks, and media corporations @xcite @xcite , whose aim is to fuel climate change scepticism (ccs). public perception is influenced by the narrative presented to them @xcite , and ccs texts use neutralization techniques to build counter-climate narratives @xcite . the cure can't be worse than the disease/problem is a phrase frequently used by climate change sceptics, 1 and also recently by donald trump in reference to covid-19. 2 though two widely different issues, neutralization is used to justify opposing a policy, lack of action, and thus promotion of either total denial of the problem @xcite or its severity. the ipcc falsely attributes natural warming and urban warming to greenhouse gas (ghg) emission warming. it ignores the compelling evidence of natural climate change before 1950 that correlates well with indicators of solar activity in social science, neutralization is defined as justification/vindication for a deviant behaviour @xcite @xcite . though initially developed in the field of criminology, it has been widely studied in different fields ranging from lack of corporate social responsibility @xcite , to fast fashion @xcite , the tobacco industry @xcite , and ccs @xcite . @xcite argued that to fully understand the neutralization narrative around ccs, there is a need to break it down into specific techniques (e.g. denial of responsibility vs. denial of victim; see section 3). our paper proposes a method to automatically classify these neutralization techniques (henceforth "nt"), as a tool to analyse ccs narrative at scale and help build counter-narratives. our contributions in this work are as follows: (1) we introduce the nt (multilabel) classification task; separately, research on fake news and propaganda has primarily operated at the article level, and focused on binary detection (presence vs. absence) @xcite . @xcite argued for the need for finer granularity in propaganda detection, both in terms of propaganda sub-types and fragment-level detection. in a similar vein, nakamura et al. (2020) proposed fine-grained classes of fake news to differentiate between misleading, manipulated, or totally false content. more recently in the climate change domain, @xcite released a stance-annotated dataset for global warming, and proposed an opinion framing task to study discourse used in the debate around global warming. one challenge in building supervised nlp models is the strong dependency on labelled data. to tackle this, one approach is apply transfer learning from pretrained language models @xcite @xcite @xcite ). another approach is semi-supervised learning. yang et al. (2017) and gururangan et al. (2019) employed variational autoencoders, and clark et al. (2018) leveraged cross-view training using a mixture of labelled and unlabelled data. more recently, pretrained models and semi-supervised learning have been combined with great success, e.g. xie et al. (2020) used bert along with consistency regularization on unlabeled data, croce et al. (2020) extended the fine-tuning process of bert to a generative adversarial setting, and chen et al. (2020) used interpolation to mix up the hidden representations of bert to create augmented data for training. ## neutralization and frames dunlap and brulle (2015), @xcite , and boussalis and coan (2016) categorised ccs arguments into 2 frames: science ("science") and policy ("policy"). science questions the scientific facts, is heavy on denial, or promotes pseudo science, whereas policy deals with issues of cost and economy (e.g. carbon tax), targets the scientists, or passes the blame for action to other nations. mckie ( 2018 ) rebranded the ccs arguments on neutralization by adapting sykes and matza (1957)'s original nt schema, establishing a connection between nt and science/policy frames. we adopt the definitions and coding schema from mckie (2018), as follows (the first four of which relate to the science frame, and the last three to the policy frame, as indicated): ## dataset we construct our neutralisaton techniques dataset from 3 sources: (1) paragraphs extracted from ccs documents @xcite ; (2) ccs sentences/paragraphs from mckie (2018); foot_1 and (3) anti-global warming opinions (sentences) from @xcite . 5 this results in a mixture of sentences and paragraphs, resulting in diversity in the dataset (with longer snippets expected to have more multilabelling). we henceforth call these text snippets "sentences" for brevity. our dataset has a total of 8000 sentences, of which 785 were annotated (and the remainder used as unlabelled data). we formulate the task as a multi-label classification problem where an annotator selects none, or one or more nt labels. to make the task easier for annotators, we split it into 2 nt annotation subtasks based on the two frames: (1) the science frame (deny-responsibility, deny-injury1, deny-injury2, deny-victim, or none); and (2) the policy frame (condemn, loyalties, justify, or none). we combine annotations by taking a majority vote within each frame, and label a sentence as none only if it is the majority-class for both sub-tasks (i.e. none of the nt labels are majority-assigned for either frame). we collect human judgements using amazon mechanical turk with 9 sentences forming a single hit, one of which acts as a quality control in the form of a labelled data instance ## automatic classification we experiment with svm as a baseline and then explore several bert-based supervised and semisupervised models for classification @xcite . as it is a multilabel classification problem, we add a number of one-vs-rest classification layers (one for each class) on top of bert, and update all parameters during fine-tuning. svm: standard linear-kernel svm used in one vs. rest mode, and adapted to a multilabel setting. bert: standard supervised bert fine-tuned using the labelled data. mtext: a semi-supervised bert-based model based on @xcite extended to a multilabel setting. mtext combines the hidden representation of 2 training instances (drawn from both labelled and unlabelled instances) via interpolation to create a large number of augmented data samples. the supervised objective (l s ) uses standard cross-entropy loss whereas the unsupervised objective uses consistency loss (l cl ) in the form of kl-divergence. l cl is computed both on labelled and unlabelled data, where the labels for the unlabelled data are inferred in a self-training manner. to encourage sharp probabilities for unsupervised instances, an entropy minimization loss l em is added, yielding the overall objective l @xmath0, where w x are tunable hyper-parameters. mtext multi : as we see in section 3, nt is associated with science and policy frames. we experiment with adding these frames (including the none class, 3 in total) as an auxiliary objective, creating another supervised loss (l f rame ). 6 the final objective is l nt + αl f rame , where α is a tunable hyper-parameter. ## experiments we split the labelled data into train/dev/test with 450/135/200 sentences. the semi-supervised models (mtext variations) also have access to the unlabelled 7215 sentences. we use the uncased bert-base as the pretrained model for all experiments. we detail the full training details and hyperparameters in supplementary material. we first look at the (fully) supervised results, and see that the baseline bert performs the worst, but adaptive pretraining (bert * ) boosts results. moving on to semi-supervised models (mtext, mtext * , mtext multi and mtext * multi ), we see consistent gains, highlighting the benefits of using unlabelled data. mtext multi with its multitask objective gives a small but appreciable gain over mtext, producing performance that is on par with human performance. interestingly, adaptive to better understand how "data efficient" these models are, we present micro-f1 over varying amounts of labelled training data in figure 1 . we see that mtext and mtext multi outperform bert and bert * substantially with only 30% training data (135 instances) and maintain their the strong performance as data quantity increases. ## conclusion we draw on social science literature in introducing the notion of "neutralisation", in the context of climate change sceptics. we collect annotations of neutralisation techniques in text relating to climate change, and experiment with supervised and semisupervised bert-based models.