Toxicity, Morality, and Speech Act Guided Stance Detection
Authors: Apoorva Upadhyaya
Marco Fisichella
Wolfgang Nejdl
Venue: EMNLP 2023 Findings (2023)
Keywords: stance detection
toxicity
morality. speech act classification
Twitter
TL;DR: We focus on stance detection while exploiting the toxic, moral, and speech act aspects of tweets by proposing a multi-tasking model that extracts the VAD aspects and injects the emotional sense into the embedded text to correctly detect the stance.

Abstract: In this work, we focus on the task of determining the public attitude toward various social issues discussed on social media platforms. Platforms such as Twitter, however, are often used to spread misinformation, fake news through polarizing views. Existing literature suggests that higher levels of toxicity prevalent in Twitter conversations often spread negativity and delay addressing issues. Further, the embedded moral values and speech acts specifying the intention of the tweet correlate with public opinions expressed on various topics. However, previous works, which mainly focus on stance detection, either ignore the speech act, toxic, and moral features of these tweets that can collectively help capture public opinion or lack an efficient architecture that can detect the attitudes across targets. Therefore, in our work, we focus on the main task of stance detection by exploiting the toxicity, morality, and speech act as auxiliary tasks. We propose a multitasking model TWISTED that initially extracts the valence, arousal, and dominance aspects hidden in the tweets and injects the emotional sense into the embedded text followed by an efficient attention framework to correctly detect the tweet's stance by using the shared features of toxicity, morality, and speech acts present in the tweet. Extensive experiments conducted on 4 benchmark stance detection datasets (SemEval-2016, P-Stance, COVID19-Stance, and ClimateChange) comprising different domains demonstrate the effectiveness and generalizability of our approach.

Toxicity, Morality, and Speech Act Guided Stance Detection
Apoorva Upadhyaya, Marco Fisichella, Wolfgang Nejdl L3S Research Center, Leibniz Universität Hannover, Germany (upadhyaya, mfisichella, nejdl)e13s.
Abstract
Weber, 2017: Li al., 2021a). A recent Ameri- can Press article' claims that misinformation about climate is widespread on Twitter and often delays appropriate climate action. addition, a study con- ducted by MIT researchers" showed that false news spread 6 times faster on Twitter than truthful news. Therefore, it is necessary to recognize the attitudes of the posts so to curb the spread of misinforma- tion. In our work, we perform stance detection of tweets to detect the opinion of the post, whether it in favor, against, or none toward a target topic. There a wealth of research on identifying atti- tudes in social media. Several popular and bench- mark stance datasets exist in the literature such as 2001 (Mohammad et al., 2016), P-Stance (Li et al., 2021b), COVID-19 19 (Glandt et al., 2021), and ClimateChange (Upadhyaya et al., 2023b) that focus on various social issues such as feminism. abortion, COVID19, climate, and many current crises whose public opinions need to be identified in order to address these issues and change soci- cty for the better. We also use these 4 benchmark datasets in our study, which covers different do- mains, to get assessment of generalizability of our approach.
In this work, we focus on the task of determin- ing the public attitude toward various social is- sues discussed on social media platforms. Plat- forms such as Twitter, however, are often used spread misinformation. fake news through polarizing views. Existing literature suggests that higher levels of toxicity prevalent in Twit- ter conversations often spread negativity and delay addressing issues. Further, the embed- ded moral values and speech acts specifying the intention of the tweet correlate with public opinions expressed on various topics. How- ever, previous works, which mainly focus on stance detection, either ignore the speech act, toxic, and moral features these tweets that can collectively help capture public opinion lack efficient architecture that can de- tect the attitudes across targets. Therefore, in work, focus on the main task of stance detection by exploiting the toxicity, morality, and speech act as auxiliary tasks. We propose multitasking model TWISTED that initially extracts the valence, arousal, and dominance as- pects hidden in the tweets and injects the emo- tional sense into the embedded text followed by an efficient attention framework to correctly detect the tweet's stanee by using the shared features of toxicity, morality, and speech acts present the tweet. Extensive experiments conducted on 4 benchmark stance detection datasets (SemEval-2016, P-Stance, COVID19. Stance, and ClimateChange) comprising differ- ent domains demonstrate the effectiveness and generalizability of our approach.
Existing literature suggests that the tweet con- tent consists of abusive and insulting statements, leading to the spread of hatred, bullying, and neg- ativity in the public sphere, often disturbing so- cial peace (Hosseinmardi et al., 2015; Cheng et al., 2017; Pavlopoulos et al., 2019). Moreover, the moral values in the tweets correlate with the differ- ent public opinions, which helps understand the intentions behind each expressed attitude, which in turn helps to culturally integrate society and mitigate such differences (Rezapour al., 2021). In the popular benchmark datasets, we also note
1 Introduction
Social media platforms are often used to express public opinions and raise awareness about vari- ous issues and current problems in socicty (Line- man et al., 2015; Kaur and Chahal, 2018; Li et al., 2021a). However, conversations on platforms like Twitter can lead to users being exposed to false, misinformed messages, often resulting in polar- izing beliefs and echo chambers (Garimella and
412222244b44700666627777222222
travel--iister--me-sttiriie

---

that the speech act of the tweets, i.e., the form of expression, statement, question, or thought, also differs significantly across viewpoints on differ- ent topics (refer Appendix B.2), which helps to determine the communicative intent of the view- point, further supporting the stance task. The fol- lowing are few examples of tweets from public datasets: (ii)Feminssm (against): HaddenTara No one wants to see Feminists naked, so the petty, vindictive bit**** want to drag everyone else down to their level Biden (favor): "Lets Go Joe #TeamJoe #Go- Joe #Biden2020 inon_toxic, care, loyalty, sugges- tion)". These relationships between toxic, moral, and speech act labels associated with the tweets' stances led us to use them to decipher their under- lying attitudes.
first cross-sectional study incorporate toxicity, morality, and speech act determine the tweet's stance. This opens a new dimension in psycho- logical and social science research to investigate in-depth the interdependencies between these toxic, moral, speech acts and public opinion. (ii.) We propose multi-tasking system TWISTED that performs the main task of stance detection (SD) by using toxicity detection (TD), morality classi- fication (MC), and speech act classification (AC) as auxiliary tasks. Our TWISTED model first ex- tracts the VAD features to capture the associated emotions and intentions of the tweet. EmoSense- Injector then induces the emotional sense to the tweet by integrating the embedded text with the VAD vectors. The emotionally enriched tweet is followed by an efficient attention framework that uses the task-specific and shared toxic, moral, and act features to identify the attitude of the tweet (favor/againss/none), (iii.) Extensive ex- periments are conducted on benchmark stance detection datasets /SemExal-2016, P-Stance, Cli- mateChange, COVID-19 110--119--ttne)) and the reported experimental results demonstrate the usefulness of our approach. The code and datasets with annota- tions are available here?.
While several works (Vychegzhanin and Kotel- nikov, 2021: Wang al., 2020; Li and Caragea, 2019) focused on the linguistic properties of the input text for stance, they often lack an advanced architecture in the form of a better encoder trained exclusively on tweets, thus failing to extract the masked toxic and moral aspects from tweets in a multi-tasking form. Other recent works such as MEMOCLIC (Upadhyaya et al., 2023b), SP-AMT (Upadhyaya et al., 2023c) have used sentiment, emotion, and toxicity aspects for stance detection, but these models focus primarily on climate change and often suffer from the drawbacks of detecting overall stance in the case of composite tweets with contrasting emotions or complexity of tweet con- tent due to sarcasm. It has been proven that the valence, arousal, and dominance features help in analyzing emotional tone and its intensity (Os- good al., 1957; Russell, 1980, 2003; Mohammad, 2022), which eventually supports the detection of hidden intent or any sarcasm present in the tweet (Vitman et al., 2022). Therefore, our approach leverages these interdependencies to extract addi- tional features of valence, arousal, and dominance (VAD) along with insult, morality, and speech act understand the psychology of the tweet and gain emotional insights. These features can then ulti- mately help identify the collective meaning and intent of the tweet, aiding in the detection of at- titudes and thus addressing the drawbacks of the presence of sarcasm and implicit stance from pre- vious works.
2 Related Works
Stance detection has been investigated in number of studies. Some of the existing works focus on exploring the impact and influence of polarizing attitudes (Chitra and Musco, 2020), while others use the characteristics of tweet texts networks classify attitudes towards a target domain by devel- oping machine and deep learning algorithms (Dutta et al., 2022; Upadhyaya et al., 2023d). Since it is critical to understand public attitudes toward any pressing issue, our study also focuses on detecting the stance of tweet text toward an urgent social issue. This can help technology companies and government agencies monitor public opinions and intervene curb the spread of fake news, misinfor- mation, or online hatred that can harm the peace.
Previous works have enriched the stance task with popular datasets covering controversial top- ics such as feminism, abortion, political leaders, COVID-19, and the climate crisis (Mohammad et al., 2016; Glandt et al., 2021). Various exist- ing studies have primarily focused on the SemEval
The main contributions of our work are as fol- lows: (i.) To the best of our knowledge, this is the
htttps://osf http://///...//eeeeed//e////////

---

2016 dataset (Vychegzhanin and Kotelnikov, 2021; Wang and Wang, 2021) while others have focused on political and COVID domains (Zheng et al., 2022; He et al., 2022). The climate change do- main, which focuses on the climate crisis, has also been recently explored for stance task (Upadhyaya et al.. 2023a.b). However, in our study, we use these benchmark datasets and investigate the per- formance of stance task that span a variety of con- trasting domains with different embedding spaces of toxic, moral, and act features to understand the practicality of our approach.
Input Tweet Text
BERTweet
VAD Embedder
Encoder
Ft
Fy
EmoSense- (Ft; Fv; Fv; Ft. Fv)
Injecting Emotional- Sense
Eeit
Emotionally Intelligent Tweet
MH
MuliHHeadAttention
Recently, research has explored multitasking and federated learning for various classification tasks such as hate speech, toxicity detection, and infrastructure damage identification, as the MTL paradigm aims to improve generalization perfor- mance by learning multiple related tasks at once (Priya et al., 2020; Vaidya et al., 2020; Mishra et al., 2021; Badar et al., 2023; However, as far as we know, this is the first cross- sectional study to utilize toxicity, morality, and speech act detection tasks in multitask learning framework to identify the attitude of a tweet.
Densesd
Densend
Densemc
Denseac
Osd
Otd
O mc
Oac
Stance
Toxicity (auxiliary)
Morality (auxiliary)
Act (auxiliary)
(primary)
Younis et al., 2023).
Figure 1: Architectural overview of our proposed
TWISTED approach
In addition, the existing literature pointing to the correlation between toxic and moral aspects in relation to public opinions expressed on Twitter motivated us use these features as auxiliary tasks (Rezapour et al., 2021; Upadhyaya et al., 2023b). Different forms of speech act further help to de- code the pragmatics of the author of the tweet and to identify the attitude encoded in the tweet (Saha et al., 2021). Models such as ESD (Vychegzhanin and Kotelnikov, 2021) and HAN (Wang et al., 2020) exploit the linguistic characteristics of tweets by incorporating sentiments, stylistic, and stance- indicative features. However, these earlier studies did not use the better embedding techniques by using the BERTweet encoder along with extract- ing VAD features that provide the emotional tone and intensity that better identify different attitudes even when similar emotions or feelings are present. Recent climate stance models use emotion and toxi- city to identify stance (Upadhyaya et al., 2023b) but suffer from problems with the presence contrast- ing emotions or sareasm in the stances. However, our model addresses these problems by extracting the VAD features along with the toxic and moral aspects, which together understand the overall in- tent of expressed public opinion and aid the stance task.
Methodology
Problem Definition Given a tweet, we propose a stance detection approach that combines the tweet text with the extracted valence, arousal, and dom- inance features while taking advantage of toxic, moral, and speech act aspects associated with the tweet efficiently detect the stance toward a target (favor/against/none)..
Our proposed framework TWISTED comprises the model components: SERRTweet-Encodeee VAD- Embedder, EmoSense-Injector, MuliiHeadAAten- tion, and Output Layer. Figure I represents the overview of our TWISTED approach. The tweet is passed in parallel the BERTweet Encoder generate the embedding of the tweet text, and to the VAD Embedder to extract the valence, arousal, and dominance features related to the tweet. The en- coded text and the VAD features are passed through the EmoSense-Injector which is responsible for in- tegrating the emotional sense in the tweet and ob- taining the emotionally intelligent representation of the input tweet. This emotion aware tweet is then followed by the MultiHHeaAAttention layer focus on the most relevant input features, followed by the task-specific dense and output layers to finally obtain the outputs of the main task of SD and the secondary tasks of TD, MC, and AC.

---

3.1 BERTweet Encoder
text representation.
The input tweet text is converted into sequence of vectors using the BERTweet encoder (Nguyen al., 2020), which is specially trained on tweets embed each word in the tweet text. First, the tweets are tokenized and divided into sequences of tokens of the form [CLS] t1,12..tn, where [CLS] is a special token that marks the beginning of a tweet. In each minibatch, the mput tokens are padded to a maximum sequence length (say m). The final state corresponding to the first token [CLS] is used as the overall representation of tweet. BERTweet Encoder generates embeddings of size 768 dimen- sions (d,) for each input token, which is then flat- tened, resulting in F. E pm(ds)
Eeit [F;; Fo;
Fo: F
(1)
3.4 MultiHead
We use the MultiHead-Attention similar to (Vaswani et al., 2017) based on the concept of query (Q), key (K), and value (V) to jointly capture the most relevant parts of the input feature from different representation subspaces. Instead of performing single attention layer (refer Equation 3). Q. K. and are linearly projected times (where de- notes the number heads) and Attention function (Equation 3) is performed in these different learned projections of Q, K, and V in parallel. These out- puts of parallel attention functions are concate- nated and once again projected, resulting in the final attention vector. To perform MHA, we em- ploy Mawteyer..MullHHaaAAAtttttttt
3.2 VAD Embedder
We employ NRC-VAD Lexicon (Mohammad, 2018) to acquire the valence, arousal, and dom- inance (VAD) features of the input tweet. The preprocessed input text (the preprocessing steps are described in the Appendix B. consists of to- kens generated by the NLTK TweerTokenizer and then passed to the NRC-VAD lexicon. The lexicon consists of 20,000 English words, with each word having probabilistic score for all three aspects of valence, arousal, and dominance. The VAD scores of each token in the input text are then multiplied by the occurrence of that token for all 3 dimensions (V, A, and D), summed, and aggregated to create sin- gle vector of dimension (dy) for each input tweet. The final feature representing the VAD embedding space for each input tweet is thus Fg e Rdo
layer', where the output of EmoSense- -njjecto (Eest) fed as query, key, and value to the MultHccadAAtention layer TensorFlow.
Q = Eeits K = Ecits V Eeit
(2)
Attention = softmar(QK*
(3)
mha Madt
h)
(4)
MH Aop mha([Q, K, VI)
(5)
Equations 4 and 5 represent the MultHHeadAAtenn tion TensorFlow layer being implemented where h denotes the number of heads indicating the number of times the single Attention function (shown in Equation 3) is performed, h, specify the size of the head, and Eest is used as query, key, and value, resulting in M MHAL Aop E R. Rxx4(d))
3.3 EmoSense--njector
The component is accountable for incorporating the emotional sense and intensity extracted from the VAD features into the embedded tweet text to obtain an overall representation of the input tweet in accordance with the associated emotions. The text (Fr) and VAD (F,) feature vectors are initially passed through a fully connected layer of dimen- sion de. The resulting text and VAD vectors are then fused using absolute difference and element- wise product (refer Equation 1), as the fusion tech- nique has been proven efficient in various classifi- cation studies (Mou al., 2015). The integrated output is then reshaped (Eea E Rlx4(d.)) and fi- nally represents the emotionally intelligent tweet that captures the essence of the emotions present along with the semantic and syntactic contextual
3.5 Output Layer
The shared output of MultiHcalAAtention (M (M component passed through the task- specific dense layers of dimension [d4] (Densead Denseta, Denseme- Denscae) followed by four softmax layers for capturing the final output specific to each task of SD (Osd). TD (Otz), MC (Omc), and AC (Oac) separately. The integrated loss function (L) of our TWISTED framework is realized in equation 6 indicating loss for each specific task of SD (Tss), TD (Iua), MC (Tmo), and AC (Loc):
L = p* Lod q * Ltd + r Lnn s Lag (6)
'https://ww/ tensor wwwpsooflow...o//adeee www..odo//ddds/////// wwwaaadddnsssppaaeeeeeeeee hacss/app_____a api_docss/

---

where p. q, r, and are the values between 0 and 1 and show the ratio of loss of each task total loss.
score provides the scores for the 10 moral classes according to the sociological theory of Moral Foundations Theory (MFT), which iden- tifies morality as a set of vice virtues, including care-harm, faimess-cheating, loyalty-hetrayal, au- thority-subversion and purity-degradation (Gra- ham et al., 2013). The tweet text is fed into the cMFDScore library (with DICT TYPE ='emdd', PROB MAP 'all', SCORE_ METH_DDTHOD 'bow', OUT METRICS 'vice-virtue" as parameter val- ues), which first preprocesses the text and assigns a list of 10 foundation scores indicating the degrec to which each tweet reflects the moral values. To facilitate the performance of morality as multi- label classification task, in our study, after manual review and careful analysis, we convert the list of probabilistic scores into list of Os and Is indi- cating the presence/absence of the corresponding moral attribute. To achieve this, we first compute the mean (mean) and standard deviation (a) of the list of 10 moral scores and consider threshold as (mean + r). If the value of an attribute is > threshold, we marked the presence of the corre- sponding attribute as "1", otherwise as "0". The final label of each tweet for the MC task results in the list of 10 elements being 0 or (for exam- ple: MC [,00==[1.0010000.000..
4
Experimental Set-up
4.1 Datasets
We use the benchmark datasets of stance detec- tion containing different targets: (i) SemEval-2016 (Mohammad et al., 2016): is a popular dataset used in 2e0 shared task 6.A that covers Atheism, Climate Change is Real Concern, Fem- inist Movement, Hillary Clinton, and Abortion as targets with tweets having favor, against, neu- tral stances. (ii) P-Stance (Li et al,, 2021b): is a political stance dataset containing 7, 953 anno- tated tweets for "Donald Trump". 7, 296 for "Joe Biden" and 6,325 for "Bernie Sanders" as target domains with favor and against stances. (ii) Climat- eChange (Upadhyaya et al., 2023b): In this bench- mark dataset, 8, 881 climate tweets are included that have believe, deny, and ambiguous views re- garding climate change. (iv) COVID-19-Stance (Glandt et al., 2021) is dataset of pandemic- related 6, 133 tweets on four controversial topics ("Anthony S. Fauci, M.D (Director of the National Institute of Allergy and Infectious Diseases)," "stay at home orders" "wearing a face mask," and "keep- ing schools closed") and consists of stances ("FA- VOR", "AGAINST", "NONE") on these topics.
suggesting the presence of the moral values care, fairness, and purity/sanctity).
Speech Act Classification (SC) We use the pre- trained benchmark speech act classifier trained on the gold-standard speech act tweet corpus (Saha et al., 2021), which contains 6749 tweets, pro- vide weak speech act labels for the instances of the stance datasets used our study. Since the pre-trained classifier performs better on the 5-class classification of speech acts, we also extract the speech act labels for the tweets of the 4 stance datasets in the form of expression, question, state- ment, suggestion, and others.
4.1.1 Data Annotation
All publicly available datasets consist of annota- tions of stances, however, to find the toxic, moral, and speech act labels, we follow the below weak- supervised approaches.
Toxicity Detection (TD) Similar to previous works (Hickey et al., 2023; Upadhyaya et al., 2023b), the Perspective APT developed by Jigsaw and Google's Counter Abuse Technology team in Conversation- AI (Hosseini et al., 2017) is used to provide labels for the TD task. We use the probability value (0-1) for the "toxicity" metric returned by the API. After careful analysis of various thresholds and in-line with existing literature (Upadhyaya et al., 2023b), we decided that if the value of the "toxicity" at- tribute is 2 0.5, we consider the tweet to be toxic so as not to miss any toxic content, otherwise, the tweet will be labeled as non_loxic.
The data pre-processing, % distribution of auxil- iary features, and manual verification of labels generated by weakly supervised approaches are mentioned in Appendix B.
4.2 Implementation Details
Hyperparameters: maximum sequence length (m)) 128; BERTweet embedding dimension (d,): 768; VAD feature dimension (d,)): 3; EmoSense-
Morality Classification (MC) We leverage the ex- tended version of the Moral Foundation Dictionary (cMFD) (Hopp et al., 2021) for MC task anno- tation. The open-source Python library "eMFD-
httpp:: thub. cww/cod////////eeee aneuroscience/ emfdscore

---

Injector dense layer dimension (de) [with ReLu activation]: 128; MultHeeaAAttention parameters: 8 [number of heads (h)), 64 [head_ size (ha)J; task- specific dense layer dimension (d,)) [with ReLu activation]: 128, output neurons/channels for aux- iliary tasks: 2 [softmax activation] (TD), 10 [sig- moid activation] (MC), and 5 [softmax activation] (AC), loss function for auxiliary; categorical cross- entropy for TD (Iea) and AC (Tac); binary cross- entropy for MC (Lme), output neurons for main SD task: P-Stance dataset= 2 output neurons and binary cross-entropy for SD (presence of "favor" and "against" stances); SemEval, ChanateChange, COVID datasels= 3 output channels and categor- ical cross-entropy loss for SD (Issd), Optimizer: Adam (0.0001 learning rate), batch_size size: 32. The best parameter values are selected using TPE in the Python library Hyperopt which minimizes loss functions. The loss weights for all tasks are fine-tuned using Grid Search from Scikit-learn (SD (p)=1, TD (q)=0.5, MC (r)=0.3, and AC (s)=0.4). Evaluation Metrics: We use already available train, validation, and test splits of the SemEval- 2016, P-Stance, and COVID- 19-Stace datasets, perform 5 independent runs of our framework to ac- count for variability, and report the macro-average of the Fl-score as the evaluation metric. For SemEVAL-2016 (does not consider None stance label for performance evaluation) and P-Stance (consists of 2 stance labels), Fang = (Fjavor + F Fagainst) ) /2, similar to their proposed works (Mo- hammad et al., 201 6; Li et al., 2021b) and Fang = (F fovo Fagainst + Fnome) )/3 for COVID-19- Stance dataset (Glandt et al., 2021; He et al., 2022). In-line with existing work that proposed Climate- Change data (Upadhyaya al., 2023b), we imple- ment our TWISTED approach with 5-fold cross- validation technique and report the average and standard deviation of macro-variant of precision, recall, F1-score as the performance metrics. Environment Details: described in Appendix C.
Model
Athelsn
Face
Fins
Climate
Firs
Feminism
Fers
Hillary
Abortion
Farg
Mia
Fexs
TWISTED
(Ours)
79.57
78.74
76,09
78.29
76.67
77.87
MEMOCTG
75.44
71.36
83.00
70.00
63.00
67.00
7000
70.5
MTFURM
76.14
53.05
63.12
74,67
70.32
67.46
SPATT
63.2
66.34
7500
1574
NNDMM
5239
63.78
67.50
67.19
64.03
ESD
06.64
13.82
02.85
61.79
61.91
61,20
HAN
70.33
49.35
37.50
61.23
10.10
61.00
AEOSS LEX
69.22
59.18
61.49
68.33
68.41
65.33
SVX-
65.19
12.35
57.46
58.63
66.12
58.01
ngram
Table I: Results Stance task on SemEval-2016 with Baselines [ denotes the 2nd-best performer]
(Fu et al., 2022), MDMT (Wang and Wang, 2021), ESD (Vychegzhanin and Kotelnikov, 2021), HAN (Wang al., 2020), AT-JSS-LEX(LI and Caragea, 2019), and SVM-ngram (Sobhani et al., 2016).
P-Stance We take the baseline and best-performing models reported in recent works (Liu al., 2023; Zheng et al., 2022; He et al., 2022): In-Domain In- target variant RoBERTa-base (Liu et al., 2023) (we re-run the model with respect to "Trump", "Biden", and "Bernie" separately as results reported in the paper on the complete dataset), WS-BERT-Dual (He et al., 2022), BERTweet (Nguyen et al., 2020), BERT (Devlin et al., 2019), PGNN (Huang and Carley, 2018), TAN (Du et al., 2017), and BiCE (Augenstein et al., 2016).
ClimateChange We use the baselines as is from the work that proposed data (Upadhyaya et al., 2023b) and report average of macro-variant of precision, recall, and F1-score: SP-AMT (Upadhyaya et al., 2023c), ROBERTa Base (Vaid et al., 2022), MT- LRM BERT (Fu al., 2022), S-MDMT (Wang and Wang. 2021), ESD (Vychegzhanin and Kotelnikov, 2021), HAN (Wang et al., 2020), MNB (Kabaghe and Qin, 2020), and DNN (Chen et al., 2019).
COVID-19-Stance Below are best-performing and baseline models mentioned in recent literature (Upadhyaya et al., 2023d; Zheng et al., 2022; He et al., 2022): STASY (Upadhyaya et al., 2023d), WS-BERT-Duse (He et al., 2022), CT-BERT-DAN (Glandt et al., 2021), GCAE (Xue and Li, 2018), ATGRU (Zhou et al., 2017), and TAN (Du et al., 2017).
4.3 Baselines
All experimental results of the baselines used on all the datasets are retrieved from the original papers. 2em We compare TWISTED with the recent works that use this dataset: Semi- Supervised(Model3) (Reveilhac and Schneider, 2023), MEMOCLiC (Upadhyaya et al., 2023b), SP- AMT (Upadhyaya et al., 2023c), MT-LRM-BERT
5 Results
Please note that we report the results of the main task SD aim improve the performance of the primary task by using other auxiliary tasks in our current study.
2val-2060 Table shows that our TWISTED

---

Model
Trump
Biden
Sanders
Avg.
Model
Precision
Recall
F1 score
TWISTED
87.3
89.7
84.6
87.2
Avg/Stt.de
Avg/St.dev
Avg/St.dev
RoBERTa- Base
82.6
81.3
78.1
80.7
TWISTED
WS- BERTT-Dam
85.8
K3,5
79.0
82.8
(Ours)
95.19/0.46
96.11/0.23
96.05/0.32
BeRTwect
85.2
82.5
78.5
82.1
MEMOCLiC
92.06/0.81
95,44/0.29
93.76/0.62
BERT
78.3
78.7
72.5
76.5
RoBERTa-Baes
83.38/1.55
85.24/1.28
84.69/1.89
PGONN
76.9
76.6
72.1
75.2
SP-AMT
87.95/1.11
90.01/1.8
89.2 89.29/1.31
TAN
77.1
77.6
71.6
75.1
MT-LRM-BERT
87.12/1.61
88.70/0.99
88,59/1.29
BICE
77.2
77.7
71.2
75.4
S-MDMT
86.12/1.02
88.67/0.39
86,91/0.44
ESD
81,55/1.72
84.39/2.05
83,28/2.3
Table Results for Stance Detection on P-Stance
HAN
84.61/1.22
84.23/1.78
84.54/1.65
MNB
170.66
79.5 17.57
78.43/1.33
DNN
77.64/1.58
76.38/1.6 .08
77.15/1.18
model outperforms the other baselines with over- all Mac-Favg of 77.87, resulting in an average Fl score performance improvement of 3.22% to 34.24%. This demonstrates the importance of us- ing features such as toxicity, morality, and speech act in the context of the SD task. In addition, we find that TWISTED performs better, especially in the feminism, Hillary, and abortion domains, be- cause of a clearer separation between favor, against, and none stances in terms of their hidden offen- sive and moral aspects (see Table 6 in Appendix). However, our approach does not beat the Model3 baseline on "atheism" domain and performs second best, although the tasks TD and AC contributed well due to the clear distinction in their embed- ding spaces with respect to the different attitudes, but the moral values between the 3 stance classes are much more closely aligned, making it difficult for TWISTED to distinguish between all stance classes (refer Appendix Table 6). WKNN achieves slightly better result than TWISTED (78.74) with 79.95 Fang on "climate". This is due the fact that the dataset contains only 29 tweets from deniers, which makes it difficult to capture different speech act and moral labels such as betrayal, subversion, purity, and question (Appendix Table 6) due to the very low availability of token words, however, the significant differences in VAD features (sce VAD features in the dataset²) between stance la- bels and the clearer separation between toxic and non-toxic content embedded using the BERTweet model trained exclusively on tweets contribute to TWISTED being the second best performer among baselines on "climate" target domain.
Table 3: Results for Stance Detection on ClinateChange
diff.) domains show better-dividing line in terms of toxic characteristics than Trump (17. 23 diff.) (re-
fer Appendix Table 7). Nevertheless, TWISTED, with an average macro-Fi score of 87.2 and an overall average improvement of 11.61%, %, outper- forms the baselines, justifying the usefulness of our multitasking approach in inducing emotional meaning in tweets using VAD feature extraction and efficient attention framework in the political domain.
ClimateChange Table indicates that TWISTED performs better than the other baselines with an average percentage improvement of 1..1% in the Fl score. Although MEMOCLiC extracts the emo- tional and toxic features in multitasking environ- ment to identify the underlying attitude of the tweet, but the performance power of our EmoSense injec- tor, which injects the VAD features, understanding not only the emotional quotient of the tweet, but also the intensity and tone along with the hidden moral and linguistic features, is one of the main reasons contributing to the success of TWISTED compared to the recent works MEMOCLiC, SP- AMT. We note that deniers' tweets have a high pro- portion of bad moral values (harm, betrayal, and humiliation) and toxic content (Appendix Table 8), demonstrating the usefulness of our approach for government agencies or technology companies that need to monitor such content before it leads to a delay in appropriate climate action. Models such as MEMOCLIC, SP-AMT, and MT-LRM-BERT, which take advantage of toxicity, emotion, senti- ment, or opinion formation tasks, perform better than other baselines, proving the effectiveness of these auxiliary tasks for the SD.
P-Stance From Table 2, our TWISTED model per- forms better than Trump in Sanders and Biden, as shown by the 1.7%, 7.1%, and 7.1% improvement in Fl results, respectively, compared to the second- best performing baseline (WS-BERT-Dual) This is mainly due to the fact that supporters and op- ponents in Sanders (28.98 diff.) and Biden (26.14
COVID-19-Stance Table proves the efficacy of our TWISTED model in detecting attitudes that occurred during the controversial COVID topics

---

Model
Fauci
Home
Schools
Avg.
100
TWISTED
91.05
91.63
85,29
89.78
89,44
SD
STASY
88.04
89.19
86.15
85.71
87.27
95
SD+MC
WS-BERT-Dual
83.6
85
86.6
82.2
84.4
SD+AC
CT-BERT-DAN
83.2
78.7
82.5
71.7
79
90
SD+TD+MC
GCAE
64
64.5
63.3
49
60
F1-score
SD+TD+AC
ATGRU
61.2
52.1
59.9
52.7
56.5
85
SD+TD-MC+AC
SD-MC+AC
TAN
54.7
53.6
54.6
53.4
54.1
Table 4: Results for Stance Detection on COVID-19
75
70
"Fauci", "Home", and "Schools" with the excep- tion of the topic "wearing face mask (Mask)". We note that WS- WSS-BERT-DUAL performs better on the "MASK "topic than TWISTED, the data distri- bution of the aspects of "toxic", "non-toxic", and "expression speech act" are close among sup- porters and opponents, and the percentage vices in the form of "harm", "cheating," " and "degrada- tion" are similar among the against and none stance labels (see Appendix Table 9), which prevents the TWISTED approach from clearly distinguishing the polarized labels for the SD task. It can also be seen from Table 4 that the STASY model, which uses the sentiments and temporal aspects present in the tweets belonging to different public opinions, gives better results than other baselines, proving the need to identify hidden aspects in the tweets that can be used for the task SD. However, the average macro F1 score of 89.44 achieved by TWISTED suggests that our approach is also well suited for the important social issues in the domain COVID.
65
SemEval PStance Climate COVID Datasets
Figure Ablation Study: Performance datasets with different combinations of auxiliary tasks for the main task of stance detection.
ther evident from the figure, toxic and non-toxic features associated with tweet text help the attitude task more effectively than the other tasks MC and AC due to their significant differences in distribu- tion among "favor", "against", and "none" stances (see Appendix Tables 6, 7, 8, and 9). This also justifies the assignment of higher loss weight TD task than MC and AC tasks. Moreover, the 3-task combination SD+TD+AC contributes more than the tasks SD+TD+MC and SD+MC+AC, fur- ther proving that expression, statement, query, and other speech acts are more consistent with the toxic aspects of tweets. However, the performance im- provement of TWISTED on all 4 data sets when all secondary tasks are used (SD+TD+MC+AC) confirms the importance of all 3 tasks for main SD.
Compared with the baselines, the results of TWISTED were statistically significant (under t-tests (p <0.05). The better performance of TWISTED in various target areas such as femi- nism, abortion, politics, climate, and COVID and comparable performance in other subject domains proves the usefulness of our approach and the gen- eralizability of our system.
The case studies and possible error scenarios of our framework are described Appendix A and Table 5. Furthermore, the ablation study show- ing the importance of our proposed components for the ClimateChange and COVID-19 datasets is mentioned in the Appendix D.
Ablation Study Here we examine the effects of single-task, all the auxiliary tasks and their com- binations for the stance task as part of the abla- tion study. Figure 2 shows the macro-average Fl score of all 4 datasets when performing the main task SD with different combinations of auxiliary tasks such as SD (Single-task), SD+TD, SD+MC, SD+AC. SD+TD+MC, SD+TC+AC, SD+AC+MC, SD+TD+MC+AC (all). Figure clearly shows that the single-task variant of our TWISTED ap- proach is the worst performer when compared with other auxiliary task combinations, suggesting the potency of using auxiliary tasks for SD. It is fur-
Conclusion
In this work, we present the first cross-sectional study focusing on the problem of stance detection using the associated toxic, moral, and speech act traits present in tweets. Our proposed approach is enriched with the emotional quotient of the tweet by extracting the VAD features and further integrat- ing them into the embedded tweet, thus evoking the emotional intent of the tweet. The experimental results on benchmark datasets indicate that multi- tasking enhances the performance the stance

---

task by exploiting the efficient attention and en- coder frameworks along with the auxiliary tasks. This demonstrates that the model is widely appli- cable in multiple domains, proving the generaliz- ability of our proposed approach. In the future, we plan to include additional tasks such as entity- recognition and aspect-based sentiment detection, which could help disambiguate the attitudes and conflicting emotions in tweet predict more accurate classification of polarized attitudes toward a target domain.
with bidirectional conditional encoding. Proceed- ings of the 2016 Conference on Empirical Methods in Natural Language Processing, EMNLP 2016, Austin, Texas, USA, November 1-4, 2016, pages 876-885. The Association for Computational Linguistics.
Maryam Badar, Wolfgang Nejdl, and Marco Fisichella. 2023. Fac-fed: Federated adaptation for fairness and concept drift aware stream classification. Machine Learning, pages 1-26.
Xingyu Chen, Lei Zou, and Bo Zhao. 2019. Detect- ing climate change deniers on twitter using deep neural network. In Proceedings of the 2019 I 11th International Conference on Machine Learning and Computing, pages 204-210.
Limitations
The error scenarios listed in Appendix A.1 (Table 5), which arise from the proximity of auxiliary fea- tures between stance categories and the ambiguity in stances, primarily describe the limitations of our approach. However, we plan to focus on the catego- rization of tweets, entity detection for identifying targets, and extraction of causes behind the stances in our future work to further improve the perfor- mance of the classification task. Moreover, in our current work, we do not investigate the SD task with the advancement of large language models such as ChatGPT. However, it would be interesting to investigate the performance of ChatGPT with ffain-of-thoughhe (CoT) prompting and find out there are inherent biases in LLMs toward partic- ular target domain for the task SD, which would lead us in future research direction.
Justin Cheng, Michael Bernstein, Cristian Danescu- Niculescu-Mizil, and Jure Leskovec. 2017. Anyone can become troll: Causes trolling behavior online discussions. In Proceedings of the 2017 ACM conference on computer supported cooperative work and social computing, pages 1217-1230.
Uthsav Chitra and Christopher Musco. 2020. Analyzing the impact of filter bubbles on social network polar- ization. Proceedings of the 13th International Conference on Web Search and Data Mining, pages 115 123.
Jacob Devlin, Ming-Wee Wei Chang, Kenton Lee, and Kristina Toutamova. 2019. BERT: pre-training of deep bidirectional transformers for language under- standing. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguisties: Human Language Tech- nologies, NAACL- 2019, Minneapolis, MN. USA, June 2-7, 2019, Volume (Long and Short Papers). pages 4171-4186.
Ethics Statement
Jiachen Du, Ruifeng Xu, Yulan He, and Lin Gui. 2017. Stance classification with target-specific neural atten- tion networks. International Joint Conferences on Artificial Intelligence.
Our study uses publicly available datasets and we only extend the datasets with the toxicity, morality, and speech act annotations. We obeyed the restric- tions on data use and did not violate any copyright issues. Since the datasets are online-generated con- tent, we only share the tweet IDs and annotations to comply with the terms of use and protect individual privacy.
Subhabrata Dutta, Samiya Caur, Soumen Chakrabarti, and Tanmoy Chakraborty. 2022. semi-supervised stance detection of tweets via distant network super- vision. In Proceedings of the fifteenth ACM inter- national conference on web search and data mining. pages 241-251.
Acknowledgments
Yujie Fu, Xiaoli Li, Yang Li, Suge Wang, Deyu Li, Jian Liao, and Jianxing Zheng. 2022. Incorporate opinion-towards for stance detection. Knowledge- Based Systems, 246: 108657.
This work was partly funded by the SoMeCliCS project under the Volkswagen Stiftung and Nieder- sächsisches Ministerium für Wissenschaft und Kul- tur.
Venkata Rama Kiran Garimella and Ingmar Weber. 2017. A long-term analysis of polarization twitter. In Proceedings of the International AAAI Conference on Web and Social Media, volume 11.
References
Kyle Glandt, Sarthak Khanal, Yingjie Li, Doina Caragea, and Cornelia Caragea. 2021. Stance de- tection in covid-19 tweets. In Proceedings of the
Isabelle Augenstein, Tim Rocktäschel, Andreas Vla- chos, and Kalina Bontcheva. 2016. Stance detection

---

59th Annual Meeting of the Association for Compu- tational Linguistics and the 11th International Joint Conference on Natural Language Processing (Vol- ume I: Long Papers), pages 1596-1611.
Manyu Li, Nadia Turki, Cassandra R Izaguirre, Chloe DeMahy, Brooklyn Labery Thibodeaux, and Tay- lor Gage. 2021a. Twitter as a tool for social move- ment: An analysis of feminist activism on social me- dia communities. Journal of community psychology.
Jesse Graham, Jonathan Haidt, Sena Koleva, Matt Motyl, Ravi Iyer, Sean P Wojcik, and Peter H Ditto. 2013. Moral foundations theory: The pragmatic va- lidity of moral pluralism. In Advances in experi- mental social psychology, volume 47, pages 55-130. Elsevier.
49(3):854 868.
Yingjie Li and Cornelia Caragea. 2019. Multi-task
stance detection with sentiment and stance lexicons. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-JCCNLP)) pages 6299-6305.
Zihao He. Negar Mokhberian, and Kristina Lerman. 2022. Infusing knowledge from wikipedia to enhance stance detection. In Proceedings of the 12th Work- shop Computational Approaches to Subjectivity, Sentiment & Social Media Analysis. WASSA @ @ACL 2022, Dublin, Treland, May 26, 2022, pages 71-77. Association for Computational Linguistics.
Yingjie Li, Tiberiu Sosea, Aditya Sawant, Ajith Ja- yaraman Nair, Diana Inkpen, and Cornelia Caragea, 202 P-stance: A large dataset for stance detection in political domain. In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021. pages 2355-2365.
Daniel Hickey, Matheus Schmitz, Daniel Fessler, Paul E Smaldino, Goran Muric, and Keith Burghardt. 2023. Auditing elon musk's impact on hate speech and bots. In Proceedings of the International AAA/ Conference on Web and Social Media, volume 17, pages 1133- 1137.
Maurice Lineman, Yuno Do, Yoon Kim, and Gea-Jae Joo. 2015. Talking about climate change and global warming. PloS one, 10(9):00138996
Zhengyuan I Liu, Yong Keong Yap. Hai Leong Chieu, and Nancy FF Chen. 2023. Guiding computational stance detection with expanded stance triangle frame- work. arXiv preprint arXiv:2305 19845.
Frederic R Hopp. Jacob T Fisher, Devin Cornell, Richard Huskey, and René Weber. 2021. The ex- tended moral foundations dictionary (emfd): De- velopment and applications of crowd-sourced ap- proach to extracting moral intuitions from text. Be- havior research methods, 53:232-246.
Sudhanshu Mishra, Shivangi Prasad, and Shubhanshu Mishra. 2021. Exploring multi-task multi-lingual
learning of transformer models for hate speech and offensive speech identification social media. SN Computer Science, 2:1-19.
Hossein Hosseini, Sreeram Kannan, Baosen Zhang, and
Radha Poovendran. 2017. Deceiving google's 's per- spective api built for detecting toxic comments. arXiv preprint arxiv: / 702.08138.
Saif Mohammad, 2018. Obtaining reliable human rat- ings of valence, arousal, and dominance for 20,000 english words. In Proceedings of the 56th annual meeting of the association for computational linguis- tics (volume 1: Long papers), pages 174-184.
Homa Hosseinmardi, Sabrina Arredondo Mattson, Ra- hat Ibn Rafiq, Richard Han, Qin Lv, and Shivakant Mishra. 2015. Analyzing labeled cy cyberbullying in- cidents on the instagram social network. In Interna- tional conference on social informatics, pages 49-66. Springer.
Saif Mohammad, Svetlana Kiritchenko, Parinaz Sob- hani, Xiaodan Zhu, and Colin Cherry. 2016. Semeval- 2016 task 6: Detecting stance in tweets. In Proceed- ings of the 10th international workshop on semantic evaluation SeemEval-2016) pages 31-41.
Binxuan Huang and Kathleen M. Carley, 2018 Parame- terized convolutional neural networks for aspect level sentiment classification. In Proceedings of the 2018 Conference on Empirical Methods Natural Lan- guage Processing, Brussels, Belgium, October 31 November 4, 2018, pages 1091-1096. Association for Computational Linguistics.
Saif M. Mohammad. 2022. Ethics sheet for automatic emotion recognition and sentiment analysis. Comput. Linguistics, 48(2):239-278
Lili Mou, Rui Men, Ge Li, Yan Xu, Lu Zhang, Rui Yan, and Zhi Jin. 2015, Natural language inference by tree-based convolution and heuristic matching. arXiv preprint arXiv:1572.000 1512.0822.
Chuma Kabaghe and Jason Qin. 2020. Classifying tweets based on climate change stance. Training. 66(60.9):61.
Dat Quoc Nguyen, Thanh Vu, and Anh Tuan Nguyen. 2020. BERTweet: A pre-trained language model for English tweets. In Proceedings of the 2020 Con- ference on Empirical Methods in Natural Language Processing: System Demonstrations, Online. Associ- ation for Computational Linguistics.
Amandeep Kaur and HS Chahal. 2018. Role of social media in increasing environmental issue awareness. Researchers World, 199:17
Alexandre Lacoste, Alexandra Luccioni, Victor Schmidt, and Thomas Dandres. 2019. Quantifying the carbon emissions of machine learning. CoRR, abs/1910.00 10009900
Charles Egerton Osgood, George J Suci, and Percy H Tannenbaum. 1957. The measurement of meaning. 47. University of Illinois press.

---

John Pavlopoulos, Nithum Thain, Lucas Dixon, and lon Androutsopoulos. 2019. Convai at semeval- 2019 task 6: Offensive language identification and categoriza- tion with perspective and bert. In Proceedings of the 13th international Workshop on Semantic Evaluation, pages 571 576-
Apoorva Upadhyaya, Marco Fisichella, and Wolfgang Nejdl. 2023c. A multi-task model for sentiment aided stance detection of climate change tweets. In Pro- ceedings of the International AAAI Conference on Web and Social Media, volume 17, pages 854-865.
Apoorva Upadhyaya, Marco Fisichella, and Wolfgang Nejdl. 2023d. Towards sentiment and temporal aided stance detection of climate change tweets. Informa- lion Processing & Management, 60(4):103325.
Shalini Priya, Apoorva Upadhyaya, Manish Bhanu, Sourav Kumar Dandapat, and Joydeep Chandra. 2020. Endea: Ensemble based decoupled adversarial learning for identifying infrastructure damage dur- ing disasters. In Proceedings of the 29th ACM in- ternational conference on information & knowledge management, pages 1245-1254.
Roopal Vaid, Kartikey Pant, and Manish Shrivastava. 2022. Towards fine-grained classification of climate change related social media text. In Proceedings of the 60th Annual Meeting of the Association for Com- putcational Linguistics: Student Research Workshop, pages 434443.
Maud Reveilhac and Gerold Schneider. 2023. Repli- cable semi-supervised approaches to state-of-the-art stance detection of tweets. Information Processing Management, 103999..
Ameya Vaidya, Feng Mai, and Yue Ning. 2020. Em- pirical analysis of multi-task learning for reducing identity bias in toxic comment detection. In Proceed- ings of the International AAAI Conference on Web and Social Media, volume 14, pages 693.
Rezvaneh Rezapour, Ly Dinh, and Jana Diesner. 2021. Incorporating the measurement of moral foundations theory into analyzing stances on controversial top- ics. In Proceedings of the 32nd ACM conference on hypertext and social media, pages 177-188.
Ashish Vaswani. Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in neural information pro- cessing systems, pages 3998-6008
James A Russell, 1980. A circumplex model of af- feet. Journal of personality and social psychology, 3966).111.
James A Russell. 2003. Core affect and the psycholog- ical construction of emotion. Psychological review, 110(1):145.
Oxana Vitman, Yevhen Kostiuk, Grigori Sidorov, and Alexander Gelbukh. 2022. Sarcasm detection frame- work using emotion and sentiment features. arXiv preprint urXiv:2211.13014
Tulika Saha, Apoorva Upadhyaya, Sriparna Saha, and Pushpak Bhattacharyya. 2021. A multitask multi- modal ensemble model for sentiment-and emotion- aided tweet act classification. JEEE Transactions on Computational Social Systems.
Sergey Vychegzhanin and Evgeny Kotelnikov. 2021. A new method for stance detection based on feature se- lection techniques and ensembles of classifiers. IEEE Access, 9: 114499--14911,
Parinaz Sobhani, Saif Mohammad, and Svetlana Kir- itchenko. 2016. Detecting stance in tweets and ana- lyzing its interaction with sentiment. In Proceedings of the fifih joint conference on lexical and computa- tional semantics, pages 159-169.
Limin Wang and Dexin Wang. 2021. Solving stance detection on tweets as multi-domain and multi-task text classification. IEEE Access, 9:157780-157789
Zhongqing Wang. Qingying Sun, Shoushan Li, Qiaom- ing Zhu, and Guodong Zhou. 2020. Neural stance detection with hierarchical linguistic representations. IEEE/ACM Transactions on Audio, Speech, and Lan- guage Processing, 28:635-645.
Robert L Spitzer. Jacob Cohen, Joseph L Fleiss, and Jean Endicott. 1967. Quantification of agreement in
psychiatric diagnosis: A new approach. Archives of General Psychiatry, 17(1):83-87.
Wei Xue and Tao Li. 2018. Aspect based sentiment anal- ysis with gated convolutional networks. In Proceed- ings of the 56th Annual Meeting of the Association for Computational Linguistics, ACL 2018, Melbourne, Australia, July 15-20, 2018, Volume 1: Long Papers, pages 2513-2523. Association for Computational Linguistics.
Apoorva Upadhyaya, Marco Fisichella, and Wolfgang Nejdl. 2023a. Intensity-vall -valued emotions help stance detection climate change twitter data. In Proceed- ings of the Thirty-Secom International Joint Confer- ence on Artificial Intelligence, IJCAI 2023, 19th-25th August 2023, Macao, SAR, China, pages 2254-255. ijcai.org.
Raneen Younis, Zahra Ahmadi, Abdul Hakmeh, and Marco Fisichella. 2023. Flames2graph: An inter- pretable federated multivariate time series classifica- tion framework. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 3 3140-3150.
Apoorva Upadhyaya. Marco Fisichella, and Wolfgang Nejdl. 2023b. A multi-task model for emotion and offensive aided stance detection of climate change tweets. In Proceedings of the ACM Web Conference 2023, pages 3948-3958.

---

Kai Zheng, Qingfeng Sun, Yaming Yang, and Fei Xu. 2022. Knowledge stimulated contrastive prompting for low-resource stance detection. In Findings of the Association for Computational Linguisties: EMNLP 2022, pages 1168-1178.
the non_toxic, harm, betrayal features that are com- mon in the none stance category of the joass domain. This leads to poor performance of our TWISTED approach in these target domains com- pared to other basic tasks.
Yiwei Zhou, Alexandra Cristea, and Lei Shi. 2017. Connecting targets to tweets: Semantic attention- based model for tprget-specific stance detection. In International Conference on Web Information Sys- tems Engineering, pages 18-32. Springer.
(ii) Ambiguity/CCnffticinng Stances: Some tweets have a contradictory attitude, either toward single target or toward multiple targets. In Example 6, can that the tweeter believes that climate change is real, but does not believe in climate ac- tion by government officials. Such scenarios are misclassified by our approach, although further categorization of the target or identification of mul- tiple targets could be further helpful for the task SD.
A
Case Studies
To gain insight into the predictive performance of our approach, we present some of the predictions by different variants of our approach in Table 5. The red markers in the labels indicate incorrect pre- dictions. Examples and 3 are correctly detected by our model. The tweets do contain contrasting emotions, with the first sentence being positive but the second part containing negative emotions, lead- ing hint of sarcasm in the tweet. However, the presence of dominant toxie, moral, and linguistic features found in the tweets of "Sanders against at- titude" and "climate denier" correctly captures the overall intent and attitude of the tweet, thus elim- inating the drawbacks of the previous works. In Example 2, it can be seen that the combination of SD +TD+ MC is not able to correctly identify the stance, the moral features harm" and "degrada- tion" are more common in the "none" and "favor" stance categories of the "feminism" " domain. Due to the toxic and suggestive language that more prevalent in the "Against" and "Climate Denier" categories, SD +TD+ AC and the combinations of all tasks correctly predict attitude, justifying the higher loss weight for the tweet act than for the moral classification task. We list some possible reasons for the errors in the SD task.
B
Dataset
B.1
Data Pre-processing
Since removing query hashtags may result the tweet not expressing the same attitude, we first also remove the query hashtags from the datasets in accordance with previous literature (Mohammad et al., 2016). We then clean up the tweets by re- moving URLs, punctuation, spaces, stop words, and unwanted characters like RT and CC. To to- kenize tweets, we use the NLTK-based TweetTo- kenizer. Tokens are converted to lowercase and then stemmed with PorterStemmer, which reduces words to their stems and suffixes.
B.2 Percentage Distribution of features in Datasets
Tables 6, 7, 8, and 9 represent the percentage dis- tribution of toxic (toxic/non_toxiq) moral (care, harm, fairness, cheating, loyalty, betrayal, authority, subversion, purity, degradation) and speech act (ex- pression, others, question, statement, suggestion) among the different categories of attitudes favor, against, none) corresponding to the SemEval-2016 P-Stance, ClimateChange, and COVID -19-Stance datasets respectively.
A.1 Error Scenarios
(i) Close Proximity of auxiliary features among stances: As shown in Examples 4 and 5, TWISTED does not correctly identify stances due
B.3 Manual Verification
to the close correspondence of attitudes with re- spect to the extracted virtue, vice, toxic, or act features, especially in the atheism and mask do- mains, which lead scenarios in which auxiliary tasks are not helpful. The toxic, harm, subversion, and degradation features, as well as the speech act expression in Example 4, are more prevalent in the favor attitude category, while Example contains
To examine the annotations of weakly supervised approaches, three trained annotators randomly se- lected 200 twects from 4 datasets used in our study, yielding total of 1000 tweets, and performed man- ual annotations provide toxic, moral, and speech act labels (the inter-annotator agreement was caleu- lated using the Fleiss Fleiss-Kappa score and was 0.82). We then matched these manual annotations with

---

Single- SD
No.
Tweet
True labels
SD4TD+AC SD-TD+MC
MC+AC
thing kids actually believe
against
#BornicSanders going to be able to off school dehi. (Sanders)
toxic
but literally
morakttarm, cheating actiesgresiin
favor
against
against
against
fominist
absolutely
no one would shove their somd hard
you. that sink AScnST (Peminism)
maral.tarm, degradation actsuggestion
toxic
favor
against
favor
against
you're are 46 hullshi!!! the biggest icon knewa to man along with. (ClimateChunge)
toxic
believe
dany
dany
moral.turm, betryaal
actresprussion
stanresagainstt
humunity with eager unticipation. We deserve nothing less #SSmST (Athcismn)
toxic
fevor
favor
favor
favor
moral: harm, subversion, degradation actickpression
favor
are doing Shame on MMn (face_
200, toxic
2006
nome
none
none
acticxpecssion
Miami Crisas! Warmaig real!!
PlanciDettrution Hapening!! all while
deny
the rich and famous keep buying waterfrons
toxic
believe
boliove
boliove
believe
#HullShir sProgagunda (CllmmeeChangee/BBuu ChangeloHullSh (ChmnteCChnge))
actistatement
Table 5: Sample tweets with true and predicted labels for single and multi-tase task models.
Climate against
Feminism against
Hillary
Attribute
favor
against
favor
favor
favor
against
favor
against
Toxicity Detection
Toxic
31.45
7.33
26.21
5,68
7.69
13.31
44.77
63.99
29.41
10.19
27.39
23.78
32.45
26.19
27.28
toxic
92.67
94.32
73.81
Moral Classification
care
14,52
16.81
15.86
14.63
15.38
12.81
7,84
8.22
5.88
13.38
12.76
18.85
9.27
10.71
7.18
harm
33.87
34.27
33.79
50.75
65.38
47.29
50.75
44.81
54.71
29.3
34.52
29.92
45.7
48.18
$1.67
fairness
15.38
12.81
cheating
20.16
18.32
19.31
10.6
17.69
16.26
27.61
28.77
27.06
29.3
31.33
31,15
22.52
25.24
22.97
loyalty
19.35
22.84
22.76
14.93
0.0
17.24
9.7
12.72
8.82
21.02
24.02
24.18
15.23
13.58
11.48
betrayal
8.06
10.13
8:28
7.46
19.23
8.37
10.45
11.74
8.82
11.46
10.32
13.11
11.26
13.96
18.18
authority
9.55
12.81
sulversion
11,29
9.7
6.21
11.94
0.0
15.27
13.81
12.53
15.29
14.01
14.07
14.75
9.27
10.71
5.74
purity
16.13
15.52
14.48
6.27
0.0
5.91
9.7
7.44
10.0
5.73
5.07
8.61
6.62
8.22
10.53
degradation
14.52
13.79
11.03
17.61
46.15
17.73
26.87
25.83
25.29
14.01
15.2
13.93
20.53
21.22
23,92
Classification
expression
33,06
29.31
27.59
17.01
26.92
15.27
35.48
26.22
21.17
26.11
18.57
15.57
48.34
35.76
21.53
others
19.35
21.34
18.62
29.55
19.23
33.49
17.54
17.61
31.18
33.12
31.52
30.74
12.58
23.52
29.19
question
4,03
2.59
4.14
5.37
0
7.88
6.34
6.06
1.76
3.82
4.88
3.69
3.97
4.39
4.78
37.44
42.07
41.17
22.52
suggestion
10.48
10.99
8.28
10.45
7.69
5.91
6.72
8.02
4.07
9.55
8.19
12.59
7.65
9.99
Table 6: Percentage distribution of toxic, moral, and features in SemEval-2016 dataset

---

Trump
Biden
Sanders
ClimateChange
Attribute
favor
against
favor
against
favor
against
Attribute
believe Toxicity
deny Detection
ambiguous
Toxicity Detection
toxic
31.94
49.17
16,66
19.18
48.16
toxic
3.94
21.65
5.34
non toxic
68.06
50.83
83.34
57.2
80.82
51.84
non toxic
96.06
78.35
94.66
Moral Classification
Moral Classification
care
7.21
6.78
11.19
10.17
9.77
care
18.67
14.68
18.15
harm
38.78
45.09
42.78
harm
43.11
45.67
39.79
41.6
38.08
39.1
fairness
14.77
9.41
13.97
fairness
11.66
11.73
13.49
13.29
14.31
12.68
cheating
16.06
15,9
14.48
cheating
32.62
34.46
28.19
33.0
30.48
32.14
loyalty
22.38
20.88
19.44
loyalty
20,53
16.25
22.85
19.2
24.42
21.87
betrayal
8.41
9.2
8.82
betrayal
11.44
12.19
10.66
10.17
12.11
12.22
authority
12.9
14.27
8. 8.76
authority
12.91
10:24
13.89
11.38
14.11
12.54
subversion
15.6
15.8
18.44
subversion
17.64
17.95
14.14
15,1
15.69
15.6
purity
10.25
9.2
9.19
purity
3.82
3.5
4.63
4.41
3.49
3.64
degradation
11.91
16.36
14.06
degradation
14.93
16.09
15.26
18.46
11.8
12.97
Speech Act Classification
Speech Act Classification
expression
44.8
46.45
37.63
expression
24.65
27.07
55.42
57.83
40.25
43.56
others
16.3
14.75
12.6
others 25.53
24.32
2.46
1.08
15.86
15.17
question 3.78
3.83
3.77
question
0.74
0,86
0.12
0.27
4.37
4.36
statement
24.94
29.12
35.94
statement
45.07
42.81
38.45
38.15
31.43
28.
suggestion
10.17
5.84
10.06
suggestion
4.01
4.93
3.54
2.67
8.08
8.54
Table 8: Percentage distribution of toxic, moral, and act features in ClimateChange dataset
Table Percentage distribution of toxic, moral, and act features in P-Stance dataset
stay home
face, masks
school closures
fauci against
Attribute
favor
against
none
favor
agalust nome Toxicity Detection
favor against
none
favor
none
toxic
19.47
34.75
12,79
40.41
42.07
34.79
24.88
14.4
17.23
42,48
47.21
40.29
non toxic
80.53
62.25
87.21
59.59
57.93
65.03
75.12
85.6
82.77
57.52
52.79
59.71
Moral Classification
care
25.26
6.0
14.32
7.65
7.19
9.25
15.82
12.8
13.85
8.94
8.2
5.25
harm
39.47
39.75
30.31
47.62
49.55
49.71
47.48
39.2
40.62
50.41
56.89
51.84
fairness
20.53
13.75
13.04
26.55
20.66
12.43
28.29
32.8
34.15
15.65
10.0
12.6
cheating
18.95
31.5
16.37
20.35
28.59
26.3
17.72
17.6
22.77
26.02
34.75
30.05
loyalty
18.42
8.5
16.75
12.7
9.73
13.87
7.8
9.6
10.77
15.85
11.15
10.24
betrayal
3,68
8.75
5.48
10.69
21.79
20.8
16.31
9.96
9.34
authority
5.5
5.34
10.15
9.55
subversion
12.11
15.5
19.34
13.6
15.69
16.26
12.95
purity
17.37
4.75
18.04
8.67
6.4
9.76
degradation
8.95
19.0
12.99
12.92
17.89
Speech Act Classification
expression
44.21
42.75
32.61
31.44
38.43
44.72
37.2
42.15
47.97
41.15
41.47
others
6.84
8.5
37.08
19,19
18.86
17.63
8.45
14
8.31
11.38
13.28
11.68
question
7,36
7
7.03
10.53
9.43
7.51
8.45
7.6
7.38
7.32
6.72
8.01
statement
30
26.5
22.37
26.84
28.29
26.88
31,21
31.2
32.92
25.81
29.84
30.84
suggestion
11.58
15.25
7.54
10.82
11.97
9.54
7.16
10
9.23
7.52
9.02
8.01
Table 9: Percentage distribution of toxic, moral, and act features in COVID-19-Stance dataset

---

Component
Precision
Recall
F1
D Significance of Components
AvgSStdev
AvgSt.ddv
Avg/St.dev
Single Task Sct-up
Tables 10 and 11 justify the significance of each proposed component of our TWISTED approach. In addition, the tables 10 and 11 present the re- sults of single-task and multi-task setups for detect- ing stance on the ClimateChange and COVID-19 datasets, respectively, and show the improvement in the performance of the model in addition to each component.
BERTwweel
33.19/1.61
86.01/2.04
84.72/1.58
BERD
87.33/0.55
88.21/0.69
87.09/1.04
VADD
89.51/1.37
87.65/0.91
88.55/1.15
+MHA
VERD
+(oie,
90.36/1.11
89.17/1.42
90.01/1.11
mprechppeechjas up
Multi Task Set-up
BERTweet
92.63/1.00 .08
91.080.66
91.75/1.42
BERD
94.71/0.62
94.24/0.58
94.64/0.49
BERT
93.19/0.46
96.11/0.23
96.05/0.32
(TWISTED)
Table 10: Sigmificance of proposed components on Cli- mateChange dataset
Component
Fauci
Home Set-up
Mask
Schools
Single
BERTweet
80.07
79.15
76.82
75.49
BERTweet+
83.45
82.27
77.81
79.18
BEKTweet
84.73
83.81
80.05
81.09
+MHA
BERTwact+VA
+MHie(
86.01
84.09
81.64
82.88
muralapeechhas ilp
Multi Task Set-up
BERTweet
87.14
86.84
80.67
81,32
BERTweeH+AD
90.06
89.52
82.11
85.66
BERTweet VAD
91.05
91.63
85,29
89.78
+MH (TWISTED)
Table 1I: Significance of proposed components on COVID-19 dataset
labels generated by semi-supervised approaches for the same 1000 tweets and found Heiss-Kappa (Spitzer et al., 1967) scores of 0.81, 0.76, and 0.79 for toxic, moral, and speech acts, respectively, indi- cating that the labels predicted by semi-supervised approaches of considerably good quality.
Environment Details
GPU Model: NVIDIA GeForce GTX 1080Ti GPU servers (TDP of 250W) with carbon efficiency of 0.38 kgCO,ee/kWh, Library Version: Tensor- flow 2.12.0, Keras 2. 2..0. Transformers 4.30.2, Computational Cost: On an average, 30 minutes training time for TWISTED for one round. Av- erage 5 rounds for each reported result for Se- mEval, P-Stance, and COVID, while the 5-fold cross-validation technique in Climate takes approx- imately 2 hours executing time, resulting in to- tal of 10 hours of computation, leading to 0.95 kgCOzeq carbon emissions. Carbon footprint is calculated using the Machine Learning Impact cal- culator (Lacoste et al., 2019).


Reviews:
[{"review_id": "frl6uhnuaq", "reviewer": "Reviewer_o8Ty", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 5, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the comment. We agree with the reviewer that multi-task learning is not a novel technique and has been utilized in previous works. However, the use of toxicity detection together with morality and speech act classification tasks for the main task of stance detection has not yet been explored in the existing literature. In our work, we focus on showing the importance of using toxic, moral, and speech act aspects associated with the tweet and utilize these interdependencies to understand the overall intent of the tweet, which ultimately captures the attitude of the tweet efficiently. Moreover, the valence, arousal, and dominance features that give emotional meaning to the tweet are leveraged by the efficient embedding technique and attention framework, which also improves the performance of our proposed approach in classifying the attitude of the input tweet.\n\n**Missing References**: We plan to cite the three mentioned works in the missing references: “Empirical Analysis of Multi-Task Learning for Reducing Identity Bias in Toxic Comment Detection” (work is mentioned twice with both arxiv and published versions), “Multi-Task Stance Detection with Sentiment and Stance Lexicons”, “Exploring Multi-Task Multi-Lingual Learning of Transformer Models for Hate Speech and Offensive Speech Identification in Social Media” that show the efficacy of using multi-task architecture for various other tasks. We would like to notify that 2 of the missing references are not directly related to the main task of stance and explore the multi-task set-up, where we tried to refer to works that use multi-task architecture and proposed their models for the stance task as baselines such as MT-LRM-BERT, SP-AMT, STASY, MEMOCLiC, AT-JSS-LEX, and S-MDMT methods. We also have used the AT-JSS-LEX model proposed in the “Multi-Task Stance Detection with Sentiment and Stance Lexicons” work as a baseline method that directly correlates with the stance task for the SemEval dataset and our model outperforms the AT-JSS-LEX model. Further, we did not mention AT-JSS-LEX results on the Climate Change dataset as other recent models such as SP-AMT (ICWSM-2023) and MEMOCLiC (WWW/ACM Web 2023) mentioned in our paper have already outperformed the AT-JSS-LEX model, and we used these SP-AMT and MEMOCLiC models in our work as baseline methods. However, we will surely mention all the works in the related section.\n\nAs mentioned by the **“Excitement”** rating of 3 that work can benefit from another round of revision, we plan to include the ablation study that shows the significance of proposing the components for our TWISTED model and justifies the intuition and motivation behind using BERTweet, VAD, and MultiHeadAttention frameworks in both single and multi-task setup (as pointed out by the Reviewer 9nz9). We currently implement the results for the ClimateChange dataset due to time constraints during the rebuttal period (refer to Table 1 mentioned in Reviewer 9nz9 rebuttal), however, we will implement the results for other datasets as well and include them in the appropriate appendix/Results section depending on the space constraints.\n\nRegarding **reproducibility**, we already share the dataset (with annotations) and code URL in the submitted manuscript for the reviewers, and will make the code publicly available so as to reproduce the results. We mention precisely all the hyperparameters required to implement our model in Section 4.2. In line with the existing works, we use already available train, validation, and test splits of the SemEval-2016, P-Stance, and COVID-19-Stance datasets and implement our approach with 5-fold cross-validation technique for the ClimateChange dataset (these details are mentioned in Section 4.2 under the heading “Evaluation Metrics”). For the users still facing any issues in reproducibility, we intend to make the pre-trained model available to those users so that anyone who tries to reproduce the results can easily use the pre-trained model without any issues.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "Well written paper\nGood extensive experiments\nSignificant improvements over other stated methods\nInclusion of qualitative examples around improvements using the various features in appendix\n", "reasons_to_reject": "Including Neural + Lexical features is quite common in NLP\nMTL is not a novel technique and has been shown to work on other tasks including multiple tweet specific tasks. See missing references section", "questions_to_authors": "", "paper_topic_and_main_contributions": "The authors propose using word count features based on Valance, Arousal, and Dominace alongside a BERT Encoder and a multi-task objective based on Toxicity, Morality, and Speech Act to demonstrate significant performance improvement on Stance Detection tasks for Tweets.", "excitement": 3, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Missing_References": "https://aaai.org/ojs/index.php/ICWSM/article/view/7334 \nhttps://arxiv.org/pdf/1909.09758.pdf \nhttps://aclanthology.org/D19-1657.pdf \nhttps://link.springer.com/article/10.1007/s42979-021-00455-5\n"}, "extra_text": {}}, {"review_id": "RVPkMZoeSI", "reviewer": "Reviewer_9nz9", "summary": "", "questions": "The one question I would have, less for a rebuttal but more for the final version of the paper, is providing additional motivation for each of the added layers. Can you provide any more intuition on why you set the model up as you did?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for complementing our work on the grounds of approach, intuition, evaluation, and performance improvement. \n\nAs pointed out by the reviewer **“Can you provide any more intuition on why you set the model up as you did?”**, We plan to add an ablation study that justifies the significance of the components used in the proposed model (BERTweet, BERTweet+VAD, BERTweet+VAD+MHA in both single-task and multi-task set-up). We currently implement the results for the ClimateChange dataset due to time constraints during the rebuttal period as it does not contain any sub-domains like other datasets, however, we will implement the results for all datasets and include them in the manuscript. Table 1 proves the importance of each component and shows the improvement in the model’s performance in addition to each component on the ClimateChange dataset.\n\n| Component  | Precision | Recall | F1 Score | \n|---------|----------------|--------|----------------|\n|---------|(Avg./Std. dev)|(Avg./Std. dev)|(Avg./Std. dev)|\n|**Single-Task Setup**|\n|BERTweet|83.19/1.61|86.01/2.04|\t84.72/1.58|\n|BERTweet+VAD|87.33/0.55|88.21/0.69|87.09/1.04|\n|BERTweet+VAD+MHA|89.51/1.37|87.65/0.91|88.55/1.15|\n|BERTweet+VAD+MHA+[toxic,moral,speech act] as input features|90.36/1.11|89.17/1.42|90.01/1.31|\n|**Multi-Task Setup**|\n|BERTweet|92.63/1.08|91.08/0.66|\t91.75/1.42|\n|BERTweet+VAD|94.71/0.62|94.24/0.58|94.64/0.49|\n|BERTweet+VAD+MHA (Ours TWISTED)|95.19/0.46|96.11/0.23|96.05/0.32|\nTable 1:  Results of Single-Task and Multi-Task Stance Detection on ClimateChange dataset in various combinations to show the significance of each component used in our TWISTED approach \n\nSince, we use Twitter data, the BERTweet encoder specially trained on tweets, helps encode the input text in a better way as mentioned in previous literature. We investigate that the VAD features reflect the emotional sense of the tweet. In this manner, we capture the emotional meaning of the input tweet by using these features in conjunction with the text of the tweet. Instead of simply averaging the encoded text and VAD features, which might lose the influence of valence, arousal, and dominance features over the text, we use the Emo-Sense Injector module, which acts as an integrating module and fuses the text and VAD features using the absolute difference and element-wise product to capture the entire emotional essence of the tweet in terms of the semantic and syntactic representation of the tweet text and not miss any related features by simply averaging or concatenating functions. Then, we use the MultiHeadAttention layer rather than the single attention layer to focus on all of the relevant parts of the emotional-sense injected tweet from all subspaces so that we do not avoid any important feature. While we have tried to mention all these details in Section 3 (Methodology), we will elaborate on these details in all subsections of Section 3 to make the motivation even clearer. We also refer to the tables for all datasets similar to those presented above (Table 1) under the heading “Significance of the proposed components”.\n\nWe also thank the reviewer for pointing out that our contribution is novel and agree that our manuscript leverages the multi-task setup. To the best of our knowledge, this is the first cross-sectional study that exploits toxic, moral, and speech act aspects hidden within the tweet as auxiliary tasks and fuses the VAD features followed by the efficient encoding and attention frameworks to perform the main task of stance detection on various datasets comprising of different domains and targets.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "+ Sound and clear methods - the approach is straightfoward (in a good way) and intuitive (also in a good way), and well explained\n\n+ Reasonable evaluation - the evaluation is comprehensive in both data and comparison methods\n\n+ Results show considerable improvement over baselines\n\n", "reasons_to_reject": "- There's nothing groundbreaking here. I don't think this is necessarily a significant reason to reject, but worth noting that this paper feels somewhat incremental in terms of the conceptual novelty of leveraging affect and a multi-task setup.  Nonetheless, teh contribution *is* novel", "questions_to_authors": "The one question I would have, less for a rebuttal but more for the final version of the paper, is providing additional motivation for each of the added layers. Can you provide any more intuition on why you set the model up as you did?", "paper_topic_and_main_contributions": "The main contribution of this paper lies in the development and evaluation of a novel approach to stance detection, known as TWISTED, which leverages additional tasks such as toxicity, morality, and speech act to enhance stance detection performance. The authors propose a multi-task learning setup where these auxiliary tasks inform the primary task of stance detection.\n\nThis approach is applied to data from Twitter; the proposed model is evaluated on four different datasets, each targeting different topics, and the results show that it outperforms several baseline models, demonstrating the effectiveness of this approach.", "excitement": 4, "reproducibility": 5, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Missing_References": "None", "Typos_Grammar_Style_and_Presentation_Improvements": "None"}, "extra_text": {}}, {"review_id": "docQwR4sKy", "reviewer": "Reviewer_ygJA", "summary": "", "questions": "", "limitations": "", "rating": 2, "confidence": 4, "soundness": 2, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**1.** We have already compared the performance of the MEMOCliC model proposed in the recent work “A multi-task model for emotion and offensive aided stance detection of climate change tweets” on SemEval-2016 and ClimateChange datasets which authors of MEMOCLiC used to evaluate their model performance. The experimental results show that our model outperforms the MEMOCLiC approach. The model has been proposed for the climate change domain and we have clearly mentioned and compared our approach with the MEMOCliC pointing out its shortcomings as well as benefits from our approach in 2 sections of the manuscript:\n\n   * **Section 1 Introduction:**\n   *Other recent works such as MEMOCLiC \\cite{upadhyaya2023multi}, SP-AMT \\cite{upadhyaya2023multisenti} have used……valence, \n    arousal, and dominance features help in analyzing emotional tone and its intensity…. supports the detection of hidden intent or any \n     sarcasm………thus addressing the drawbacks of the presence of sarcasm and implicit stance from previous works.*\n   * **Section 5 ClimateChange:**\n    *Although MEMOCLiC extracts the emotional and toxic features….., but the performance power of our EmoSense injector, which injects \n    the VAD features, understanding ….hidden moral and linguistic features, is one of the main reasons contributing to the success of \n    TWISTED compared to the recent works MEMOCLiC, SP-AMT.* \n\n**2.** We have incorporated the qualitative analysis of our model by mentioning the case studies in Appendix A “Case Studies”, discussing some of the predictions of the different variants of our proposed approach. We have already presented the ablation study under the heading “Ablation Study” in Section 5, where we examine and present the F1 values (Figure 5) of the single task, all auxiliary tasks, and their combinations for the stance task. We also plan to include the ablation study, which shows the importance of proposing the components for our TWISTED model and justifies the motivation behind the use of the BERTweet, VAD, and MultiHeadAttention components (as highlighted by Reviewer 9nz9). Due to time constraints during the rebuttal period, we currently implemented the results for the ClimateChange dataset (refer to Table 1 mentioned in Reviewer 9nz9 rebuttal). However, we will implement the results for all datasets and include them in the manuscript.\n\n\n\n**3.** We thank the reviewer for suggesting the references. As indicated in the comment, one of the papers (A Holistic Framework for Analyzing the COVID-19 Vaccine Debate) examines the stance, reason, and moral grounds in COVID tweets, while another (Understanding COVID-19 Vaccine Campaign on Facebook using Minimal Supervision) proposes a multitasking setup for identifying themes and moral foundations for Facebook ads related to COVID. The works are quite interesting and inspire us to look into the direction of how ads and themes are relevant to the formation of public opinion as part of future work. Since our main focus in the current work is to identify the attitude of the public towards different target areas, we use the interdependencies between moral foundations, toxicity, and speech acts as auxiliary tasks for the main stance task. We have also tried to incorporate relevant and popular works for all datasets and domains we use in our manuscript. However, we will certainly include the suggested references in our manuscript and explore the dimensions mentioned in these works as part of our future research direction.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "Interesting use of toxicity, morality, and speech acts presented in the tweet as auxiliary tasks of detecting stance (main task).", "reasons_to_reject": "1. Small-scale contribution. This work is not much different than MEMOCLiC (recent work named \"A multi-task model for emotion and offensive aided stance detection of climate change tweets.\"). They just injected the VAD features.\n2. Lack of qualitative analysis.\n3. Need more exploration in related work. Authors should explore more works related to stance and moral foundation especially on climate change, politics and COVID-19 related work.\n\nAuthors should include the stance detection task on climate change by Islam et al 2023 (Analysis of Climate Campaigns on Social Media using Bayesian Model Averaging). In this work, authors showed how theme information improves stance detection.\n\nAuthors should explore the stance detection task on political campaigns by Islam et al 2023 (Weakly Supervised Learning for Analyzing Political Campaigns on Facebook). \n\nAuthors should include the stance detection task on COVID-19 by Pacheco et al 2022 (A holistic framework for analyzing the covid-19 vaccine debate). In this work authors have annotation both for stance and moral foundation as well as morality. \n\nAlso for COVID-19 moral foundation, authors could look into work by Islam and Goldwasser 2022 (Understanding COVID-19 Vaccine Campaign on Facebook using Minimal Supervision).", "questions_to_authors": "", "paper_topic_and_main_contributions": "The authors presented a stance detection task using the associated toxic, moral, and speech act traits present in tweets (they proposed a multitasking model named TWISTED). They conducted  4 benchmark datasets to show that multi-tasking enhances the performance of the stance task compared to the single-task and baseline methods by leveraging the auxiliary tasks.", "excitement": 2, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {"Missing_References": "Islam et al 2023 (Analysis of Climate Campaigns on Social Media using Bayesian Model Averaging).\n\nIslam et al 2023 (Weakly Supervised Learning for Analyzing Political Campaigns on Facebook). \n\nPacheco et al 2022 (A holistic framework for analyzing the covid-19 vaccine debate). \n\nIslam and Goldwasser 2022 (Understanding COVID-19 Vaccine Campaign on Facebook using Minimal Supervision)."}}]

Author rebuttal:

Meta-review: This paper proposes a new approach to social media stance detection based on speech act, toxicity, and moral features of tweets. In extensive experiments, they demonstrate that multi-task learning improves stance detection. 

The following concerns have been raised by reviewers:
* It does not seem that the proposed methods are truly novel. This is because lexical and neural features and MTL are quite common in NLP.
 * Lack of qualitative analysis and small-scale experiment
 * References and comparisons with prior works are missing. 
    - Empirical Analysis of Multi-Task Learning for Reducing Identity Bias in Toxic Comment Detection
    - A multi-task model for emotion and offensive aided stance detection of climate change tweets.
    - Exploring Multi-Task Multi-Lingual Learning of Transformer Models for Hate Speech and Offensive Speech Identification in Social Media
    - Task Stance Detection with Sentiment and Stance Lexicons
Decision: Accept-Findings