Not All Counterhate Tweets Elicit the Same Replies: A Fine-Grained Analysis
Year: 2023

Abstract:
Counterhate arguments can effectively fight and limit the spread of hate speech. However, they can also exacerbate the hate, as some people may respond with aggression if they feel threatened or targeted by the counterhate. In this paper, we investigate replies to counterhate arguments beyond whether the reply agrees or disagrees with the counterhate argument. We present a corpus with 2,621 replies to counterhate arguments countering hateful tweets, and annotate them with fine-grained characteristics. We show that (a) half of the replies (51%) to the counterhate arguments disagree with the argument, and (b) this kind of reply often supports the hateful tweet (40%). We also analyze the language of counterhate arguments that elicit certain types of replies. Experimental results show that it is feasible to anticipate the kind of replies a counterhate argument will elicit.

heading: 1 Introduction
text: Hate messages and offensive language are commonplace in social media platforms. Twitter reported that more than 1.1 million accounts spread hateful content in the second half of 2020, a 77% increase with respect to the first half of the same year.1 In a recent survey of 10,093 adults in the U.S., 41% of participants reported online harassment on a personal level, and almost two-thirds of adults under the age of 30 reported experiencing internet harassment (Vogels, 2021). These figures, alongside other surveys,2,3 demonstrate the prevalence of hate speech on the internet. To address this problem, the European Commission partnered with popular social media platforms to announce a "Code of conduct on countering illegal hate speech online" (European Commission, 2019), which contains several commitments to prevent the spread of online hate speech across Europe.
1https://time.com/6080324/twitter-hate-speech-penalties/ 2https://legalresearch.elsa.org/library/ohs/ 3https://rm.coe.int/1680700016
The enormous amount of daily data makes these platforms rely on users who manually flag hateful content (Crawford and Gillespie, 2016). This approach requires spending millions of dollars yearly on manual hate speech verification and moderation (Seetharaman, 2018). An alternative is to automatically fight hate speech by using hate speech classifiers (Section 2). However, removing users’ content—as effective as it may be—restricts free speech. According to the Pew Research Center (Duggan, 2017), “Despite this broad concern over online harassment, 45% of Americans say it is more important to let people speak their minds freely online, and 53% feel that it is more important for people to feel welcome and safe online.”
A complementary strategy to address hateful content that does not interfere with free speech is to counter the hate with counterhate arguments in order to divert the discourse away from hate. Counterhate arguments can effectively fight and limit the spread of hate speech without removing or blocking any content (Gagliardone et al., 2015; Schieb and Preuss, 2016). Counterhate arguments usually are positive arguments that oppose hate speech with logic and facts. However well-intentioned, counterhate arguments may worsen the situation, as some people may respond with aggression if they feel threatened or targeted by the argument (Rains, 2013; Clayton et al., 2019).
Upon these motivations, we study the kind of replies counterhate arguments elicit. Specifically, we investigate replies to counterhate arguments beyond whether the reply agrees or disagrees with the counterhate argument. We consider Twitter threads consisting of (a) a hateful tweet, (b) a counterhate tweet countering (a), and (c) all replies to the counterhate tweet. We define a hateful tweet as any tweet that contains abusive language directed to individuals or groups of people. On the other hand, a counterhate tweet is a response tweet that explicitly or implicitly disagrees with the hateful
71
tweet. A reply is any response to the counterhate tweet. Consider the example in Figure 1. The hateful tweet contains hateful content towards a man (shown in a picture in the original tweet). The reply to the first counterhate tweet not only agrees with the counterhate tweet, but also includes additional counterhate arguments (e.g., he’s done a great job). Conversely, the reply to the second counterhate tweet not only disagrees with the counterhate tweet, but also includes an opinion supporting the hateful tweet (i.e., And you agree with letting convicted criminals run free). While the author of the second counterhate tweet may have had good intentions, the tweet elicited more hate and made the discourse undesirable. This paper presents a finegrained characterization of replies to counterhate tweets and opens the door to forecasting which counterhate tweets may elicit more hate instead of alleviating the spread of hate.
In summary, the main contributions of this paper are:4 (a) a corpus with 2,621 (hateful tweet, coun-
4https://github.com/albanyan/counterhate_reply
terhate tweet, reply) triples annotated with finegrained characteristics (whether the reply agrees with the counterhate tweet, supports the hateful tweet, attacks the author of the counterhate tweet, or adds additional counterhate); (b) linguistic analysis of the counterhate tweets depending on our finegrained characterization of the replies they elicit; (c) experimental results showing it is feasible to anticipate the kind of replies a counterhate tweet will elicit, and modest improvements using data augmentation and blending related datasets; and (d) qualitative analysis revealing when it is harder to perform any of the four classification tasks.

heading: 2 Previous Work
text: Recently, considerable literature has grown around identifying hateful content in user-generated content (Fortuna and Nunes, 2018). Existing research has created a variety of datasets to detect hate speech from several sources, including Twitter (Waseem and Hovy, 2016; Davidson et al., 2017), Reddit (Qian et al., 2019), Fox News (Gao and Huang, 2017), Yahoo! (Nobata et al., 2016; Djuric et al., 2015), and Gab (Mathew et al., 2021). Other studies have worked on identifying the target of hate, including whether the hateful content was directed toward a group, a person, or an object (Basile et al., 2019; Zampieri et al., 2019a; Ousidhoum et al., 2019). Another area of research aims to explore the role of context in hate and counterhate speech detection (Yu et al., 2022).
Previous efforts also detect and generate counterhate content. For counterhate detection, Garland et al. (2020) work with hateful and counterhate German tweets from two well-known groups. Mathew et al. (2020) collect and analyze pairs of hateful tweets and replies using the hate speech template I hate <group>, and detect whether a reply to a hateful tweet is a counterhate reply or not. In addition to analyzing or detecting counterhate replies, Albanyan and Blanco (2022) identify four fine-grained aspects of the relationship between a hateful tweet and a reply (e.g., whether the reply counters the hateful tweet with a justification). For counterhate generation, some studies have worked on collecting datasets with the help of crowd workers (Qian et al., 2019) or trained operators (Fanton et al., 2021; Chung et al., 2019).
There are several attempts to predict whether content will lead to additional hateful content. Zhang et al. (2018) identify whether a reply will
result in a personal attack. Liu et al. (2018) predict the number of hateful comments that an instgram post would receive. On the other hand, there are few efforts on investigating the impact of counterhate content, as stated in a recent survey by Alsagheer et al. (2022). Mathew et al. (2019) analyze YouTube comments and found that counterhate comments received more likes and interactions than non-counterhate comments. Other studies found that there is a positive association between counterhate efficiency and both its author’s ethnicity (Munger, 2017) and how immediate the response to the hateful content is posted (Schieb and Preuss, 2018). Finally, Garland et al. (2022) analyze hateful and counterhate German tweets and find that organized counterhate tweets elicit more counterhate replies and decrease the severity of the hate speech. Unlike these previous studies, we consider Twitter threads consisting of hateful tweets, a counterhate argument, and all replies to the counterhate argument. To our knowledge, we are the first to analyze the replies with fine-grained characteristics and tackle the problem of forecasting what kind of replies a counterhate arguments will elicit.

heading: 3 Dataset Collection and Annotation
text: We start our study by collecting triples consisting of hateful tweets, counterhate tweets, and replies to counterhate tweets. Then, we annotate the triples with our fine-grained characterization of the replies to the counterhate tweets. Unlike previous works (Section 2), our corpus enables us to (a) investigate whether counterhate tweets are successful at stopping the hate (Section 4), (b) analyze the language people use in counterhate tweets depending on the replies they elicit (Section 4), and (c) predict the kind of replies a counterhate tweet will elicit (Section 5).
Collecting Hateful Tweets, Counterhate Tweets, and Replies We use three strategies to collect a sufficient number of hateful tweets, counterhate tweets, and replies. The first strategy is to start with corpora consisting of (hateful tweet, counterhate tweet) pairs that include the tweet identifiers (Mathew et al., 2020; Albanyan and Blanco, 2022). Then, we use the Twitter API to collect all replies to the counterhate tweets. This strategy resulted in only 260 triples because some tweets are no longer available and not all counterhate tweets have replies. Note that other corpora not including identifiers cannot be used.
In the second strategy, we start collecting hateful tweets from corpora that only provide hateful tweets (Mathew et al., 2021; Chandra et al., 2021; He et al., 2021; Vidgen et al., 2020) including tweet identifiers. Then, we follow these steps:
1. Collect the replies to the hateful tweets. Let us consider them candidate counterhate tweets. 2. Select actual counterhate tweets from the candidates using an existing counterhate classifier (Albanyan and Blanco, 2022). 3. Collect the replies to the counterhate tweets to construct (hateful tweets, counterhate tweet, reply) triples. This strategy resulted in 230 triples. Since the total number of triples is relatively low (490 triples), we designed a third strategy.
The third strategy is the same than the second but with an alternative approach to collect the hateful tweets. Instead of using existing corpora, we use (a) the hate pattern I <hateful_verb> <target_group> defined by Silva et al. (2021) to select candidate hate tweets and (b) HateXPlain (Mathew et al., 2021) to select actual hate tweets. These strategy resulted in 3,820 triples.
The total number of triples after combining the three strategies is 4,310. We finalized the collection process by validating the triples. The final size of our corpus after the validation process is 2,621 (hateful tweet, counterhate tweet, reply) triples. The total number of hateful tweets is 1,147, while the number of counterhate tweets is 1,685. The number of counterhate tweets per hateful tweet ranges between 1 and 20, while the number of replies per counterhate tweet ranges between 1 and 88.
Annotation Guidelines Along with determining whether a reply agrees with the counterhate tweet, we identify finer-grained characteristics of the replies. Accordingly, we define three steps to answer four questions in the annotation process.
The first step is determining whether the reply agrees with the counterhate tweet. We consider that a reply agrees if it does not oppose the counterhate tweet either explicitly or implicitly. On the other hand, we consider that a reply disagrees if it opposes the counterhate tweet, including sarcasm (e.g., you are missing something!) or casting doubt (e.g., are you kidding?).
The second step provides fine-grained characteristics when the reply disagrees with the counterhate tweet. First, we ask whether the reply supports
the hateful tweet. We consider the reply to support the hateful tweet if it includes a justification for the hateful content (e.g., the news says the opposite!) or introduces additional hateful content (e.g., first example in Table 1). Second, we identify whether the reply attacks the author of the counterhate tweet. We include in the definition of attack any mockery or insults towards the author of the counterhate tweet (e.g., stupid never understand!).
The third step provides fine-grained characteristics when the reply agrees with the counterhate tweet. Finally, when the reply agrees with the counterhate tweet, we distinguish whether the reply includes additional counterhate. Namely, we identify whether the reply contains additional counterhate by providing a new opinion or factual argument to support the counterhate tweet (e.g., he is also known for his charitable work and donations). Only agreeing with the counterhate tweet (e.g., you are correct!) does not contain additional arguments.
Examples Table 1 shows examples from our corpus. In the first example, the reply not only disagrees with the counterhate tweet but also supports the hateful tweet with new hate content against the mentioned people. Note that replies can also show disagreement without including any support for the hateful tweet (e.g., do you have any evidence?!!).
In the second example, the reply attacks the author of the counterhate tweet without including any justification or support for the hateful tweet. This also indicates that the reply disagrees with the counterhate tweet. Note that replies can disagree with the counterhate tweet without attacking the author (e.g., don’t be their lawyer).
Finally, the reply in the third example not only agrees with the counterhate tweet, but also includes additional counterhate (honest vs. successful government). Note that replies can agree with the counterhate tweet without adding additional counterhate (e.g., convincing response!).
Annotation Process and Inter-Annotator Agreements We used the Label Studio annotation tool.5 The tool showed the hateful tweet, counterhate tweet, and reply. It displayed the screenshots of the tweets taken from the Twitter website to prevent
5https://github.com/heartexlabs/label-studio
readability issues when displaying the tweets (e.g., special characters). Additionally, annotators are provided with instructions for each question (i.e., definitions and examples).
The 2,621 (hateful tweet, counterhate tweet, reply) triples were independently annotated by two graduate students active on social media platforms. We are interested in how regular social media users interpret hateful tweets, counterhate tweets, and replies. Table 2 presents the inter-annotator agreements. For all questions, the observed agreements are almost 90%. Cohen’s k coefficients indicate (a) substantial agreement in two questions: whether the reply supports the hateful tweet and attacks the author of the counterhate tweets, and (b) nearly perfect agreements in two questions: whether the reply agrees with the counterhate tweet and includes additional counterhate. k coefficients between 0.60 and 0.80 are considered substantial agreement, and above 0.80 are considered nearly perfect (Artstein and Poesio, 2008). We note that it is easier to determine whether a reply agrees and adds additional counterhate tasks than supports and attacks the author tasks. This is due to the use of sarcasm and nuanced language when the reply supports the hateful tweet or attacks the author of the counterhate tweet. After the two annotators finished all the annotations independently, they debated the points of disagreement and decided on the final label.

heading: 4 Corpus Analysis
text: Label Distribution Table 3 presents the percentages of yes and no labels per question. Around half of the replies to the counterhate tweets do not agree with the counterhate tweet (51%), and it is common for them to support the hateful tweet when they do not agree (40%). In addition, it is somewhat rare for these replies to attack the author of the counterhate tweet when they disagree (24%). On the other hand, it is less likely for the replies to include additional counterhate arguments when they agree (35%). This shows that most replies that agree with the counterhate tweet do not include any additional
arguments to support the counterhate tweet (e.g., you are correct).
Linguistic Insights We analyze the language people use in the counterhate tweets that lead to certain types of replies. We count the number of tokens, pronouns, and proper nouns using spaCy (Neumann et al., 2019). We use the lexicons of offensive words6 and lexicons by Mohammad and Turney (2013) to count offensive, positive, negative, and sadness words. Finally, we use Profanity-check7 to calculate the profanity score and TextBlob8 to calculate the subjectivity score. All correlations between linguistic features are below 0.30, except for a few that involve the number of tokens (Appendix A, Figures 2–5). We check the predictive power of the selected features using t-test. We also report if a test passes the Bonferroni correction (Table 4). The p-values reveal several interesting insights:
• Counterhate tweets with more tokens or pronouns elicit replies that do not attack the author of the counterhate tweet or include additional counterhate if they agree. • Counterhate tweets with more question marks lead to replies that (a) agree with the counterhate tweets and do not add additional counterhate, or (b) support the hateful tweet and do not attack the author. • We find that (a) positive words elicit replies that do not attack the author or add additional counterhate, (b) negative words elicit replies that do not add additional counterhate, and (c) offensive words elicit replies that agree with the counterhate, or attack the author. • Profanity in counterhate tweets elicits replies that agree with the counterhate tweet or do not support the hateful tweet. • Comparing hateful tweets and counterhate tweets reveals that counterhate tweets with (a) less offensive content lead to replies that agree with the counterhate tweet or do not support the hateful tweet, (b) less sadness words elicit replies that agree with the counterhate or do not attack the author of the counterhate tweet, and (c) less subjectivity lead to replies that attack the author of the counterhate or do not add additional counterhate.
6https://github.com/zacanger/profane-words 7https://github.com/vzhou842/profanity-check 8https://github.com/sloria/TextBlob

heading: 5 Experiments and Results
text: We create a binary classifier for each task, namely, whether the reply: (a) agrees with the counterhate tweet, (b) supports the hateful tweet, (c) attacks the author of the counterhate tweet, or (d) includes additional counterhate arguments. We split the dataset into 70:10:20 ratios for training, validation, and testing. Each instance consists of a hateful tweet, a counterhate tweet, and a reply.
Baselines The baseline models we use in our experiments are the majority and random models. In the majority model, the majority label is predicted (no label for all tasks, Table 3). In the random model, a random label of no or yes is predicted.
Neural Network Architecture and Training In all experiments, we used the transformer-based BERTweet model (Nguyen et al., 2020). BERTweet is a BERT-based (Devlin et al., 2019) model but was pre-trained using the RoBERTa training strategy (Liu et al., 2019) on 850M English tweets. The neural architecture consists of the base architecture of BERTweet followed by a linear layer with 128 neurons and ReLU activation. Then, we added a final linear layer with 2 neurons and a Softmax activation to do the binary classification between labels yes and no. We perform the experiments using different textual inputs:
1. the hateful tweet alone,
2. the counterhate tweet alone, 3. the reply alone, and 4. combinations of (1–3) above.
We use the ’
’ special token to concatenate the inputs. Then, we apply three strategies to enhance the performance of neural models:
Data Augmentation We adapt Easy Data Augmentation Marivate and Sefara (2020) called. Specifically, we use Synonym Replacement (randomly replacing a word), Random Insertion (inserting a synonym of a random word), and Random Swap (randomly swapping the positions of two words).
Concatenating Language Features Language features have been shown to improve pre-trained models’ performance in text classification tasks (Lim and Tayyar Madabushi, 2020). To this end, we experiment with complementing embeddings with manually defined language features. Inspired by the analyses in Section 4, we calculate count-based language features for the replies, such as the number of tokens, pronouns, nouns, verbs, negative and positive words (using the lexicons by Mohammad and Turney (2013)), question marks, proper nouns, and first-person pronouns. Examples are shown in Appendix C (Table 7). We then use the significance test (t-test) to keep the significant features (p< 0.05). The common significant features between the tasks are the number of tokens, bad words, nouns and verbs, and positive words. We concatenate these
features with each other and with the input embeddings using the ’
’ special token. Blending Complementary Corpora We finally investigate pretraining with complementary tasks. We adopt the method by Shnarch et al. (2018), which integrates labeled data from related tasks with various ratios in each training epoch. This is done by blending the related task instances with our dataset for training, and decrease the ratio in each epoch to reach zero in the last one. The corpora we blend with are: (a) a stance dataset (Mohammad et al., 2016) consisting of 4,163 tweets about abortion, atheism, climate change, feminism, and Hillary Clinton annotated with in favor, against, or none; (b) an offensive dataset (Zampieri et al., 2019b) containing over 14K tweets annotated with offensive or not offensive, and (c) a hateful tweetreply dataset (Albanyan and Blanco, 2022), annotated with whether the reply counters the hateful tweet (5,652 pairs), counters the hate with justification (1,145), attacks the author of the hateful tweet (1,145), and includes additional hate (4,507).

heading: 5.1 Quantitative Results
text: Table 5 shows the results using the F1-score for no and yes labels, and the weighted average. Appendix D (Tables 8–11) contains detailed results showing the precision, recall, and F1-score. The
F1-scores for the majority baseline are 0.34, 0.45, 0.66, and 0.51.
The results using the neural models with different inputs (the hateful tweet, the counterhate tweet, the reply, or a combination of different tweets) reveal several insights:
• Using only the hateful tweet or counterhate tweet as an input outperforms the baselines, showing that certain hateful tweets or counterhate tweets elicit particular kinds of replies. • Feeding to the network only the reply yields the best results out of all single-tweet inputs. • Combining the reply with the hateful tweet outperforms the models in support the hateful tweet task since, in this task, the reply is related to the hateful tweet. On the other hand, including the counterhate tweets improves the results in the other three tasks. We note that it barely affects the attacks the author task. We hypothesize this is because the attack can be detected from the reply alone. • Including a third input (either the counterhate tweet or hateful tweet) to the best pairs (reply+counterhate tweet or reply+hateful tweet) worsens the results (0.73, 0.78, 0.83, and 0.85 vs. 0.73, 0.75, 0.81, and 0.83).
Additionally, the results show modest improvements when applying the three strategies we work
with (Data Augmentation, Language Features, and Blending Complementary Corpora). We find that:
• Data augmentation benefits the neural network trained with the best input combination in two tasks: agree with the counterhate tweet and support the hateful tweet. • Language features are barely beneficial. • Blending complementary corpora always
yields higher results. More details about the related datasets that lead to the best results in all tasks can be found in Appendix D. We also tried combining the strategies and found out that doing so does not improve results.
When do the best models make errors? While our best models in each task produce strong results (best input + blending, Table 5), we manually analyzed the wrong predictions made by each model. Table 6 shows the error types we found. We started the analysis by randomly selecting 100 samples from the model produced in the agree task. We considered all the wrong predictions for the other three tasks since they were less than 100 samples. They were 59 samples in the support task, 46 in the attacks the author task, and 43 in the additional counterhate task. The error types are:
• Intricate text (24%, 25%, 22%, and 24%), which involves using sarcasm and implicit meaning, or mentioning many individuals or entities (e.g., Reply: don’t block me I need you so bad. Agree? Gold: No, Predicted: Yes). • General knowledge (16%, 19%, 17%, and 12%), which requires world knowledge and commonsense to understand the meaning of the tweet (e.g., Reply: it’s on sky news mate!. Supports? Gold: Yes, Predicted: No). • Short text (20%, 12%, 21%, and 14%), tweets with less than 5 tokens (e.g., Reply: chill out. Attack the Author? Gold: No, Predicted: Yes).
• Misspellings and abbreviations (11%, 9%, 11%, and 12%), (e.g., Reply: @auscoups Why r they trending these things. Addit. counterhate? Gold: Yes, Predicted: No). • Rhetorical question (8%, 14%, 9%, and 9%), where a question in a tweet is asked to deliver a point (e.g., Reply: you think this is funny?. Agree? Gold: Yes, Predicted: No).

heading: 6 Conclusions
text: Countering hateful content is an effective way to fight hate speech (Gagliardone et al., 2015). Additionally, countering hate speech—unlike blocking— does not interfere with free speech. However wellintentioned, however, counterhate arguments may worsen the situation by eliciting additional hate.
In this work, we analyze the discourse following a counterhate tweet. Specifically, we analyze all replies to counterhate tweets and reveal finegrained characteristics beyond whether the reply agrees with the counterhate argument. Namely, we determine whether the reply (a) not only disagrees with the counterhate tweet but also supports the hateful tweet or attacks the author of the counterhate arguments, or (b) not only agrees with the counterhate tweet but also adds additional counterhate arguments. To our knowledge, this work is the first to analyze the language of counterhate tweets based on the replies they elicit.
The work presented here is empirical and explores genuine counterhate arguments and the replies they elicit. We believe that it is critical to analyze genuine social media discourse and how hate spreads (and does not spread). We avoid making any causal claims; instead, we draw insights from genuine social media discourse around hateful content. Our future work includes generating counterhate arguments (a) customized to specific
hateful content and (b) following the characteristics we found to be more effective at stopping hatred. We hypothesize that doing so will be more effective than generic or even expert-driven counterhate.
Limitations
In the data collection process (Section 3), we collect (hateful tweet, counterhate tweet, and reply) triples from existing hateful tweet-reply and hateful tweet corpora (the first and second strategies). However, this ends with fewer triples since some tweets are no longer available and not all counterhate tweets have replies. In addition, we use hate speech and counterhate classifiers to discard non-hateful and non-counterhate tweets. This step might (a) discard actual hateful or counterhate tweets that are detected wrongly and (b) keep hateful or counterhate tweets that should be discarded. Another limitation is that we only consider the tweet text. However, some tweets contain text accompanied by images or sometimes images only. Including the tweets’ images in the analysis may add more insights.

heading: C Language Features
text: Table 7 presents examples of applying the language feature strategy on the replies (Section 5). We experiment with concatenating language features presented in the table with input embeddings. The selected language features are number of tokens, pronouns, nouns and verbs, negative and positive words, question marks, proper nouns, and firstperson pronouns.

heading: D Detailed Results
text: Tables 8–11 show the detailed results presented in Table 5. We provide Precision, Recall and F1-score (a) using different tweet combinations and (b) applying the three strategies to enhance the results. In addition, we show the results of each related dataset used in the Blending with Complementary Tasks strategy. The related datasets that lead to the best results in each task are:
• stance dataset to determine whether the reply agrees with counterhate tweet; • hateful tweet-reply pair dataset regarding if a reply includes additional hate, to determine whether the reply supports the hateful tweet task; • hateful tweet-reply pair dataset regarding if a reply attacks the author of the hateful tweet, to determine whether the reply attacks the author of the counterhate tweet; and • hateful tweet-reply pair dataset regarding if a reply counters the hate with justification, to determine whether the reply adds additional counterhate.

heading: Future Work
text: Our future work includes generating counterhate arguments (a) customized to specific
hateful content and (b) following the characteristics we found to be more effective at stopping hatred. We hypothesize that doing so will be more effective than generic or even expert-driven counterhate. Limitations
In the data collection process (Section 3), we collect (hateful tweet, counterhate tweet, and reply) triples from existing hateful tweet-reply and hateful tweet corpora (the first and second strategies). However, this ends with fewer triples since some tweets are no longer available and not all counterhate tweets have replies. In addition, we use hate speech and counterhate classifiers to discard non-hateful and non-counterhate tweets. This step might (a) discard actual hateful or counterhate tweets that are detected wrongly and (b) keep hateful or counterhate tweets that should be discarded. Another limitation is that we only consider the tweet text. However, some tweets contain text accompanied by images or sometimes images only. Including the tweets’ images in the analysis may add more insights