SlowGuess/NAACL_2022_OCR huggingface dataset:
Genevieve Gorrell, Mehmet E Bakir, Ian Roberts, Mark A Greenwood, Benedetta Iavarone, and Kalina Bontcheva. 2019. Partisanship, propaganda and posttruth politics: Quantifying impact in online debate. arXiv preprint arXiv:1902.01752.
Binxuan Huang and Kathleen M Carley. 2019. Syntax-aware aspect level sentiment classification with graph attention networks. arXiv preprint arXiv:1909.02606.
Taichi Ishiwatari, Yuki Yasuda, Taro Miyazaki, and Jun Goto. 2020. Relation-aware graph attention networks with relational position encodings for emotion recognition in conversations. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 7360-7370.
Andrew H Johnston and Gary M Weiss. 2017. Identifying Sunni extremist propaganda with deep learning. In 2017 IEEE Symposium Series on Computational Intelligence (SSCI), pages 1-6. IEEE.
Thomas N Kipf and Max Welling. 2016. Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907.
Lingpeng Kong, Nathan Schneider, Swabha Swayamdipta, Archna Bhatia, Chris Dyer, and Noah A Smith. 2014. A dependency parser for tweets. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1001-1012.
Chuanrong Li, Lin Shengshuo, Leo Z Liu, Xinyi Wu, Xuhui Zhou, and Shane Steinert-Threlkeld. 2020. Linguistically-informed transformations (lit): A method for automatically generating contrast sets. arXiv preprint arXiv:2010.08580.
Jianshu Li, Jian Zhao, Fang Zhao, Hao Liu, Jing Li, Shengmei Shen, Jiashi Feng, and Terence Sim. 2016. Robust face recognition with deep multi-view representation learning. In Proceedings of the 24th ACM international conference on Multimedia, pages 1068-1072.
Jinfen Li, Zhihao Ye, and Lu Xiao. 2019. Detection of propaganda using logistic regression. In Proceedings of the Second Workshop on Natural Language Processing for Internet Freedom: Censorship, Disinformation, and Propaganda, pages 119-124.
Yang Liu and Mirella Lapata. 2019. Hierarchical transformers for multi-document summarization. arXiv preprint arXiv:1905.13164.
Yijia Liu, Yi Zhu, Wanxiang Che, Bing Qin, Nathan Schneider, and Noah A Smith. 2018. Parsing tweets into universal dependencies. arXiv preprint arXiv:1804.08228.
Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692.
Harish Tayyar Madabushi, Elena Kochkina, and Michael Castelle. 2020. Cost-sensitive bert for generalisable sentence classification with imbalanced data. arXiv preprint arXiv:2003.11563.
Daniel Taninecz Miller. 2019. Topics and emotions in russian twitter propaganda. First Monday.
Leonardo Nizzoli, Marco Avvenuti, Stefano Cresci, and Maurizio Tesconi. 2019. Extremist propaganda tweet classification with deep learning in realistic scenarios. In Proceedings of the 10th ACM Conference on Web Science, pages 203-204.
Hannah Rashkin, Eunsol Choi, Jin Yea Jang, Svitlana Volkova, and Yejin Choi. 2017a. Truth of varying shades: Analyzing language in fake news and political fact-checking. In Proceedings of the 2017 conference on empirical methods in natural language processing, pages 2931-2937.
Hannah Rashkin, Eunsol Choi, Jin Yea Jang, Svitlana Volkova, and Yejin Choi. 2017b. Truth of varying shades: Analyzing language in fake news and political fact-checking. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 2931-2937, Copenhagen, Denmark. Association for Computational Linguistics.
Franco Scarselli, Marco Gori, Ah Chung Tsoi, Markus Hagenbuchner, and Gabriele Monfardini. 2008. The graph neural network model. IEEE transactions on neural networks, 20(1):61-80.
Michael Schlichtkrull, Thomas N Kipf, Peter Bloem, Rianne Van Den Berg, Ivan Titov, and Max Welling. 2018. Modeling relational data with graph convolutional networks. In European semantic web conference, pages 593-607. Springer.
Linfeng Song, Zhiguo Wang, Mo Yu, Yue Zhang, Radu Florian, and Daniel Gildea. 2018. Exploring graph-structured passage representation for multi-hop reading comprehension with graph neural networks. arXiv preprint arXiv:1809.02040.
Hang Su, Subhransu Maji, Evangelos Kalogerakis, and Erik Learned-Miller. 2015. Multi-view convolutional neural networks for 3d shape recognition. In Proceedings of the IEEE international conference on computer vision, pages 945-953.
Hao Su, Min Sun, Li Fei-Fei, and Silvio Savarese. 2009. Learning a dense multi-view representation for detection, viewpoint classification and synthesis of object categories. In 2009 IEEE 12th International Conference on Computer Vision, pages 213-220. IEEE.
Yao-Hung Hubert Tsai, Shaojie Bai, Paul Pu Liang, J Zico Kolter, Louis-Philippe Morency, and Ruslan Salakhutdinov. 2019. Multimodal transformer for unaligned multimodal language sequences. In Proceedings of the conference. Association for Computational Linguistics. Meeting, volume 2019, page 6558. NIH Public Access.
Ming Tu, Guangtao Wang, Jing Huang, Yun Tang, Xiaodong He, and Bowen Zhou. 2019. Multi-hop reading comprehension across multiple documents by reasoning over heterogeneous graphs. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 2704-2713, Florence, Italy. Association for Computational Linguistics.
Petar Velicković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. 2017. Graph attention networks. arXiv preprint arXiv:1710.10903.
Kai Wang, Weizhou Shen, Yunyi Yang, Xiaojun Quan, and Rui Wang. 2020a. Relational graph attention network for aspect-based sentiment analysis. arXiv preprint arXiv:2004.12362.
Liqiang Wang, Xiaoyu Shen, Gerard de Melo, and Gerhard Weikum. 2020b. Cross-domain learning for classifying propaganda in online contents. arXiv preprint arXiv:2011.06844.
Ruize Wang, Duyu Tang, Nan Duan, Zhongyu Wei, Xuanjing Huang, Guihong Cao, Daxin Jiang, Ming Zhou, et al. 2020c. K-adapter: Infusing knowledge into pre-trained models with adapters. arXiv preprint arXiv:2002.01808.
Ruize Wang, Duyu Tang, Nan Duan, Wanjun Zhong, Zhongyu Wei, Xuanjing Huang, Daxin Jiang, and Ming Zhou. 2020d. Leveraging declarative knowledge in text and first-order logic for fine-grained propaganda detection. arXiv preprint arXiv:2004.14201.
William Williamson III and James Scrofani. 2019. Trends in detection and characterization of propaganda bots. In Proceedings of the 52nd Hawaii International Conference on System Sciences.
Jaewon Yang and Jure Leskovec. 2011. Patterns of temporal variation in online media. In Proceedings of the fourth ACM international conference on Web search and data mining, pages 177-186.
Rowan Zellers, Ari Holtzman, Hannah Rashkin, Yonatan Bisk, Ali Farhadi, Franziska Roesner, and Yejin Choi. 2019. Defending against neural fake news. arXiv preprint arXiv:1905.12616.
Xingxing Zhang, Furu Wei, and Ming Zhou. 2019. Hibert: Document level pre-training of hierarchical bidirectional transformers for document summarization. arXiv preprint arXiv:1905.06566.
Xuanyu Zhang. 2020. Cfgnn: Cross flow graph neural networks for question answering on complex tables. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 9596-9603.
Yuhao Zhang, Peng Qi, and Christopher D Manning. 2018. Graph convolution over pruned dependency trees improves relation extraction. arXiv preprint arXiv:1809.10185.
A Experiment Platform
All the experiments were conducted on a Ubuntu 20.04 system, with  $2.10\mathrm{GHz}$  Intel(R) Xeon(R) CPU and 8-core NVIDIA GeForce GTX 1080Ti/11GB. Our models were implemented using Pytorch 1.4 with CUDA 10.1.
B Additional Information and Examples
Table B1 shows the statistics of the TweetSpin dataset. Table B2 lists the common phrases related to different propaganda techniques used in our data collection process. Table B3 shows sample tweets that were misclassified by our model variants in comparison to our full MV-PROP model. Finally, Figure B1 illustrates the speaker dependency graph related to a sample tweet thread.
Dataset Statistics
#Total Propaganda Tweets	157,327
#Total Non-Propaganda Tweets	53,165
% of tweets with discussion context	59.06
Avg. discussion context length	3.15
Avg. #users in discussion context	2.26
Propaganda technique	# Tweets
Loaded Language	18,365
Name Calling/Labeling	17,096
Reductio Ad Hitelerium	15,677
Doubt	14,993
Appeal To Fear/Prejudice	14,654
Whataboutism	13,887
Repetition	13,285
Slogans	10,190
Appeal To Authority	8,539
Flag-Waving	7,675
Exaggeration, Minimization	5,416
Black-And-White Fallacy	4,872
Thought-terminating cliches	3,781
Bandwagon	2,547
Red Herring	2,315
Causal oversimplification	1,790
Straw man	1,265
O, I, C	1,048
Table B1: TweETSPIN Dataset statistics. O, I, C refers to "Obfuscation, Intentional Vagueness, Confusion".
Propaganda Techniques	Common Phrases
Loaded Language	List of words/phrases: https://examples.yourdictionary.com/loaded-language-examples.html
Name Calling	"Commie", "Fascist", "Pig", "Yuppie", "Libtard", "Extremist", "Terrorist", "Snowflake", "Cuck"
Appeal to Authority	"Experts have warned", "Experts say ... ", "As an expert in ... ", "As a[n] [occupation*], I can say ..., "[PERSON] advises/ urges/ suggests" *https://learnersdictionary.com/3000-words/topic/job-professions
Doubt	"Lied to us", "Lying to us", "covering up", "cover up", "not being told the truth", "not adding up", "official story", "fake story"
Bandwagon	"Almost all/ Most/ Majority of [Nation or ethnic groups]*" *https://en.wikipedia.org/wiki/Lists_of_people_by_nationality
Flag Waving	"[Nation/State] first", "Nation" + [Positive Word*], "Anti-[Nation/State]", "True patriots/nationalist" *https://ptrckprry.com/course/ssf/data/positive-words.txt
Reductio Ad Hitlerum	"Hitler, Stalin", "[PERSON] is a communist/marxist/nazi/fascist", "[ORG] are communists/marxists/nazis/fascists"
Black & White Fallacy	"No other way...", "No alternative to ..., "No other option ..., "no better way"
Whataboutism	"The media ignores", "Nobody talks/mentions/speaks about ..., "But What about", "Don't focus on ..., but ..."
Table B2: List of common phrases related to different propaganda techniques used in the data collection. Note that some of the language used here is potentially offensive.
Tweet	True Label	Model	Predicted Label
Context: A discussion about AR15 Guns for hunting And no one needs a BMW o a$1000 suit or a Rolex instead of a Timex. It's what one wants, not needs. Many people want an AR15...You remind me of an unselected bureaucrat in a position of power,...	Straw man	MV-PROP-S	[Loaded language, Whatabutism]
MV-PROP	[Straw man, Whatabutism]
Context: A discussion about student loans You're right. Being able to sign away your life & go to war & die is definitely less that debt.	Red Herring	MV-PROP - R	[Straw man, Exaggera-tion]
MV-PROP	[Red Her-ring, Straw man]
Context: A discussion about Presidential Elections 2020 This is 'Democracy', Venezuela-style. Or Cuba. Or China. Or the Soviet Union. Or a certain Central European country in the 1930's.	Reductio Ad Hitlerum	MV-PROP - K	[Non Propa-ganda, Red Herring]
MV-PROP	[Reductio Ad Hitlerum, Straw man]
Table B3: Sample tweets which were misclassified by our model variants in comparison to our full MV-PROP model. We report the top 2 ranked predictions.
Figure B1: (a) Sample tweet thread where speaker dependencies can determine the propaganda technique; (b) Speaker-dependent structural relationship for the discussion thread in (a).