SlowGuess/NAACL_2022_OCR huggingface dataset:
Interestingly, UCDCC achieves the best precision of  $78.8\%$  and accuracy of  $79.7\%$  on Tweets dataset. Besides, THU-NGN gets the best recall at  $80.1\%$  on Tweets. This is because UCDCC designs targeted rules to preprocess the input text and it achieves the best performance on SemEval 2018 Task 3 Subtask A. Rules could improve precision effectively, but they are hard to take recall into account at the same time. So the  $F1$  is not good enough. The last place performance of UCDCC on IAC-V1/V2 also supports this point. These designed rules are hard to fit missing instances and other domains. Similarly, THU-NGN uses linguistic knowledge such as sentiment and syntactic, so it
$^{6}$ https://pytorch.org
<sup>7</sup>We employ spaCy toolkit to derive dependency tree.
Table 3: The precision, recall, and macro  $F1$  of models including BERT, DC-Net with BERT as Encoder, and DC-Net with Bi-LSTM as Encoder.
Model	Tweets
Pre.	Rec.	F1	Acc.
BERT	69.1	67.6	68.1	71.6
DC-Net (w/ BERT)	70.2	70.7	70.4	71.3
DC-Net (w/ Bi-LSTM)	76.4	77.5	76.3	76.7
achieves the highest recall on Tweets but it cannot perform equally well on other datasets. That is, rules have limitations in handling this task.
The previous state-of-the-art ADGCN achieves second-best on IAC-V2 and Tweets. However, on IAC-V1 dataset, ADGCN performs not as well as the result reported in their paper. IAC-V1 dataset is relatively small so the train-valid/test split has a significant impact. Our experiments also show that MIARN's performance is not as good as expected. This indicates that the basic utilization of word pair correlation is not enough to improve the performance of sarcasm detection. Bi-LSTM, AT-LSTM, and CNN-LSTM-DNN methods are all based on LSTM. Thus the performances of these models on Tweets and IAC-V1 are close.
4.4 Comparison with BERT
BERT has contributed to significant improvements on various NLP tasks. To do a comprehensive comparison, we apply the dual-channel framework to BERT (Devlin et al., 2019) model by using BERT as the encoder. The new model with BERT is named DC-Net (w/ BERT). Table 3 reports the experimental results.
As expected, the DC-Net (w/ BERT) model achieves significant improvement compared with the basic BERT. This result shows that our dual-channel framework is adaptable and effective. Interestingly, we observe that BERT-based methods perform not well enough compared with its huge improvement on other NLP tasks. This can be attributed to the fact that the corpus of pre-trained BERT contains more deterministic data (e.g., only one meaning without sentiment conflict). However, sarcasm is a niche linguistic phenomenon. The poor performance of BERT further reinforces that sarcasm recognition is a difficult task. It tells us that applying well-performed classification methods directly is difficult to achieve desirable performance.
Table 4: Ablation study on Tweets dataset.  $J_{s}$  denotes using sarcasm loss only.  $J_{s} + J_{d}$  means using sarcasm and implied loss.  $J_{s} + J_{l}$  means using sarcasm and literal loss.  $J_{s} + J_{l} + J_{d}$  denotes using sarcasm, literal, and implied loss.
Objective	Tweets
Pre.	Rec.	F1	Acc.
Js	74.6	75.4	74.8	75.4
Js+Jd	74.2	75.2	74.0	74.4
Js+Jl	73.0	74.0	72.8	73.1
Js+Jl+Jd	76.4	77.5	76.3	76.7
Recall that the model training (see Section 3) contains three objectives: sarcasm recognition, literal sentiment classification, and implied sentiment classification. To study the effect of the three objectives, we conduct ablation study on Tweets dataset.
Table 4 lists the result of ablation study. As expected, the model with both literal and implied losses performs the best. Interestingly, the model using sarcasm recognition loss with single channel loss (i.e., literal and implied) performs worse than the model using only sarcasm recognition loss. This is because adding literal and implied sentiment classification objectives interferes with the judgment of the model. By adding both literal and implied sentiment classification losses, the model's performance improves 1.5 points in  $F1$  score. This is very important because it reveals that the dual channels are effective. There is no effect or the opposite effect when single channel is applied alone. However, once dual-channel is used, the performance improves largely. It reveals that the dual channels complement each other. Conflict detection could recognize sarcasm when both of them are considered.
4.6 Effectiveness of DC-Net by Visualization
To verify the rationality and effectiveness of our proposed DC-Net, we adopt t-SNE (Van der Maaten and Hinton, 2008) to visualize high-dimensional vector representations based on the test sub-dataset of IAC-V2 (with largest data).
To figure out the effect of each channel, we visualize the representations of the literal channel and the implied channel. Figure 3(a) shows the visualization of literal representation  $v_{L}^{\prime}$  and implied representation  $v_{D}^{\prime}$ . Recall that the decomposer module splits the original text into sentiment words and the
(a)Literal and implied reps.of each channel
Figure 3: Results of t-SNE visualization
(b) Sarc. and non-sarc. reps. in analyzer module
remaining. We observe that there is a clear separation between literal and implied representations from Figure 3(a). This strongly indicates that our dual-channel framework is capable of effectively separating the representations of the two channels.
To get into the essence of sarcastic and non-sarcastic texts, we visualize the sarcastic and non-sarcastic representations. Figure 3(b) shows the sarcastic and non-sarcastic representations in the analyzer module. We observe that non-sarcastic texts focus on the upper right corner, while sarcastic texts scatter on the lower left corner. It reveals that the sarcasm patterns are complex and changeable. Nevertheless, the dividing line between the two is relatively clear. To this end, explicitly separating the literal and implied channels is necessary and effective. Further, DC-Net makes a distinct difference between sarcastic and non-sarcastic representations, which greatly promotes the performance of the dual-channel framework. We also plot the sarcastic representation and non-sarcastic representation of each channel respectively, which show the same trend as Figure 3(b). So they are not detailed here.
4.7 Flexibility of Dual-Channel Framework
Flexibility of encoder. The dual-channel framework is flexible and generic, and can be realized by plugging in existing sarcasm recognition models, e.g., MIARN, or classification models, e.g., ATLSTM, Bi-LSTM, and BERT. Therefore, we use these methods as the encoder to examine the flexibility of our proposed framework. The changing range on macro  $F1$  from original baseline models to dual-channel models is shown in Table 5.
As expected, the performance of baseline models has different degrees of improvement on all datasets after applying dual-channel framework. For relatively simple models such as MIARN and
Table 5: The macro  $F1$  changes from basic models to dual-channel based models.
Basic Model	Changing Range on F1
IAC-V1	IAC-V2	Tweets
AT-LSTM	↑0.4	↑1.1	↑1.5
BERT	↑0.4	↑1.7	↑2.3
MIARN	↑1.1	↑2.8	↑4.8
Bi-LSTM	↑1.8	↑2.4	↑4.6
Table 6: Comparisons of different analyzer methods.
Analyzer	F1
IAC-V1	IAC-V2	Tweets
Subtraction	65.1	80.7	75.2
Concatenation	66.4	82.1	76.3
Bi-LSTM, the improvement could be up to  $4.8\%$ . Interestingly, for complex models like BERT, the improvement is up to  $2.3\%$ . As we mentioned earlier, the basic BERT performs not good enough because sarcasm is a niche language phenomenon and the training dataset of BERT contains few sarcasm texts. After applying the dual-channel framework to BERT, the performance for sarcasm recognition improves a lot. These indicate that our designed framework is able to fit various encoders with a significant improvement.
Flexibility of analyzer. The analyzer module is used to measure the difference between the literal channel and the implied channel. As we described in Section 3.4, other analyzer methods such as concatenation and subtraction could be applicable. To this end, we compare different analyzer methods. Table 6 shows the results. We observe that concatenation performs better than subtraction on all datasets. It is because concatenation holds more useful information and DC-Net could compare the difference between the two input representations. However, subtraction only outputs the margin between the two representations. It loses the original values which also contain useful information.
In this study, we argue that the essential characteristic of sarcastic text is the conflict between literal and implied sentiments in the same sentence. To this end, we propose a dual-channel framework to recognize sarcasm by decomposing the input text
into the literal channel and the implied channel. Based on this framework, we develop DC-Net. DC-Net is capable of exploiting the literal sentiment by encoding the sentiment words of input text, and exploiting the implied sentiment by encoding the remaining text. Experiments show that the proposed DC-Net achieves state-of-the-art performance.
Sarcasm as a complex linguistic phenomenon has various patterns, e.g., text with word/phrase pair sentiment conflict. Nevertheless, sentiment conflicts are common in sarcasm texts. In this paper, we make the very first attempt to recognize sarcasm by detecting sentiment conflict. More importantly, our proposed dual-channel framework could be further developed to detect more sentiment conflict patterns. For now, we use sentiment words as a static decomposer. This intuitive method can cover common sarcasm patterns but not all. Therefore, how to minimize the dependence on sentiment words is an important research direction.
Another limitation is that we assume that sentiment polarity is decided by the sentiment lexicon approximately in the analyzer module. While the assumption is widely accepted, there is still a gap between approximate label and groundtruth. In the current design, we adopt a soft weighting mechanism to detect sentiment conflict between the two channels. We expect that the model could output the opposite sentiment labels directly, which is a more effective way to express conflict.
This work is supported by the National Science Foundation of China (NSFC No. 62106249, No. 61902382 and No. 61972381), the Youth Innovation Promotion Association CAS under Grants No. 20144310, the Lenovo-CAS Joint Lab Youth Scientist Project, and the Foundation and Frontier Research Key Program of Chongqing Science and Technology Commission (No. cstc2017jcyjBX0059).
Nastaran Babanejad, Heidar Davoudi, Aijun An, and Manos Papagelis. 2020. Affective and contextual embedding for sarcasm detection. In Proceedings of the 28th International Conference on Computational Linguistics, COLING 2020, Barcelona, Spain (Online), December 8-13, 2020, pages 225-243.
Erik Cambria, Yang Li, Frank Z. Xing, Soujanya Poria, and Kenneth Kwok. 2020. Senticnet 6: Ensemble application of symbolic and subsymbolic AI for sentiment analysis. In CIKM '20: The 29th ACM International Conference on Information and Knowledge Management, Virtual Event, Ireland, October 19-23, 2020, pages 105-114.
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2019, Minneapolis, MN, USA, June 2-7, 2019, Volume 1 (Long and Short Papers), pages 4171-4186.
Jacob Eisenstein. 2017. Unsupervised learning for lexicon-based classification. In Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence, February 4-9, 2017, San Francisco, California, USA, pages 3188-3194.
Bjarke Felbo, Alan Mislove, Anders Søgaard, Iyad Rahwan, and Sune Lehmann. 2017. Using millions of emoji occurrences to learn any-domain representations for detecting sentiment, emotion and sarcasm. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, EMNLP 2017, Copenhagen, Denmark, September 9-11, 2017, pages 1615-1625.
Aniruddha Ghosh and Tony Veale. 2016. Fracking sarcasm using neural network. In Proceedings of the 7th Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysis, WASSA@NAACL-HLT 2016, June 16, 2016, San Diego, California, USA, pages 161-169.
Aniruddha Ghosh and Tony Veale. 2018. *Ironymagnet at seveal-2018 task 3: A siamese network for irony detection in social media. In *Proceedings of The 12th International Workshop on Semantic Evaluation*, SemEval@NAACL-HLT 2018, New Orleans, Louisiana, USA, June 5-6, 2018, pages 570-575.
Debanjan Ghosh, Weiwei Guo, and Smaranda Muresan. 2015. Sarcastic or not: Word embeddings to predict the literal or sarcastic meaning of words. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, EMNLP 2015, Lisbon, Portugal, September 17-21, 2015, pages 1003-1012.
Debanjan Ghosh, Avijit Vajpayee, and Smaranda Muresan. 2020. A report on the 2020 sarcasm detection shared task. In Proceedings of the Second Workshop on Figurative Language Processing, Fig-Lang@ACL 2020, Online, July 9, 2020, pages 1-11.
Roberto González-Ibanez, Smaranda Muresan, and Nina Wacholder. 2011. Identifying sarcasm in twitter: a closer look. In The 49th Annual Meeting of the Association for Computational Linguistics: Human
Language Technologies, Proceedings of the Conference, 19-24 June, 2011, Portland, Oregon, USA - Short Papers, pages 581-586.
Roberto I. González-Ibanez, Smaranda Muresan, and Nina Wacholder. 2011. Identifying sarcasm in twitter: A closer look. In *The 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies*, Proceedings of the Conference, 19-24 June, 2011, Portland, Oregon, USA - Short Papers, pages 581-586.
Devamanyu Hazarika, Soujanya Poria, Sruthi Gorantla, Erik Cambria, Roger Zimmermann, and Rada Mihalcea. 2018. CASCADE: contextual sarcasm detection in online discussion forums. In Proceedings of the 27th International Conference on Computational Linguistics, COLING 2018, Santa Fe, New Mexico, USA, August 20-26, 2018, pages 1837-1848.
Cynthia Van Hee, Els Lefever, and Véronique Hoste. 2018a. Semeval-2018 task 3: Irony detection in english tweets. In Proceedings of The 12th International Workshop on Semantic Evaluation, SemEval@NAACL-HLT 2018, New Orleans, Louisiana, USA, June 5-6, 2018, pages 39-50.
Cynthia Van Hee, Els Lefever, and Véronique Hoste. 2018b. We usually don't like going to the dentist: Using common sense to detect irony on twitter. Comput. Linguistics, 44(4).
Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. *Neural Comput.*, 9(8):1735-1780.
Minqing Hu and Bing Liu. 2004. Mining and summarizing customer reviews. In Proceedings of the Tenth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Seattle, Washington, USA, August 22-25, 2004, pages 168-177.
Aditya Joshi, Pushpak Bhattacharyya, and Mark James Carman. 2017. Automatic sarcasm detection: A survey. ACM Comput. Surv., 50(5):73:1-73:22.
Aditya Joshi, Vinita Sharma, and Pushpak Bhattacharyya. 2015. Harnessing context incongruity for sarcasm detection. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 2: Short Papers), pages 757-762.
Mikhail Khodak, Nikunj Saunshi, and Kiran Vodrahalli. 2018. A large self-annotated corpus for sarcasm. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation, LREC 2018, Miyazaki, Japan, May 7-12, 2018.
Yoon Kim. 2014. Convolutional neural networks for sentence classification. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, EMNLP 2014, October 25-29, 2014, Doha, Qatar; A meeting of SIGDAT, a Special Interest Group of the ACL, pages 1746-1751. ACL.
Diederik P. Kingma and Jimmy Ba. 2015. Adam: A method for stochastic optimization. In 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings.
Thomas N. Kipf and Max Welling. 2017. Semi-supervised classification with graph convolutional networks. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings.
Y. Alex Kolchinski and Christopher Potts. 2018. Representing social media users for sarcasm detection. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium, October 31 - November 4, 2018, pages 1115-1121.
Jiangnan Li, Hongliang Pan, Zheng Lin, Peng Fu, and Weiping Wang. 2021. Sarcasm detection with commonsense knowledge. IEEE ACM Trans. Audio Speech Lang. Process., 29:3192-3201.
Bing Liu. 2012. Sentiment Analysis and Opinion Mining. Synthesis Lectures on Human Language Technologies. Morgan & Claypool Publishers.
Chenwei Lou, Bin Liang, Lin Gui, Yulan He, Yixue Dang, and Ruifeng Xu. 2021. Affective dependency graph for sarcasm detection. In SIGIR '21: The 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, Virtual Event, Canada, July 11-15, 2021, pages 1844-1849.
Stephanie Lukin and Marilyn Walker. 2013. Really? well. apparently bootstrapping improves the performance of sarcasm and nastiness classifiers for online dialogue. In Proceedings of the Workshop on Language Analysis in Social Media, pages 30-40, Atlanta, Georgia. Association for Computational Linguistics.
Andrew John Morrison. 2008. Sarcasm and other mixed messages: The ambiguous way people use language, by patricia ann rockwell. Journal of Politeness Research, 4(2):331-334.
Shereen Oraby, Vrindavan Harrison, Lena Reed, Ernesto Hernandez, Ellen Riloff, and Marilyn A. Walker. 2016. Creating and characterizing a diverse corpus of sarcasm in dialogue. In Proceedings of the SIGDIAL 2016 Conference, The 17th Annual Meeting of the Special Interest Group on Discourse and Dialogue, 13-15 September 2016, Los Angeles, CA, USA, pages 31-41.
Braja Gopal Patra, Soumadep Mazumdar, Dipankar Das, Paolo Rosso, and Sivaji Bandyopadhyay. 2016. A multilevel approach to sentiment analysis of figurative language in twitter. In Computational Linguistics and Intelligent Text Processing - 17th International Conference, CIC Ling 2016, Konya, Turkey, April 3-9, 2016, Revised Selected Papers, Part II, volume 9624 of Lecture Notes in Computer Science, pages 281-291. Springer.
Jeffrey Pennington, Richard Socher, and Christopher D Manning. 2014. Glove: Global vectors for word representation. In Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), pages 1532-1543.
Soujanya Poria, Erik Cambria, Devamanyu Hazarika, and Prateek Vij. 2016. A deeper look into sarcastic tweets using deep convolutional neural networks. In COLING 2016, 26th International Conference on Computational Linguistics, Proceedings of the Conference: Technical Papers, December 11-16, 2016, Osaka, Japan, pages 1601-1612.
Tomáš Ptáček, Ivan Habernal, and Jun Hong. 2014. Sarcasm detection on czech and english twitter. In COLING 2014, 25th International Conference on Computational Linguistics, Proceedings of the Conference: Technical Papers, August 23-29, 2014, Dublin, Ireland, pages 213-223.
Ellen Riloff, Ashequl Qadir, Prafulla Survey, Lalindra De Silva, Nathan Gilbert, and Ruihong Huang. 2013. Sarcasm as contrast between a positive sentiment and negative situation. In Proceedings of the 2013 conference on empirical methods in natural language processing, pages 704-714.
Richard Socher, Cliff Chiung-Yu Lin, Andrew Y. Ng, and Christopher D. Manning. 2011. Parsing natural scenes and natural language with recursive neural networks. In Proceedings of the 28th International Conference on Machine Learning, ICML 2011, Bellevue, Washington, USA, June 28 - July 2, 2011, pages 129-136. Omnipress.
Maite Taboada, Julian Brooke, Milan Tofiloski, Kimberly D. Voll, and Manfred Stede. 2011. Lexicon-based methods for sentiment analysis. Comput. Linguistics, 37(2):267-307.
Yi Tay, Anh Tuan Luu, Siu Cheung Hui, and Jian Su. 2018. Reasoning with sarcasm by reading in between. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, ACL 2018, Melbourne, Australia, July 15-20, 2018, Volume 1: Long Papers, pages 1010-1020.
Laurens Van der Maaten and Geoffrey Hinton. 2008. Visualizing data using t-sne. Journal of machine learning research, 9(11).
Yequan Wang, Minlie Huang, Xiaoyan Zhu, and Li Zhao. 2016. Attention-based LSTM for aspect-level sentiment classification. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, EMNLP 2016, Austin, Texas, USA, November 1-4, 2016, pages 606-615.
Theresa Wilson, Janyce Wiebe, and Paul Hoffmann. 2005. Recognizing contextual polarity in phrase-level sentiment analysis. In HLT/EMNLP 2005, Human Language Technology Conference and Conference on Empirical Methods in Natural Language
Processing, Proceedings of the Conference, 6-8 October 2005, Vancouver, British Columbia, Canada, pages 347-354.
Chuhan Wu, Fangzhao Wu, Sixing Wu, Junxin Liu, Zhigang Yuan, and Yongfeng Huang. 2018. Thu_ngn at semeval-2018 task 3: Tweet irony detection with densely connected LSTM and multi-task learning. In Proceedings of The 12th International Workshop on Semantic Evaluation, pages 51-56.
Tao Xiong, Peiran Zhang, Hongbo Zhu, and Yihui Yang. 2019. Sarcasm detection with self-matching networks and low-rank bilinear pooling. In The World Wide Web Conference, WWW 2019, San Francisco, CA, USA, May 13-17, 2019, pages 2115-2124.
Meishan Zhang, Yue Zhang, and Guohong Fu. 2016. Tweet sarcasm detection using deep neural network. In COLING 2016, 26th International Conference on Computational Linguistics, Proceedings of the Conference: Technical Papers, December 11-16, 2016, Osaka, Japan, pages 2449-2460. ACL.