SlowGuess/NAACL_2021_OCR huggingface dataset:
55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 634-642.
Shantanu Godbole and Sunita Sarawagi. 2004. Discriminative methods for multi-labeled classification. In Pacific-Asia Conference on Knowledge Discovery and Data Mining, pages 22-30.
Chenyang Huang, Amine Trabelsi, and Osmar Zaiane. 2019. ANA at SemEval-2019 Task 3: Contextual emotion detection in conversations through hierarchical LSTMs and BERT. In Proceedings of the 13th International Workshop on Semantic Evaluation, pages 49-53.
Chenyang Huang, Osmar Zaiane, Amine Trabelsi, and Nouha Dziri. 2018. Automatic dialogue generation with expressed emotions. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers), pages 49-54.
Diederik P. Kingma and Jimmy Ba. 2015. Adam: A method for stochastic optimization. In Proceedings of the International Conference on Learning Representations.
Minh-Thang Luong, Hieu Pham, and Christopher D Manning. 2015. Effective approaches to attention-based neural machine translation. In Proceedings of Conference on Empirical Methods in Natural Language Processing, pages 1412-1421.
Hardik Meisheri and Lipika Dey. 2018. TCS research at SemEval-2018 Task 1: Learning robust representations using multi-attention architecture. In Proceedings of the 12th International Workshop on Semantic Evaluation, pages 291-299.
Mike Mintz, Steven Bills, Rion Snow, and Daniel Jurafsky. 2009. Distant supervision for relation extraction without labeled data. In Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing, pages 1003-1011.
Saif Mohammad. 2012. #Emotional Tweets. In *SEM 2012: The First Joint Conference on Lexical and Computational Semantics, pages 246-255.
Saif Mohammad. 2018. Word affect intensities. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation, pages 174-181.
Saif Mohammad, Felipe Bravo-Marquez, Mohammad Salameh, and Svetlana Kiritchenko. 2018. SemEval-2018 Task 1: Affect in Tweets. In Proceedings of the 12th International Workshop on Semantic Evaluation, pages 1-17.
Saif M Mohammad and Svetlana Kiritchenko. 2015. Using hashtags to capture fine emotion categories from Tweets. Computational Intelligence, 31(2):301-326.
Ji Ho Park, Peng Xu, and Pascale Fung. 2018. PlusEmo2Vec at SemEval-2018 Task 1: Exploiting emotion knowledge from emoji and #hashtags. In Proceedings of the 12th International Workshop on Semantic Evaluation, pages 264-272.
Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014. GloVe: Global vectors for word representation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, pages 1532-1543.
Matthew Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep contextualized word representations. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 2227-2237.
Robert Plutchik. 1980. A general psychoevolutionary theory of emotion. Theories of Emotion, pages 3-33.
Soujanya Poria, Alexander F. Gelbukh, Erik Cambria, Amir Hussain, and Guang-Bin Huang. 2014. EmoSenticSpace: A novel framework for affective common-sense reasoning. Knowledge-Based Systems, 69:108-123.
Alec Radford, Rafal Jozefowicz, and Ilya Sutskever. 2017. Learning to generate reviews and discovering sentiment. arXiv preprint arXiv:1704.01444.
Jesse Read, Bernhard Pfahringer, Geoff Holmes, and Eibe Frank. 2009. Classifier chains for multi-label classification. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pages 254-269.
David J Rogers and Taffee T Tanimoto. 1960. A computer program for classifying plants. Science, 132(3434):1115-1118.
Kashfia Sailunaz, Manmeet Dhaliwal, Jon G. Rokne, and Reda Alhajj. 2018. Emotion detection from text and speech: A survey. Social Network Analysis Mining, 8(1):28:1-28:26.
Victor Sanh, Thomas Wolf, and Sebastian Ruder. 2019. A hierarchical multi-task approach for learning embeddings from semantic tasks. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pages 6949-6956.
Robert E. Schapire and Yoram Singer. 1999. Improved boosting algorithms using confidence-rated predictions. Machine Learning, 37(3):297-336.
Klaus R Scherer and Harald G Wallbott. 1994. Evidence for universality and cultural variation of differential emotion response patterning. Journal of Personality and Social Psychology, 66(2):310.
Ameneh Gholipour Shahraki and Osmar R. Zaiane. 2017. Lexical and learning-based emotion mining from text.
Carlo Strapparava and Alessandro Valitutti. 2004. WordNet Affect: an affective extension of WordNet. In Proceedings of the Fourth International Conference on Language Resources and Evaluation, pages 1083-1086.
Duyu Tang, Furu Wei, Nan Yang, Ming Zhou, Ting Liu, and Bing Qin. 2014. Learning sentiment-specific word embedding for Twitter sentiment classification. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1555-1565.
Ryoko Tokuhisa, Kentaro Inui, and Yuji Matsumoto. 2008. Emotion classification using massive examples extracted from the Web. In Proceedings of the 22nd International Conference on Computational Linguistics, pages 881-888.
Shiyang Wen and Xiaojun Wan. 2014. Emotion classification in microblog texts using class sequential rules. In Proceedings of the Twenty-Eighth AAAI Conference on Artificial Intelligence, pages 187-193.
Ali Yadollahi, Ameneh Gholipour Shahraki, and Osmar R Zaiane. 2017. Current state of text sentiment analysis from opinion to emotion mining. ACM Computing Surveys, 50(2):1-33.
Pengcheng Yang, Xu Sun, Wei Li, Shuming Ma, Wei Wu, and Houfeng Wang. 2018. SGM: Sequence generation model for multi-label classification. In Proceedings of the 27th International Conference on Computational Linguistics, pages 3915-3926.
Jianfei Yu, Luís Marujo, Jing Jiang, Pradeep Karuturi, and William Brendel. 2018. Improving multi-label emotion classification via sentiment classification with dual attention transfer network. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 1097-1102.
Hao Zhou, Minlie Huang, Tianyang Zhang, Xiaoyan Zhu, and Bing Liu. 2018. Emotional chatting machine: Emotional conversation generation with internal and external memory. In Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, pages 730-739.
Table 4 shows the statistics of both SemEval'18 and GoEmotions datasets. Noticeably, the majority of the data samples in SemEval'18 are labeled with at least two emotions. The GoEmotions dataset is mostly annotated with one label for an utterance, although multiple emotions do exist. This suggests that SemEval'18 may contain more correlated emotions on average.
Dataset	# emo.	# sample	% multi-emo.	# avg. emo.
SemEval'18	11	10690	86.1	2.37
GoEmotions	24	54263	16.2	1.17
Table 4: Data statistics: the number of the emotion categories, the number of data samples, the percentage of multi-labeled samples, and the average number of emotions per utterance.
In Figure 2, we visualize the attention layer of Seq2Emo by plotting the heat map over the attention scores. The emotions shown in each example are the groundtruth labels of the corresponding utterance.
We observe that Seq2Emo is able to focus on relevant words when predicting the emotion of interest. In Case 3, for example, the emotions joy and love highly resemble each other, both focusing on the word "laughter." On the other hand, the decoder of Seq2Emo can focus on entirely different words if the emotions are different. In Case 1, we see the emotion anticipation mainly focuses on "see free," whereas the emotion optimism mainly focuses on "is lining up volunteers."
Case 1: shriekfest is lining up volunteers! date - number, only serious inquiries please! email see free films!
Case 2: parish elongated + sad song = prefect night feeling alone
Case 3: treat joy and laughter as a form of worship and spiritual warfare! laughter live victory worship
Case 4: user ' operation echoes ' is gathering momentum . . . tense feel sick excited
Figure 2: Case study.