Unified Humor Detection Based on Sentence-pair Augmentation and Transfer Learning (2020)

Abstract: We propose a unified multilingual model for humor detection which can be trained under a transfer learning framework. 1) The model is built based on pre-trained multilingual BERT, thereby is able to make predictions on Chinese, Russian and Spanish corpora. 2) We step out from single sentence classification and propose sequence-pair prediction which considers the inter-sentence relationship. 3) We propose the Sentence Discrepancy Prediction (SDP) loss, aiming to measure the semantic discrepancy of the sequence-pair, which often appears in the setup and punchline of a joke. Our method achieves two SoTA and a second-place on three humor detection corpora in three languages (Russian, Spanish and Chinese), and also improves F1-score by 4%-6%, which demonstrates the effectiveness of it in humor detection tasks.

## introduction machine learning has been adopted in computational linguistic for understanding natural languages for several decades. with the development of representation learning, rich semantics can be encoded into the dense vectors named as embedding, which significantly improves the ability of algorithms in understanding fine-grained emotions, for example, judging whether a sentence is humorous, often formulated as a binary classification problem. there can be many applications of humor detection such as language understanding in c 2020 the authors. this article is licensed under a creative commons 3.0 licence, no derivative works, attribution, cc-by-nd. figure 1 : an example from haha corpus shows that the semantic discrepancy exists in a joke, where the urinated stones is a disease in the left picture and is an action in the right image, originated from the second and the third sentence in the joke: "-doctor, my kidney hurts a lot -have you urinated stones? -yes doctor, i urinated stones, cars, trees, posts ..." many algorithms has been used to solve these problems such as conventional machine learning algorithms like tf-idf representation with svm classifier, or deep learning based like bert @xcite . however, most of these algorithms are typically designed for universal tasks but ignoring the difference (e.g. the paragraph structure and semantic features) between humor detection and other document classification tasks. from a linguistic perspective, there are two critical features that often appear in jokes, which inspire us to model them explicitly and make specific optimization for the task: by reviewing features of jokes, we can start our study by making two assumptions. 1) most of jokes have punchline, and can be appropriately modeled. 2) most of punchlines have semantic discrepancies with setup, and can be considered as a factor in the determination of humorous. therefore, we propose a method for humor detection which can be described as three stages. 1) data augmentation with paragraph decomposition. chinese humor [ c1, .cn] spain humor [ s1, .sn] ruassian humor [ r1, .rn] c1 cls c2 c3 ... cn -1 cn c1 cls c2 ... cm [sep] ... s1 cls s2 s3 ... sn-1 sn s1 cls s2 sm [sep] ... r1 cls r2 r3 ... rn-1 rn r1 cls r2 rm [sep] ... in recent years, many studies on humor detection have been published. some researchers focuses on employing state-of-the-art studies like bert @xcite to make better predictions, others attempts to improve simple networks like lstm @xcite and cnn @xcite or even conventional machine learning algorithms to compete with deep neural networks. at the same time, researchers have made available several high-quality datasets in different languages which significant help investigations on this area. @xcite propose a bert based humor detection model, fine-tuned on corpus collected from reddit, short jokes and pun of the day @xcite , which achieves significant improvement on the performance comparing with many cnn based models. @xcite summaries a series of works from teams who build models and conduct experiments on haha dataset in the iber-lef 2019. @xcite propose the method based on a pre-trained multilingual bert, and further pre-train it on the domain dataset. finally, the model is fine-tuned with task specific labels. apart from that, they combine the prediction of naive bayes with tf-idf and nn outputs with logistic regression to produce the final prediction, which achieves the best result in the haha 2019 challenge. other teams also follows the framework by combining deep pre-trained models with conventional algorithms to acquire competitive predictions. @xcite @xcite release large corpus in different languages like russian and spanish, which give chances for researchers to build and evaluate their models on more diverse datasets. at the same time, they evaluate their datasets with proposed models and make detailed analysis which successfully demonstrates the good quality of the corpus. by reviewing previous works and analyzing their results, we choose to follow a similar pipeline to start our work based on the pre-trained multilingual bert and evaluate our method on three datasets in different languages aiming to investigate whether the feature of punchline exists in jokes from different cultures and can be detected with the model. ## approach in this section, we introduce details of our method in the three stages which is shown in figure 2 , and we also discuss the advantages of our method comparing with others. ## experiments in this section, we introduce the details of the datasets, as well as the experimental setup. ## analysis the experimental results is shown in zh 猫似乎只是在削尖他们的爪子。 [sep] 实际上， 他们正在锻炼腿部肌肉。 0.61 猫似乎只是在削尖他们的爪子。 实际上， 他们正在锻炼腿部肌肉。[sep] 0.55 en cats seem to be just sharpening their claws. [sep] in fact, they are exercising leg muscles. 0.61 cats seem to be just sharpening their claws. in fact, they are exercising leg muscles. [sep] 0.55 we find a representative case from ccl dataset, which is shown in table .3. we can see that decomposing the joke from the start of the second sentence achieves higher probability and the second sentence is actually the punchline of this joke. although the score of haha is acceptable, we find some cases showing that the tweets published in haha is relatively unclean, with noisy characters like hashtags or being barely readable even by human, which also happens in fun. as shown in table . 4, repeatedly appeared "ja" and hashtags may corrupt the paragraph decomposition algorithm and produce unreasonable paragraph pairs. at the same time, bert is not pretrained on tweets or corpus from social networks which means the token representations of fun and haha is insufficient to encode correct semantics. ## conclusion we propose the sdp and paragraph decomposition to for humor detection, by linking the classification label to the inter-sentence discrepancy prediction. our proposed method achieves competitive performance on three dataset with different languages. although our sdp algorithm has achieved great performance on humor detection tasks, how to generalize it to other nlp tasks remains as our future work. es ¿tu? ¿gustarme? ja ja ja ja ja ja ja ja ja ja ja ja ja ja ja ja ja ja ja ja ja ja ja ja ja ja ja ja j á tengo que disimular un poco mas. #20cosasquehacerantesdemorir: enseñarles la diferencia entre: -hay de haber -ahí de lugar -ay de exclamar -ai se eu te pego. rt con el pollo asado #premiosfenix en ¿your? ¿like me? ja ja ja ja ja ja ja ja ja ja ja ja ja ja ja ja ja ja ja ja ja ja ja ja ja ja ja ja j á i have to hide a little more. #20things to do before you die: teach them the difference between: -there is a place -ay to exclaim -i hit you there. rt with roast chicken #premiosfenix