SlowGuess/ACL_2020_OCR huggingface dataset:
$\mathbf{R}^{3}$ : Reverse, Retrieve, and Rank for Sarcasm Generation with Commonsense Knowledge
Tuhin Chakrabarty $^{1,2*}$ , Debanjan Ghosh $^{3}$ , Smaranda Muresan $^{2,4}$  and Nanyun Peng $^{1}$
<sup>1</sup>Information Sciences Institute, University of Southern California
$^{2}$ Department of Computer Science, Columbia University
$^{3}$ Educational Testing Service,  $^{4}$ Data Science Institute, Columbia University
{tuhin.chakrabarty, smara}@columbia.edu
dghosh@ets.org, npeng@isi.edu
We propose an unsupervised approach for sarcasm generation based on a non-sarcastic input sentence. Our method employs a retrieve-and-edit framework to instantiate two major characteristics of sarcasm: reversal of valence and semantic incongruity with the context, which could include shared commonsense or world knowledge between the speaker and the listener. While prior works on sarcasm generation predominantly focus on context incongruity, we show that combining valence reversal and semantic incongruity based on commonsense knowledge generates sarcastic messages of higher quality based on several criteria. Human evaluation shows that our system generates sarcasm better than human judges  $34\%$  of the time, and better than a reinforced hybrid baseline  $90\%$  of the time.
Studies have shown that the use of sarcasm or verbal irony, can increase creativity on both the speakers and the addressees (Huang et al., 2015), and can serve different communicative purposes such as evoking humor and diminishing or enhancing critique (Burgers et al., 2012). Thus, developing computational models that generate sarcastic messages could impact many downstream applications, such as better conversational agents and creative or humorous content creation. While most computational work has focused on sarcasm detection (Davidov et al., 2010; González-Ibanez et al., 2011; Riloff et al., 2013; Ghosh et al., 2015; Joshi et al., 2015b; Muresan et al., 2016; Ghosh and Veale, 2017; Ghosh et al., 2017, 2018), research on sarcasm generation is in its infancy (Joshi et al., 2015a; Mishra et al., 2019). Sarcasm generation
Literal Input 1	I hate getting sick from fast food.
GenSarc1	I love getting sick from fast food.
GenSarc2	[I love getting sick from fast food.][Stomach ache is just an additional side effect.]
Human 1	Shout out to the Mc donalds for giving me bad food and making me sick right before work in two hours.
Literal Input 2	I inherited unfavorable genes from my mother.
GenSarc3	I inherited great genes from my mother.
GenSarc4	[I inherited great genes from my mother.][Ugly goes down to the bone.]
Human 2	Great I inherited all of my mother’s GOOD genes
Table 1: Table showing a literal or non sarcastic input sentence and respective sarcastic outputs. GenSarc1 and GenSarc3 simply reverses the valence, while GenSarc2 and GenSarc4 add commonsense context to create incongruity or enhance the humorous effect.
is a challenging problem since the generated utterance should have at least five characteristics (a.k.a. "sarcasm factors") (Burgers et al., 2012): 1) be evaluative; 2) be based on a reversal of valence between the literal and intended meaning; 3) be based on a semantic incongruity with the context, which can include shared commonsense or world knowledge between the speaker and the addressee; 4) be aimed at some target, and 5) be relevant to the communicative situation in some way. To simplify the problem, we focus on the task of generating a sarcastic utterance starting from a non-sarcastic utterance that conveys the speaker's intended meaning and that is evaluative. Consider the examples in Table 1. Given the literal input "I hate getting sick from fast food" or "I inherited unfavorable genes from my mother", our task is to generate a sarcastic message that would convey this intended literal meaning. In this simplifying task, we are not concerned with the fifth characteristic, while the first and to some degree, the fourth are specified by the input (literal) utterances.
* The research was conducted when the author was at USC/ISI.
7976
Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 7976-7986 July 5 - 10, 2020. ©2020 Association for Computational Linguistics
Given the lack of "training" data for the sarcasm generation task, we propose a novel unsupervised approach that has three main modules guided by the above mentioned sarcasm factors:
1. Reversal of Valence: To generate sarcastic utterances that satisfy the second characteristic we identify the evaluative word and use negation or lexical antonyms to generate the sarcastic utterance by reversing the valence (Section 4.1). For example, given, "I hate getting sick from fast food" this module will generate "I love getting sick from fast food" (GenSarc1 in Table 1).
2. Retrieval of Commonsense Context: Adding commonsense context could be important to make explicit the semantic incongruity factor (e.g., GenSarc4 vs. GenSarc3 in Table 1), or could enhance the humorous effect of the generated sarcastic message (e.g., GenSarc2 vs. GenSarc1 in Table 1).
We propose an approach where retrieved relevant commonsense context sentences are to be added to the generated sarcastic message. At first, we use a pre-trained language model fine-tuned on the ConceptNet (Speer et al., 2017) called COMET (Bosselut et al., 2019) to generate relevant commonsense knowledge. COMET gives us that, "inherited unfavorable genes from my mother" causes "to be ugly" or that "getting sick from fast food" causes "stomach ache" (Section 4.2.1). The derived commonsense concept is then used to retrieve relevant sentences — from a corpus — that could be added to the sentence obtained through reversal of valence (e.g., "Stomach ache is just an additional side effect" in Table 1) (Section 4.2.2).
3. Ranking of Semantic Incongruity: The previous module generates a list of candidate commonsense contexts. Next, we measure contradiction between each of these commonsense contexts and the sentence generated by the reversal of valence approach (module 1) and select the commonsense context that received the highest contradiction score. Finally, we concatenate the selected context to the sentence obtained through reversal of valence. Here, conceptually, contradiction detection is aimed to capture the semantic incongruity between the output of valence reversal and its
context. Contradiction scores are obtained from a model trained on the Multi-Genre NLI Corpus (Williams et al., 2018) (Section 4.3).
We test our approach on 150 non-sarcastic utterances randomly sampled from two existing data sets. We conduct human evaluation using several criteria: 1) how sarcastic is the generated message; 2) how humorous it is; 3) how creative it is; and 4) how grammatical it is. Evaluation via Amazon's Mechanical Turk (MTurk) shows that our system is better  $34\%$  of the time compared to humans and  $90\%$  of the time compared to a recently published reinforced hybrid baseline (Mishra et al., 2019). We also present a thorough ablation study of several variations of our system demonstrating that incorporating more sarcasm factors (e.g., reversal of valence, commonsense context, and semantic incongruity) lead to higher quality sarcastic utterances. We make the code and data from our experiments publicly available.
2.1 Sarcasm Generation
Research on sarcasm generation is in its infancy. Joshi et al. (2015a) proposed SarcasmBot, a sarcasm generation system that implements eight rule-based sarcasm generators, each of which generates a certain type of sarcastic expression. Peled and Reichart (2017) introduced a novel task of sarcasm interpretation, defined as the generation of a non-sarcastic utterance conveying the same message as the original sarcastic one. They use supervised machine translation models for the same in presence of parallel data. However, it is impractical to assume the existence of large corpora for training supervised generative models using deep neural nets; we hence resort to unsupervised approaches. Mishra et al. (2019) employed reinforced neural seq2seq learning and information retrieval based approaches to generate sarcasm. Their models are trained using only unlabeled non-sarcastic and sarcastic opinions. They generated sarcasm as a disparity between positive sentiment context and negative situational context. We, in contrast, model sarcasm using semantic incongruity with the context which could include shared commonsense or world knowledge.
<sup>1</sup>https://github.com/tuhinjubcse/SarcasmGeneration-ACL2020
7977
2.2 Style Transfer
Prior works looked into unsupervised text style/sentiment transfer (Shen et al., 2017; Fu et al., 2017; Li et al., 2018), which transfers a sentence from one style to another without changing the content. This is relevant to the reversal of valence for sarcasm generation. However, these transformations are mainly at the lexical and syntax levels rather than pragmatic level; in contrast, sarcastic utterances often include additional information associated with the context they occur (Regel, 2009), which is beyond text style/sentiment transfer.
2.3 Use of Commonsense for Irony Detection
The study of irony and sarcasm are closely related as sarcasm is defined as, "the use of verbal irony to mock someone or show contempt". Van Hee et al. (2018) addressed the challenge of modeling implicit or prototypical sentiment in the framework of automatic irony detection. They first manually annotated stereotypical ironic situations (e.g., flight delays) and later addressed the implicit sentiment held towards such situations automatically by using both a lexico-semantic commonsense knowledge base and a data-driven method. They however used it for irony detection, while we are focused on sarcasm generation.[2]
3 Sarcasm Factors Used in Generation
A sarcastic utterance must satisfy the sarcasm factors, i.e., the inherent characteristics of sarcasm (Attardo, 2000; Burgers et al., 2012). In this research, we leverage the use of two particular factors to generate sarcasm. One is the reversal of valence and the other is the semantic incongruity with the context, which could include shared commonsense or world knowledge between the speaker and the hearer.
3.1 Reversal of Valence
The first key sarcasm factor is the reversal of valence between the literal and the intended meaning (Burgers et al., 2012). Reversal of valence can be achieved in two ways: when the literal meaning of the sarcastic message is positive (e.g., "that is a great outfit" if the outfit is ugly) or when the literal
meaning is negative (e.g., "that is an ugly dress" if the dress is really beautiful). Arguably, the former is more likely to appear in sarcastic utterances. As the intended meaning is generally the opposite of its literal meaning in sarcastic utterances (Gibbs, 1986), using lexical antonym of negative sentiment words or negation can be used to convert a non-sarcastic utterance to its sarcastic version. For example, given a non-sarcastic utterance "Zero visibility in fog makes driving difficult", one could identify the evaluative negative word difficult and replace it with its antonym easy, thereby converting the utterance to the sarcastic "Zero visibility in fog makes driving easy". Likewise, "Drunk driving should be taken seriously" can be converted to its sarcastic counterpart, "Drunk driving should not be taken seriously" by using negation. We propose a generation approach that is able to capture the reversal of valence (Section 4.1).
3.2 Semantic Incongruity
The second sarcasm factor, semantic incongruity, appears between the literal evaluation and the context, as in the example "I love getting sick from fast food", where we have semantic incongruity between the positive word "love" and the negative situation "getting sick". However, often, the negative situation is absent from the utterance, and thus additional pragmatic inference is needed to understand the sarcastic intent. For example, the listener might miss the sarcastic intent in "zero visibility in fog makes driving easy", where the speaker meant to convey that it can cause "accidents". Adding "suffered three cracked ribs in an accident." makes the sarcastic intent more explicit, while maintaining the acerbic wit of the speaker. In the next section, we propose a novel generation approach that incorporates such relevant commonsense knowledge as context for semantic incongruity (Section 4.2 and Section 4.3).
4 Unsupervised Sarcasm Generation
An overview of the sarcasm generation pipeline is shown in Figure 1. In this section, we detail the three main modules that are designed to instantiate the key sarcasm factors.
4.1 Reversal of Valence
As sarcasm is a type of verbal irony used to mock or convey contempt, in most sarcastic messages we encounter a positive sentiment towards a nega
2While we do not directly model the negative intent in sarcasm, the generated output could lead to sarcastic messages rather than just ironic depending on the initial target given in the non-sarcastic message (E.g a sample generation "Our politicians have everything under control. The nation is in danger of falling into anarchy.")
Figure 1: Our complete pipeline for sarcasm generation. The components with highlighted background denote Reversal of Valence, Retrieval of Commonsense Context and Ranking based on Semantic Incongruity respectively
tive situation (i.e., ironic criticism (Kreuz and Link, 2002)). This observation is also supported by research on sarcasm detection, particularly on social media. Hence, for our sarcasm generation task, we focus on transforming a literal utterance with negative valence into positive valence.
To implement the reversal of valence, as highlighted in the yellow background in Figure 1, we first identify the evaluative words and replace them with their lexical antonyms using WordNet (Miller, 1995). As we expect the evaluative words to be negative words, we rely on the word level negative scores obtained from SentiWordNet (Esuli and Sebastiani, 2006). In the absence of words with negative polarity, we check if there is the negation word not or words ending with  $n$  't and remove these words. In case there are both negative words and not (or words ending in  $n$  't), we handle only one of them. Given the non sarcastic example "zero visibility in fog makes driving difficult" shown in Figure 1 and which we use as our running example, the reversal of valence module generates "zero visibility in fog makes driving easy".
4.2 Retrieval of Commonsense Context
As discussed before, a straightforward reversal of valence might not generate sarcastic messages that display a clear semantic incongruity, and thus, additional context is needed. We propose an approach to retrieve relevant context for the sarcastic message based on commonsense knowledge. First, we generate commonsense knowledge based on Con
cepNet (e.g., "driving in zero visibility" causes "accidents") (Section 4.2.1). Second, we retrieve candidate context sentences that contain the commonsense concept from a retrieval corpus (Section 4.2.2) and edit them for grammatical consistency with the input message (Section 4.2.3).
4.2.1 Commonsense Reasoning
We extract nouns, adjectives, adverbs, and verbs from the non-sarcastic input messages and feed them as input to COMET (Bosselut et al., 2019) model to generate commonsense knowledge (highlighted in green background in Figure 1). COMET is an adaptation framework for constructing commonsense knowledge based on pre-trained language models. It initiates with a pre-trained GPT (Radford et al., 2018) model and fine-tune on commonsense knowledge tuples (in our case, ConceptNet (Speer et al., 2017)). These tuples provide COMET with the knowledge base structure and relations that must be learned, and COMET adapts the representations that the language model learned from the pre-training stage to add novel nodes to the seed knowledge graph. Our work only leverages the causes relation. For instance, from our running example, we first remove the stopwords and then extract nouns, adjectives, adverbs, and verbs including the terms zero, visibility, fog, makes driving, and difficult to feed to COMET as inputs. In turn, COMET returns the probable causes with their probability scores. For the running example, COMET returns with the highest probability that
7979
these terms may cause an accident (illustrated in Figure 2). For further details regarding COMET please see Bosselut et al. (2019).
4.2.2 Retrieving Sentences Containing Commonsense Concepts
Once we obtain the most probable output from COMET, the next step is to retrieve sentences containing the commonsense word or phrase from a retrieval corpus. We impose several constraints: (a) the retrieved sentences should contain the commonsense concept at the beginning or at the end; (b) sentence length should be less than twice the number of tokens in the non-sarcastic input to keep a consistency between the length of the non-sarcastic input and its sarcastic version. If none of the commonsense phrase is present in the retrieval corpus, we retrieve sentences containing the nouns within the top most phrase. For example, if COMET yields microwave burger awful causes the phrase food to spoil, and this phrase does not appear in any sentence in the retrieval corpus, we search for food and later replace it in the retrieved sentence with food to spoil. COMET often returns output with common phrases such as you to be, you to get, person will be, you have which we also removed while keeping the main content word (i.e the commonsense concept) We use Sentencedict.com, an online sentence dictionary as the retrieval corpus, where one can find high quality sentences for almost every word obeying the above constraints.
4.2.3 Grammatical Consistency
We first check whether the retrieved sentences are consistent with the non-sarcastic input in terms of the pronouns. If the pronouns are mismatched, then we modify the pronoun of the retrieved sentence to match the pronoun of the non-sarcastic input. In case, the non-sarcastic input does not have any pronoun, but the retrieved sentence does, we simply change that pronoun to "I". For example, if the non-sarcastic input sentence is "Ignoring texts is literally the worst part of communication." and the retrieved commonsense sentence is "He has never suffered the torment of rejection.", we modify the retrieved sentence to "I have never suffered the torment of rejection." to have consistency among the pronoun use. After correcting the pronouns and proper names (in the same way as pronoun correction), we feed the corrected sentences into the Neural Grammatical Error Corrections System
(Zhao et al., 2019) to correct any pronoun or gender specific errors introduced by the replacements.
4.3 Ranking for Semantic Incongruity
After the grammatical error correction, the next step is to select the best context sentence from the retrieved results. Since we expect the context sentences to be incongruous with the sentence generated by the reversal of valence approach (Section 4.1), we rank the context sentences by semantic incongruity scores and select the best candidate.
We frame the problem of semantic incongruity based on the Natural Language Inference (NLI) (Bowman et al., 2015) task. The Multi-Genre NLI (Williams et al., 2018) covers a range of genres of spoken and written text, and supports a distinctive cross-genre generalization, making it an ideal choice as our NLI Dataset. We first fine-tune RoBERTa-large (Liu et al., 2019), a state-of-the-art pre-trained language model for a 3-way classification (i.e., contradiction, entailment, and neutral) by training on the Multi-NLI dataset. Next, for each retrieved sentence, we treat it as the premise and the sentence generated by the reversal of valence as the hypothesis, and thus, obtain a contradiction score from the trained model. Finally, the scores obtained for the contradiction class are used as a proxy for the degree of semantic incongruity and we select the context with the highest score. Figure 1 shows the region with light purple background as our incongruity ranking module.
4.4 Implementation Details
We use the pre-trained COMET model  ${}^{4}$  for commonsense reasoning with a greedy decoding of five to generate a commonsense phrase and return the topmost that has no lexical overlap with the input. If the generated phrase contains stopwords in the beginning we remove them. For incorporating semantic incongruity, we use the RoBERTa-large model with 355M parameters and fine-tune on MNLI. For grammatical error correction model, we use an open source pre-trained model.  ${}^{5}$
Ghosh et al. (2020) released a dataset of 4,762 pairs of speakers sarcastic messages and hearers interpretations by conducting a crowdsourcing experiment.
<sup>4</sup>https://github.com/atcbosselut/comet-commonsense
<sup>5</sup>https://github.com/zhawe01/fairseq-gec
<sup>3</sup>https://sentencedict.com/
Figure 2: Model predictions from COMET. The edges are sorted by probability
Peled and Reichart (2017) introduced a dataset of 3,000 sarcastic tweets, each interpreted by five human judges and present a novel task of sarcasm interpretation. Both datasets were collected using the hashtag #sarcasm from Twitter. We merge these two datasets and choose non-sarcastic utterances no longer than 15 words. For each literal non-sarcastic utterance we also keep the corresponding gold sarcastic message, which is useful for evaluation and comparison purposes. We randomly select 150 utterances as part of the test set (i.e., five times more than the size of the test data in Mishra et al. (2019)), while assuring such utterances do not contain high lexical overlap. We allow this constraint to evaluate how our method(s) deal with diverse data.
5.2 Systems for Experiment
Here, we benchmark the quality of the generated sarcastic messages by comparing multiple systems.
1. Full Model (FM): This model consists of all the three modules aimed at capturing reversal of valence, commonsense context, and semantic incongruity, respectively.
2. Reversal of Valence (RV): This model relies only on the reversal of valence component.
3. No Reversal of Valence (NoRV): This model only retrieves commonsense context and ranks them based on semantic incongruity.
4. No Semantic Incongruity (NSI): This model relies only on the reversal of valence and retrieval of commonsense context, without ranking based on semantic incongruity. A randomly selected retrieved sentence is used.
5. MTS2019: We make use of the model released by Mishra et al. (2019) as it is the state-of-the-art sarcasm generation system. $^6$
6. Human (Gold) Sarcasm: As described in Section 5.1, we have gold sarcasm created by humans for every non-sarcastic utterance.
5.3 Evaluation Criteria
BLEU (Papineni et al., 2002) is one of the most widely used automatic evaluation metric for generation tasks such as Machine Translation. However, for creative text generation, it is not ideal to expect significant n-gram overlaps between the machine-generated and the gold-standard utterances. Hence, we performed a human evaluation. We evaluate a total of 900 generated utterances since our ablation study consisted of six different systems with 150 utterances each.
Sarcasm is often linked with intelligence, creativity, and wit; thus we propose a set of 4 criteria to evaluate the generated output: (1) Creativity ("How creative are the utterances ?"), (2) Sarcasticness ("How sarcastic are the utterances ?"), (3) Humour ("How funny are the sentences ?") (Skalicky and Crossley, 2018), and (4) Grammaticality ("How grammatical are the sentences ?"). We design a MTurk task where Turkers were asked to rate outputs from all the six systems. Each Turker was given the non-sarcastic utterance as well as a group of sarcastic utterances generated by all the six systems (randomly shuffled). Each criteria was rated on a scale from 1 (not at all) to 5 (very). Finally, each utterance was rated by three individual Turkers. 55, 59, 66, and 60 Turkers
COMeT Predictions Graph
The model has predicted these relationships for 'zero visibility fog makes driving difficult'
$^{6}$ https://github.com/TarunTater/sarcasm_generation
7981