Venue: NAACL 2022

# Semantically Informed Slang Interpretation

Zhewei Sun\({}^{1}\), Richard Zemel\({}^{1,2,4}\), Yang Xu\({}^{1,3,4}\)

###### Abstract

Slang is a predominant form of informal language making flexible and extended use of words that is notoriously hard for natural language processing systems to interpret. Existing approaches to slang interpretation tend to rely on context but ignore semantic extensions common in slang word usage. We propose a semantically informed slang interpretation (SSI) framework that considers jointly the contextual and semantic appropriateness of a candidate interpretation for a query slang. We perform rigorous evaluation on two large-scale online slang dictionaries and show that our approach not only achieves state-of-the-art accuracy for slang interpretation in English, but also does so in zero-shot and few-shot scenarios where training data is sparse. Furthermore, we show how the same framework can be applied to enhancing machine translation of slang from English to other languages. Our work creates opportunities for the automated interpretation and translation of informal language.

+
Footnote †: _Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_, pages 5213 - 5231

[FOOTNOTE:2][EN

(Radford et al., 2019) based language infill model Donahue et al. (2020). Each of these words can be considered a candidate paraphrase for the target slang _steamed_ conditioned on its surrounding words. Although the groundtruth meaning "angry" is among the list of top candidates, this model infers "sick" as the most probable interpretation. A similar context-based approach has been explored in a previous study led by Ni and Wang (2017) showing that a sequence-to-sequence model trained directly on a large number of pairs of slang-contained sentences along with their corresponding definitions from Urban Dictionary can be a useful starting point toward the automated interpretation of slang.

We present an alternative approach to slang interpretation that builds on but goes beyond the context-based models. Inspired by recent work on generative models of slang Sun et al. (2019, 2021), we consider slang interpretation to be the inverse process of slang generation and propose a semantically informed framework that takes into account both contextual information and knowledge about slang meaning extensions (e.g., "vapor"\(\rightarrow\)"angry") in inferring candidate interpretations. Our framework incorporates a semantic model of slang that uses contrastive learning to capture semantic extensions that link conventional and slang meanings of words Sun et al. (2021). Under this framework, meanings that are otherwise far apart can be brought close, resulting in a semantic space that is sensitive to the flexible extended usages of slang. Rather than using this learned semantic space to generate novel slang usages, we apply it to the inverse problem of slang interpretation by checking whether a candidate interpretation may be suitably expressed as a slang using the to-be-interpreted slag expression. For example, "sick" and "angry" can both replace the slang _steamed_ in a given context, but "angry" may be a more appropriate meaning to be expressed using _steamed_ in the slang context. As such, we build a computational framework that takes into account the semantic knowledge of words as well as the context of slang in the interpretation process.

Figure 2 illustrates the workflow of our approach. We begin with a set of candidate interpretations informed by a context-based model (e.g., a language infill model), where the set would contain a list of possible meanings that fit reasonably in the given context. We then rerank this set of candidate interpretations by selecting the meaning that is most likely to be extended as slang from the to-be-interpreted slang expression.

For the scope of this work, we focus on interpreting slang expressions with existing word forms because extensive studies in slang have suggested that a high proportion of slang usages relies on the extended reuse of existing word forms Warren (1992); Green (2010); Eble (2012). We show that our framework can enhance state-of-the-art language models in slang interpretation in English and slang translation from English to other languages.1

Footnote 1: Code and data available at: [https://github.com/zhewei-sun/slanginterp](https://github.com/zhewei-sun/slanginterp)

## 2 Related Work

### Natural Language Processing for Slang

Existing approaches in the natural language processing for slang focus on efficient construction, extension, and retrieval from dictionary-based resources for detection Pal and Saha (2013); Dhuliawala et al. (2016), interpretation Gupta et al. (2019), and sentiment analysis of slang Dhuliawala et al. (2016); Wu et al. (2018). These studies often rely on heuristic measures to determine or retrieve the meaning of slang and cannot generalize beyond what was available in the training data. Recent work such as Kulkarni and Wang (2018) and Pei et al. (2019) proposed deep learning based approaches to generalize toward unseen slang.

Closely related to our study is Ni and Wang (2017) that formulated English slang interpretation as a translation task (although they did not tackle slang machine translation _per se_). In this work, each slang query sentence in English is paired with the groundtruth slang definition (also in English), and such pairs are fed into a translation model. In addition, the spellings of slang word forms are also considered as input. In their model, both the context and the slang form are encoded using separate LSTM encoders. The two encoded representations are then linearly combined to form the encoded input for a sequence-to-sequence network Sutskever et al. (2014). During training, the combined state is passed onto an LSTM decoder to train against

Figure 2: Workflow of the proposed framework.

 

the corresponding definition sentence. During test time, beam search (Graves, 2012) is applied to decode a set of candidate definition sentences.

One key problem with this approach is that the Dual Encoder tends to rely on the contextual features surrounding the target slang but does not model flexible meaning extensions of the slang word itself. Similar issues are present in a language-model based approach, whereby one can use an infill model to infer the meaning of a target slang based solely on its surrounding words. Our work extends these context-based approaches by jointly considering the contextual and semantic appropriateness of a slang expression in a sentence, using generative semantic models of slang.

### Generative Semantic Models of Slang

Recent work by Sun et al. (2019, 2021) proposed a neural-probabilistic generative framework for modeling slang word choice in novel context. Given a query sentence with the target slang blanked out and the intended meaning of that slang, their framework predicts which word(s) would be appropriate slang choices that fill in the blank. Relevant to their framework is a semantic model of slang that uses contrastive learning from Siamese networks (Baldi and Chauvin, 1993; Bromley et al., 1994) to relate conventional and slang meanings of words. This model yields a semantic embedding space that is sensitive to flexible slang meaning extensions. For example, it may learn that meanings associated with "vapor" can extend to meanings about "angry" (as in the _steamed_ example in Figure 1).

Differing from slang generation, our work concerns the inverse problem of slang interpretation that has more direct applications in natural language processing particularly machine translation (e.g., of informal language). Building on work of slang generation, we incorporate the generative semantic model of slang in a semantically informed interpretation framework that integrates context to infer the intended meaning of a target slang.

## 3 Computational Framework

Our computational framework is comprised of three key components following the workflow illustrated in Figure 2: 1) A context-based baseline interpreter that generates an n-best list of candidate interpretations for a target slang in a query sentence; 2) A semantic model of slang that checks the appropriateness of a candidate interpretation to the slang context; 3) A reranker informed by the semantic model in 2) that re-prioritizes the candidate interpretations from the context-based interpreter in 1). We use this framework for both interpreting slang within English and translating slang from English to other languages.

### Context-based Interpretation

We define slang interpretation formally as follows. Given a target slang term \(S\) in context \(C_{S}\) of a query sentence, interpret the meaning of \(S\) by a definition \(M\). The context is an important part of the problem formulation since a slang term \(S\) may be polysemous and context can be used to constrain the interpretation of its meaning. We define a slang interpreter \(I\) probabilistically as:

\[I(S,C_{S})=\operatorname*{arg\,max}_{M}P(M|S,C_{S}) \tag{1}\]

Given this formulation, we retrieve an n-best list of candidate interpretations \(\mathcal{K}\) (i.e., \(|\mathcal{K}|=n\)) based on an interpretation model of choice \(P(M|S,C_{S})\). Here, we consider two alternative models for \(P(M|S,C_{S})\): 1) a language-model (LM) based approach that treats slang interpretation as a cloze task, and 2) a sequence-to-sequence based approach similar to work by Ni and Wang (2017).

LM-based interpreter.The first model we consider is a language infill model in a cloze task, in which the model itself is based on large pre-trained language models such as GPT-2 (Radford et al., 2019). Although slang expressions may make sporadic appearances during training, this model is not trained specifically on a slang related task and thus serves as a baseline that reflects the state-of-the-art language-model based NLP systems (e.g., Donahue et al., 2020).

Given context \(C_{S}\) containing target slang \(S\), we blank out \(S\) in the context and ask the language infill model to infer the most likely words to fill in the blank. This results in a probability distribution \(P(w|C_{S}\backslash S)\) over candidate words \(w\). The infilled words can then be viewed as candidate interpretations of the slang \(S\):

\[I(S,C_{S})= D[\operatorname*{arg\,max}_{w}LM(w|C_{S}\backslash S)\] \[+\mathbbm{1}_{T(w)}[T(C_{S}\backslash S)]] \tag{2}\]

Here, \(D\) is a dictionary lookup function that maps a candidate word \(w\) to a definition sentence. In this case, we constrain the space of meanings considered to the set of all meanings corresponding 

to words in the lexicon. Additionally, we apply a Part-of-Speech (POS) tagger \(T\) to check whether the candidate word \(w\) shares the same POS tag as the blanked-out word in the usage context. Words that share the same POS tags are preferred in the list of n-best retrievals.

This baseline approach by itself does not take into account any (semantic) information from the target slang \(S\). In the case where two distinctive slang terms may be placed in the same context, the model would generate the exact same output. However, this LM based approach does not require task-specific data to train. We show later that by reranking language model outputs, it is possible to achieve state-of-the-art performance using much less on-task data than existing approaches.

Dual encoder.Ni and Wang (2017) partly addressed the context-only limitation by encoding the slang term using a character-level recurrent neural network in an end-to-end model inspired by the sequence-to-sequence architecture for neural machine translation Sutskever et al. (2014). We implement their dual encoder architecture as an alternative context-based interpreter to LM. In this model, separate LSTM encoders are applied on the context \(C_{S}\) and the character encoding of the to-be-interpreted slang \(S\) respectively. The two encoders are then linearly combined using learned parameters. The combined state is passed onto an LSTM decoder to train against the corresponding definition sentence in Urban Dictionary (as in the original work of Ni and Wang 2017). For inference, beam search Graves (2012) is applied to decode an n-best list of candidate definition sentences.

While this approach is trained directly on slang data and considers the slang word forms, it requires a large on-task dataset to be trained effectively. This model also does not take into account the appropriateness of meaning extension in slang usage. We next describe how a semantic model of slang can be incorporated to enhance the context-based interpreters.

### Semantic Model of Slang

Given an n-best list of candidate interpretations \(\mathcal{K}\) for the target slang \(S\) in context \(C_{S}\), we wish to model the semantic plausibility of each candidate interpretation \(k\in\mathcal{K}\). Specifically, we ask how likely one would relate the (conventional meaning of) target slang expression \(S\) to a candidate interpretation \(k\). Sun et al. (2019, 2021) modeled the relationship between a to-be-expressed meaning and a word form using the prototype model Rosch (1975); Snell et al. (2017). We adapt this model in the context of slang interpretation:

\[f(k,S) =sim(E_{k},E_{S})\] \[=\exp(-\frac{d(E_{k},E_{S})}{h_{m}}) \tag{3}\]

\(E_{k}\) is an embedding for a candidate interpretation \(k\) and \(E_{S}\) is the prototypical conventional meaning of \(S\) computed by averaging the embeddings of its conventional meanings in dictionary (\(\mathcal{E}_{S}\)):

\[E_{S}=\frac{1}{|\mathcal{E}_{S}|}\sum_{E_{S_{i}}\in\mathcal{E}_{S}}E_{S_{i}} \tag{4}\]

The similarity function \(f\) can then be computed by taking the negative exponential of the Euclidean distance between the two resulting semantic embeddings. \(h_{m}\) is a kernel width hyperparameter.

Following Sun et al. (2021), we learn semantic embeddings \(E_{k}\) and \(E_{S_{i}}\) under a max-margin triplet loss scheme, where embeddings of slang sense definitions (\(E_{SL}\)) are brought close in Euclidean space to those of their conventional sense definitions (\(E_{P}\)) yet kept apart from irrelevant word senses (\(E_{N}\)) by a pre-specified margin \(m\):

\[Loss=\Big{[}d(E_{SL},E_{P})-d(E_{SL},E_{N})+m\Big{]}_{+} \tag{5}\]

The resulting contrastive sense encodings are shown to be sensitive to slang semantic extensions that have been observed during training. We leverage this knowledge to check whether pairing a candidate interpretation \(k\) with the slang expression \(S\) is likely given the common semantic extensions observed in slang usages.

### Semantically Informed Reranking

We define a semantic scorer \(g\) over the set of candidate interpretations \(\mathcal{K}\) and the to-be-interpreted slang \(S\). The candidates are reranked based on the resulting scores to obtain semantically informed slang interpretations (SSI):

\[SSI(\mathcal{K})=\arg\max g(k,S) \tag{6}\]

We define \(g(\mathcal{K},S)\) as a score distribution over the set of candidates \(\mathcal{K}\) given slang \(S\), where each score is computed by checking the semantic appropriateness of a candidate meaning \(k\in\mathcal{K}\) with respect to 

target slang \(S\) by querying the semantic model \(f\) from Equation 3:

Footnote 1: [https://github.com/google-learning/](https://github.com/google-learning/)

\[g(k,S)=P(k|S)\propto f(k,S) \tag{7}\]

In addition, we apply collaborative filtering Goldberg et al. (1992) to account for a small neighborhood of words \(L(S)\) akin to the slang expression \(S\) in conventional meaning:

\[g^{*}(k,S)\propto\sum_{S^{\prime}\in L(S)}sim(S,S^{\prime})g(k, S^{\prime}) \tag{8}\] \[sim(S,S^{\prime})=\exp(-\frac{d(S,S^{\prime})}{h_{cf}}) \tag{9}\]

Here, \(d(S,S^{\prime})\) is the cosine distance between the two slang's word vectors and \(h_{cf}\) is a hyperparameter controlling the kernel width. The collaborative filtering step encodes intuition from studies in historic semantic change that similar words tend to extend to express similar meanings Lehrer (1985); Xu and Kemp (2015), which was found to extend well in the case of slang Sun et al. (2019, 2021).

## 4 Datasets

We use two online English slang dictionary resources to train and evaluate our proposed slang interpretation framework: 1) the Online Slang Dictionary (OSD)2 dataset from Sun et al. (2021) and 2) a collection of Urban Dictionary (UD)3 entries from 1999 to 2014 collected by Ni and Wang (2017). Each dataset contains slang gloss entries including a slang's word form, its definition, and at least one corresponding example sentence containing the slang term. We use the same training and testing split provided by the original authors and only use entries where a corresponding non-informal entry can be found in the online version of the Oxford Dictionary (OD) for English4, which allows the retrieval of conventional senses for all slang expressions considered. We also filter out entries where the example usage sentence contains none or more than one exact references of the corresponding slang expression. When a definition entry has multiple example usage sentences, we treat each example sentence as a separate data entry, but all data entries corresponding to the same definition entry will only appear in the same data split. Table 1 shows the size of the datasets after pre-processing. While OSD contains higher quality entries, UD offers a much larger dataset. We thus use OSD to evaluate model performance in a low resource scenario and UD for evaluation of larger neural network based approaches.

Footnote 2: OSD: [http://onlineslangdictionary.com](http://onlineslangdictionary.com)

Footnote 3: UD: [https://www.urbandictionary.com](https://www.urbandictionary.com)

Footnote 4: OD: [https://en.oxforddictionaries.com](https://en.oxforddictionaries.com)

## 5 Evaluation and Results

### Evaluation on Slang Interpretation

We first evaluate the semantically informed and baseline interpretation models in a multiple choice task. In this task, each query is paired with a set of definitions that construe the meaning of the target slang in the query. One of these definitions is the groundtruth meaning of the target slang, while the other definitions are incorrect or negative entries sampled from the training set (i.e., all taken from the slang dictionary resources described). To score a model, each definition sentence is first compared with the model-predicted definition by computing the Euclidean distance between their respective Sentence-BERT Reimers and Gurevych (2019) embeddings. The ideal model should produce a definition that is semantically closer to the groundtruth definition, more so than the other competing negatives. For each dataset, we sample two sets of negatives. The first set of negative candidates contains only definition sentences from the training set that are distinct from the groundtruth definition. We consider two definition sentences to be distinct if the overlap in the number of content words is less than 50%. The other set of negative definitions is sampled randomly. We measure the performance of the models by computing the standard mean reciprocal rank (MRR) of the groundtruth definition's rank when checked against 4 other sampled negative definitions.

We train the semantic reranker on all definition entries in the respective training sets from the two data resources. When training the Dual Encoder, we use 400,431 out-of-vocabulary slang entries (i.e., entries with a slang expression that does not contain a corresponding lexical entry in the standard dictionary) from UD in addition to the invocabulary entries used to train the reranker. This is necessary since the baseline Dual Encoder performs poorly without a large number of training entries. Similarly, training the Dual Encoder directly on the OSD training set does not result in an adequate model for comparison. We instead train the Dual Encoder on all UD entries and experiment with the resulting interpreter on OSD. Any UD 

entries corresponding to words found in the OSD testset are filtered out in this particular experiment. Detailed training procedures for all models can be found in Appendix A.

Table 2 summarizes the multiple-choice evaluation results on both slang datasets. In all cases, applying the semantically informed slang interpretation framework improves the MRR of the respective baselines under both types of negative candidate sampling. On the UD evaluation, even though the language infill model (LM Infill) is not trained on this specific task, LM infall based SSI is able to select better and more appropriate interpretations than the dual encoder baseline, which is trained specifically on slang interpretation with more than 7 times the number of definition entries for training. We also find that while increasing the beam size (specified by \(n\)) in the sequence-to-sequence based Dual Encoder model impairs its performance, SSI can take advantage of the additional variation in the generated candidates and outperform its counterpart with a smaller beam size.

Table 3 provides example interpretations predicted by the models. The _lit_ example shows a case where the semantically informed models were able to correctly pinpoint the intended definition, among alternative definitions that describe individuals. The _lush_ example suggests that the SSI model is not perfect and points to common errors made by the model including predicting definitions that are more general and applying incorrect semantic extensions. In this case, the model predicts the slang _lush_ to mean "something that is not cool" because polarity shift is a common pattern in slang usage Eble (2012), even though the groundtruth definition does not make such a polarity shift in this specific example.

Note that the improvement brought by SSI is less prominent in the OSD experiment where the Dual Encoder trained on UD was used. This is

\begin{table}
\begin{tabular}{p{56.9pt} p{85.4pt} p{85.4pt} p{85.4pt} p{85.4pt} p{85.4pt}} Dataset & \# of unique & \# of slang & \# of context & \# of definitions & \# of context sentences \\  & slang word forms & definition entries & sentences & in the test set & in the test set \\ \hline OSD & 1,635 & 2,979 & 3,718 & 299 & 405 \\ UD & 9,474 & 65,478 & 65,478 & 1,242 & 1,242 \\ \end{tabular}
\end{table}
Table 1: Summary of basic statistics for the two online slang dictionaries used in the study.

\begin{table}
\begin{tabular}{c c c c} Model & & Distinctively & Randomly \\  & & sampled & sampled \\  & & candidates & candidates \\ \hline Dataset 1: Online Slang Dictionary (OSD) Sun et al. (2021) & & & \\ Language Infill Model (LM Infill) Donahue et al. (2020), \(n\) = 50 & 0.532 & 0.502 \\  & + Semantically Informed Slang Interpretation (SSI) & **0.557** & **0.563** \\ Dual Encoder* Ni and Wang (2017), \(n\) = 5 & 0.584 & 0.583 \\  & & **0.592** & **0.588** \\ Dual Encoder*, \(n\) = 50 & 0.568 & 0.602 \\  & & **0.616** & **0.607** \\ \multicolumn{4}{p{56.9pt}}{* Dual Encoders trained on UD data after filtering out slang in OSD test set.} \\ \hline Dataset 2: Urban Dictionary (UD) Ni and Wang (2017) & & & \\ LM Infill, \(n\) = 50 & 0.517 & 0.521 \\  & & **0.569** & **0.579** \\ Dual Encoder, \(n\) = 5 & 0.556 & 0.555 \\  & & **0.573** & **0.572** \\ Dual Encoder, \(n\) = 50 & 0.547 & 0.550 \\  & & **0.582** & **0.584** \\ \end{tabular}
\end{table}
Table 2: Evaluation of English slang interpretation measured in mean-reciprocal rank (MRR). Predictions are ranked against 4 negative candidates distinctively or randomly sampled, yielding MRR\(=\)0.457 for the random baseline.

 

expected because the Dual Encoder is trained to generate definition sentences in the style of UD entries, whereas the SSI is trained on OSD definition sentences instead. The mismatch in style between the two datasets might have caused the difference in performance gain.

### Zero-shot and Few-shot Interpretation

Recent studies in deep learning have shown that large neural network based models such as GPT-3 excel at learning new tasks in a few-shot learning setting Brown et al. (2020). We examine to what extent the superior performance of our SSI framework may be affected by fine-tuning the LM baseline model in zero-shot and few-shot scenarios. We finetune the language infill model (LM Infill) on the first example usage sentence that correspond to each definition entry in the OSD dataset, resulting in 2,979 sentences. Given an example sentence, we mask out the slang expression and train the language infill model to predict the corresponding slang term. We randomly shuffle all examples and finetune LM Infill for one epoch. We then compare the resulting model with the off-the-shelf LM using examples in the test set that were not used in fine-tuning (i.e., entries with usage sentences that do not correspond to the first example usage sentence of a definition entry). This results in 106 novel examples for evaluation.

Table 4 shows the result of this experiment. While finetuning does improve test performance (a 6 point gain in MRR), it remains beneficial to consider semantic information in slang context. In both the zero-shot and the few-shot cases, SSI brings significant performance gain even though SSI itself is only trained on entries from the training set.

### Evaluation on Slang Translation

We next apply the slang interpretation framework to neural machine translation. Existing machine translation systems have difficulty in translating source sentences containing slang usage partly because they lack the ability to properly decode the intended slang meaning. We make a first attempt in addressing this problem by exploring whether machine interpretation of slang can lead to better translation of slang. Given a source English sentence containing a slang expression \(S\), we apply the LM based slang interpreters to generate a paraphrased word to replace \(S\). The paraphrased sentence would then contain the intended meaning of the slang in its literal form. Here, we take advantage of the LM-based approaches' ability to directly generate a paraphrase instead of a definition sentence (i.e., without dictionary lookup \(D\) in Equation 2), which allows direct insertion of the resulting interpretation into the original sentence.

\begin{table}
\begin{tabular}{l c r} Model & Distinct & Random \\  & negatives & negatives \\ \hline LM Zero-shot, \(n\) = 50 & 0.444 & 0.443 \\   + SSI & **0.571** & **0.565** \\ LM Few-shot, \(n\) = 50 & 0.504 & 0.513 \\   + SSI & **0.567** & **0.564** \\ \end{tabular}
\end{table}
Table 4: Interpretation results on OSD measured in mean-reciprocal rank (MRR) before and after finetuning the language infill model.

\begin{table}
\begin{tabular}{p{85.4pt} p{113.8pt}} Query (target slang in _bold italic_): & That chick is _lit!_! \\ Groundtruth definition of target slang: & Attractive. \\ LM Infill baseline prediction: & Cute, beautiful, adorable. \\ LM Infill + SSI prediction: & Hot, cool, fat. \\ Dual Encoder baseline prediction: & Another word for bitch. \\ Dual Encoder + SSI prediction: & Word used to describe someone who is very attractive. \\ \hline Query: & That Louis Vuitton purse is _hash!_ \\ Groundtruth definition of target slang: & High quality, luxurious. (British slang.) \\ LM Infill baseline prediction: & Amazing, beautiful, unique. \\ LM Infill + SSI prediction: & Lovely, stunning, expensive. \\ Dual Encoder baseline prediction: & Something that is cool or awesome. \\ Dual Encoder + SSI prediction: & An adjective used to describe something that is not cool. \\ \end{tabular}
\end{table}
Table 3: Example queries from OSD and top predictions made from both the baseline language infill models (LM Infill) and the Dual Encoder models with \(n\) = 50, along with top predictions from the enhanced semantically informed slang interpretation (SSI) models. Additional examples can be found in Appendix B.1.

 

We perform our experiment on the OSD test set because it contains higher quality example sentences than UD. To mitigate potential biases, we consider only entries that correspond to single word slang expressions, and that the slang has not been seen during training (where the slang attaches to a different slang meaning than the one in the test set). For the remaining 102 test entries, we obtain gold-standard translations by first manually replacing the slang word in the example sentence with its intended definition, condensed to a word or short phrase to fit into the context sentence. We then translate the sentences to French and German using machine translation.

We make all machine translations using pretrained 6-layer transformer networks Vaswani et al. (2017) from MarianMT Tiedemann and Thottingal (2020), which are trained on a collection of web-based texts in the OPUS dataset Tiedemann (2012). Here, we select models pre-trained on web-based texts to maximize the baseline model's ability to correctly process slang. We evaluate the translated sentences using three metrics: 1) Sentence-level BLEU scores Papineni et al. (2002) computed using _sentence_bleu_ implementation from NLTK Bird et al. (2009) with smoothing (_method4_ in NLTK, Chen and Cherry (2014) to account for sparse n-gram overlaps; 2) BLEURT scores Sellam et al. (2020) computed using the pre-trained _BLEURT-20_ checkpoint; 3) COMET scores Rei et al. (2020) computed using the pre-trained _wmt20-comet-da_checkpoint. For COMET scores, we replace slang expressions in the source sentences with their literal equivalents to reduce confusion that the COMET model might have on slang.

Figure 3 summarizes the results. Overall, the semantically informed approach tends to outperform the baseline approaches for the range of top retrievals (from 1 to 20) under all three metrics considered, with the exception of BLEURT evaluated on German where the semantically informed approach gives very similar performance as the

Figure 3: Translation scores of translated sentences with the slang replaced by n-best interpretations. Curves show sentence-level BLEU, BLEURT, and COMET scores of the best translation within the top-n retrievals. Aggregate scores integrated over the first 20 retrievals are shown in parenthesis. Baselines are obtained by directly translating the original sentence containing slang.

 

language model baseline. While not all predicted interpretations correspond to the groundtruth definitions, the set of interpreted sentences often contain plausible interpretations that result in improved translation of slang. Table 5 provides some example translations. We observe that quality translations can be found reliably with a small number of interpretation retrievals (i.e., around 5) and the quality generally improves as we retrieve more candidate interpretations. Our approach may be ultimately integrated with a slang detector (e.g., Pei et al.2019) to produce fully automated translations in natural context that involves slang.

## 6 Conclusion

The flexible nature of slang is a hallmark of informal language, and to our knowledge we have presented the first principled framework for automated slang interpretation that takes into account both contextual information and knowledge about semantic extensions of slang usage. We showed that our framework is more effective in interpreting and translating the meanings of English slang terms in natural sentences in comparison to existing approaches that rely more heavily on context to infer slang meaning.

Future work in this area may benefit from principled approaches that model the coinage of slang expressions with novel word forms and multi-word expressions with complex formation strategies, as well as how slang terms emerge in specific individuals and groups. Our current study shows promise for advancing methodologies in informal language processing toward these avenues of future research.

## Ethical Considerations

We analyze entries of slang usage in our work and acknowledge that such usages may contain offensive information. We retain such entries in our datasets to preserve the scientific validity of our results, as a significant portion of slang usage aligns to possibly offensive usage context. In the presentation our of results, however, we strive to select examples or illustrations that minimize the extent to which offensive content is represented. We also acknowledge that models trained on datasets such as the Urban Dictionary have a greater tendency to generate offensive language. All model outputs shown are results of model learning and do not reflect opinions of the authors and their affiliated organizations. We hope that our work will contribute to the greater good by enhancing AI system's ability to comprehend such offensive language use, allowing better filtering of online content that may be potentially harmful.

## Acknowledgements

We thank the ARR reviewers for their constructive comments and suggestions, and Walter Rader for

\begin{table}
\begin{tabular}{l l} Query (target slang in _bold italic_): & I want to go get coffee but it’s _bitter_ outside. \\ Definition of target slang: & Abbreviated form of bitterly cold. \\ Groundtruth interpreted sentence: & I want to go get coffee but it’s _bitterly cold_ outside. \\ Original query sentence translation: & Je veux aller prendre un cafe mais c’est amer dehors. \\ Gold-standard translation: & Je veux aller prendre un cafe, mais il fait tres froid debors. \\ LM Infill interpretation \& translation: \\ (1) I want to go get coffee but it’s _raining_ outside. & Je veux aller prendre un cafe mais il _pleut_ dehors. \\ (2) I want to go get coffee but it’s _closed_ outside. & Je veux aller prendre un cafe mais il _et fermé_ dehors. \\ LM Infill + SSI interpretation \& translation: \\ (1) I want to go get coffee but it’s _cold_ outside. & Je veux aller prendre un cafe, mais il fait _froid_ dehors. \\ (2) I want to go get coffee but it’s _warm_ outside. & Je veux aller prendre un cafe mais il fait _chaud_ dehors. \\  & (BLEU: 78.1, BLEURT: 79.1, COMET: 1.12) \\ \end{tabular}
\end{table}
Table 5: An example of machine translation of slang, without or with the application of the SSI framework. The top 2 interpreted and translated sentences are shown for each model with BLEU, BLEURT, and COMET scores against the gold-standard translation shown in parentheses. More examples can be found in Appendix B.4.

 

permission to use The Online Slang Dictionary. This work was supported by a NSERC Discovery Grant RGPIN-2018-05872, a SSHRC Insight Grant #435190272, and an Ontario ERA Award to YX.

## References

* Akbik et al. (2018) Alan Akbik, Duncan Blythe, and Roland Vollgraf. 2018. Contextual string embeddings for sequence labeling. In _COLING 2018, 27th International Conference on Computational Linguistics_, pages 1638-1649.
* Baldi and Chauvin (1993) Pierre Baldi and Yves Chauvin. 1993. Neural networks for fingerprint recognition. _Neural Computation_, 5(3):402-418.
* Bird et al. (2009) Steven Bird, Ewan Klein, and Edward Loper. 2009. _Natural language processing with Python: analyzing text with the natural language toolkit_. " O'Reilly Media, Inc..".
* Bojanowski et al. (2017) Piotr Bojanowski, Edouard Grave, Armand Joulin, and Tomas Mikolov. 2017. Enriching word vectors with subword information. _Transactions of the Association for Computational Linguistics_, 5:135-146.
* Braun and Kitzinger (2001) Virginia Braun and Celia Kitzinger. 2001. "Snatch," "Hole," "Honey-pot?" Semantic categories and the problem of nonspecificity in female genital slang. _The Journal of Sex Research_, 38(2):146-158.
* Bromley et al. (1994) Jane Bromley, Isabelle Guyon, Yann LeCun, Eduard Sackinger, and Roopak Shah. 1994. Signature verification using a "siamese" time delay neural network. In J. D. Cowan, G. Tesauro, and J. Alspector, editors, _Advances in Neural Information Processing Systems 6_, pages 737-744. Morgan-Kaufmann.
* Brown et al. (2020) Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020. Language models are few-shot learners. In _Advances in Neural Information Processing Systems_, volume 33, pages 1877-1901. Curran Associates, Inc.
* Chen and Cherry (2014) Boxing Chen and Colin Cherry. 2014. A systematic comparison of smoothing techniques for sentence-level BLEU. In _Proceedings of the Ninth Workshop on Statistical Machine Translation_, pages 362-367, Baltimore, Maryland, USA. Association for Computational Linguistics.
* Dhuliawala et al. (2016) Shehzaad Dhuliawala, Diptesh Kanojia, and Pushpak Bhattacharyya. 2016. SlangNet: A WordNet like resource for English slang. In _Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC 2016)_, pages 4329-4332, Portoroz, Slovenia. European Language Resources Association (ELRA).
* Donahue et al. (2020) Chris Donahue, Mina Lee, and Percy Liang. 2020. Enabling language models to fill in the blanks. In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_, pages 2492-2501, Online. Association for Computational Linguistics.
* Eble (2012) Connie C Eble. 2012. _Slang & sociability: In-group language among college students_. University of North Carolina Press, Chapel Hill, NC.
* Goldberg et al. (1992) David Goldberg, David Nichols, Brian M. Oki, and Douglas Terry. 1992. Using collaborative filtering to weave an information tapestry. _Commun. ACM_, 35:61-70.
* Graves (2012) Alex Graves. 2012. Sequence transduction with recurrent neural networks. In _International Conference of Machine Learning (ICML) 2012 Workshop on Representation Learning_.
* Green (2010) Jonathan Green. 2010. _Green's Dictionary of Slang_. Chambers, London.
* Gupta et al. (2019) Anshita Gupta,anya Bathla Taneja, Garima Malik, Sonakshi Vij, Devendra K. Tayal, and Amita Jain. 2019. Slangzy: a fuzzy logic-based algorithm for english slang meaning selection. _Progress in Artificial Intelligence_, 8(1):111-121.
* Hamilton et al. (2016) William L. Hamilton, Jure Leskovec, and Dan Jurafsky. 2016. Diachronic word embeddings reveal statistical laws of semantic change. In _Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_, pages 1489-1501, Berlin, Germany. Association for Computational Linguistics.
* Kingma and Ba (2015) Diederik P. Kingma and Jimmy Ba. 2015. Adam: A method for stochastic optimization. In _3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings_.
* Kulkarni and Wang (2018) Viwek Kulkarni and William Yang Wang. 2018. Simple models for word formation in slang. In _Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)_, pages 1424-1434, New Orleans, Louisiana. Association for Computational Linguistics.
* Lehrer (1985) Adrienne Lehrer. 1985. The influence of semantic fields on semantic change. _Historical Semantics: Historical Word Formation_, 29:283-296.
* Mattiello (2009) Elisa Mattiello. 2009. Difficulty of slang translation. In _Translation Practices_, pages 65-83. Brill Rodoppi.

 

* Ni and Wang (2017) Ke Ni and William Yang Wang. 2017. Learning to explain non-standard English words and phrases. In _Proceedings of the Eighth International Joint Conference on Natural Language Processing (Volume 2: Short Papers)_, pages 413-417, Taipei, Taiwan. Asian Federation of Natural Language Processing.
* Pal and Saha (2013) Alok Ranjan Pal and Diganta Saha. 2013. Detection of slang words in e-data using semi-supervised learning. _International Journal of Artificial Intelligence and Applications_, 4(5):49-61.
* Papineni et al. (2002) Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. BLEU: A method for automatic evaluation of machine translation. ACL '02, page 311-318, USA. Association for Computational Linguistics.
* Pei et al. (2019) Zhengqi Pei, Zhewei Sun, and Yang Xu. 2019. Slang detection and identification. In _Proceedings of the 23rd Conference on Computational Natural Language Learning (CoNLL)_, pages 881-889, Hong Kong, China. Association for Computational Linguistics.
* Radford et al. (2019) Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language models are unsupervised multitask learners. _OpenAI blog_, 1(8):9.
* Rehurek and Sojka (2011) Radim Rehurek and Petr Sojka. 2011. Gensim-python framework for vector space modelling. _NLP Centre, Faculty of Informatics, Masaryk University, Brno, Czech Republic_, 3(2).
* Rei et al. (2020) Ricardo Rei, Craig Stewart, Ana C Farinha, and Alon Lavie. 2020. COMET: A neural framework for MT evaluation. In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_, pages 2685-2702, Online. Association for Computational Linguistics.
* Reimers and Gurevych (2019) Nils Reimers and Iryna Gurevych. 2019. Sentence-BERT: Sentence embeddings using Siamese BERT-networks. In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_, pages 3982-3992, Hong Kong, China. Association for Computational Linguistics.
* Rosch (1975) Eleanor Rosch. 1975. Cognitive representations of semantic categories. _Journal of Experimental Psychology: General_, 104:192-233.
* Sellam et al. (2020) Thibault Sellam, Dipanjan Das, and Ankur Parikh. 2020. BLEURT: Learning robust metrics for text generation. In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_, pages 7881-7892, Online. Association for Computational Linguistics.
* Snell et al. (2017) Jake Snell, Kevin Swersky, and Richard S. Zemel. 2017. Prototypical networks for few-shot learning. In _Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, 4-9 December 2017, Long Beach, CA, USA_, pages 4080-4090.
* Sun et al. (2019) Zhewei Sun, Richard Zemel, and Yang Xu. 2019. Slang generation as categorization. In _Proceedings of the 41st Annual Conference of the Cognitive Science Society_, pages 2898-2904. Cognitive Science Society.
* Sun et al. (2021) Zhewei Sun, Richard Zemel, and Yang Xu. 2021. A computational framework for slang generation. _Transactions of the Association for Computational Linguistics_, 9:462-478.
* Sutskever et al. (2014) Ilya Sutskever, Oriol Vinyals, and Quoc V Le. 2014. Sequence to sequence learning with neural networks. In _Advances in Neural Information Processing Systems_, volume 27, pages 3104-3112. Curran Associates, Inc.
* Tiedemann (2012) Jorg Tiedemann. 2012. Parallel data, tools and interfaces in OPUS. In _Proceedings of the Eighth International Conference on Language Resources and Evaluation (LREC'12)_, pages 2214-2218, Istanbul, Turkey. European Language Resources Association (ELRA).
* building open translation services for the world. In _Proceedings of the 22nd Annual Conference of the European Association for Machine Translation_, pages 479-480, Lisboa, Portugal. European Association for Machine Translation.
* Vaswani et al. (2017) Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, undefinedukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In _Proceedings of the 31st International Conference on Neural Information Processing Systems_, NIPS'17, page 6000-6010, Red Hook, NY, USA. Curran Associates Inc.
* Warren (1992) Beatrice. Warren. 1992. _Sense Developments: A Contrastive Study of the Development of Shang Senses and Novel Standard Senses in English_. Acta Universitatis Stockholmensis: Stockholm Studies in English. Almqvist & Wiksell International.
* Wu et al. (2018) Liang Wu, Fred Morstatter, and Huan Liu. 2018. SlangSD: Building, expanding and using a sentiment dictionary of slang words for short-text sentiment classification. _Lang. Resour. Eval._, 52(3):839-852.
* Xu and Kemp (2015) Yang Xu and Charles Kemp. 2015. A computational evaluation of two laws of semantic change. In _Proceedings of the 37th Annual Conference of the Cognitive Science Society_, pages 2703-2708. Cognitive Science Society.

 

Training Procedures

### Baseline Models

We train two context-based slang interpreters described in Section 3.1 as our baseline models. For the LM-based interpreter, we use a pre-trained language infill model from Donahue et al. (2020) based on the GPT-2 Radford et al. (2019) architecture. Here, we obtain the n-best list of interpretations by retrieving the list of infilled words with the highest infill probability. Words containing non-alphanumeric characters are filtered out. For the dictionary lookup function \(D\) in Equation 2, if a matching dictionary entry can be found in Oxford Dictionary (OD), the top definition sentence is retrieved as the definition sentence for the input word. Otherwise, the word itself is used as the definition. In addition to the word's original form, we apply lemmatization or stemming to the original form using NLTK Bird et al. (2009) to find matching dictionary entries. To check for Part-of-Speech (POS) tags, we apply the Flair tagger Akbik et al. (2018) on the context sentence with the slang expression replaced by a mask token and use counts from Histwords Hamilton et al. (2016) to determine POS tags for individual words.

To train the Dual Encoder, we use LSTM encoders with 256 and 1024 hidden units to encode a slang expression's spelling and its usage context respectively, with 100 and 300 dimensional input embeddings for the characters and words respectively. Following Ni and Wang (2017), we use random initialization for the input embeddings and use stochastic gradient descent (SGD) with an adaptive learning rate. We train the model for 20 epochs beginning with a learning rate of 0.1 and add an exponential decay of 0.9 every epoch. We reserve 5% of the training examples as a development set for hyperparameter tuning. We train the model for 20 epochs on a Nvidia Titan V GPU and took 12 hours to complete. During inference, we obtain the n-best list of interpretations by running a beam search of corresponding beam width on the LSTM decoder.

### Semantic Reranker

We obtain the contrastive sense encodings (CSE) described in Section 3.2 by using 768-dimensional Sentence-BERT Reimers and Gurevych (2019) embeddings as our baseline embedding. Following Sun et al. (2021), we train the contrastive network with a 1.0 margin (\(m\) in Equation 5) using Adam Kingma and Ba (2015) with a learning rate of \(2^{-5}\), resulting in 768-dimensional definition sense presentations. We reserve 5% of the training examples as a development set for hyperparameter tuning. The contrastive models are trained on a Nvidia Titan V GPU for 4 epochs. The OSD model took 85 minutes to train and the UD model took 8 hours. We follow the training procedure from Sun et al. (2021) to estimate the kernel width parameters (\(h_{m}\) in Equation 3 and \(h_{cf}\) in Equation 9) via generative training when it is computationally feasible to do so and otherwise use 0.1 as our default value.

We check the similarity between two expressions in Equation 9 by comparing their fastText Bojanowski et al. (2017) embeddings. For collaborative filtering, the neighborhood of words \(L(S)\) in Equation 8 is defined as the 5 closest words (including the query word itself) in the dataset's slang expression vocabulary to the query word, measured in terms of cosine similarity between their respective fastText embeddings. We use the list of stopwords from NLTK Bird et al. (2009) to check whether a word is a content word. We apply the _simple_preprocess_ routine from Gensim Renurek and Sojka (2011) before checking for the degree of content word overlap between two sentences.

## Appendix B Additional Results

### Additional Interpretation Examples

Table 7 show additional example interpretations made by the models evaluated in Section 5.1. The first three examples illustrate cases where the semantically informed models were not able to predict the exact definitions, but came up with definitions that are more closely related to the groundtruth compared to the baseline. The latter two examples show cases where the semantically informed models fail to make an improvement.

### Effect of Context Length

In the model evaluation described in Section 5.1, we control for the content-word length of the usage context sentence to examine its effect with respect to interpretation performance for both the baseline and the semantically informed models. Figure 4 shows the results partitioned by the number of content words in the example usage sentence excluding the slang expression, evaluated against four distinctively sampled candidates. To our surprise, we do 

not observe any consistent trends when controlling for context length. Interpretation performance for both the context-based baseline models and their semantically informed variants is fairly consistent under different context length.

### Finetuning Dual Encoder

We consider the case of finetuning the Dual Encoder by training it on all available UD data entries and test on the full OSD test set. Under this scenario, the Dual Encoder model would have seen examples of slang in the OSD test set, though the difference between the definition sentences and usage examples would not allow it to memorize the exact answer. While examining how much knowledge can be transfered from one dataset to another, we also apply the SSI reranker trained on OSD training data on the finetuned results to simulate a stronger baseline model. Table 6 shows the results. When compared to the zero-shot results in Table 2, finetuning on entries corresponding to the same slang, albeit coming from two very different resources, does noticeably improve interpretation accuracy. Moreover, applying SSI to the improved interpretation candidates from the finetuned Dual Encoder further increases interpretation accuracy. This finding suggests that the improvement brought by SSI can indeed generalize in cases where the baseline context-based interpretation model outputs better interpretation candidates.

### Machine Translation Examples

Table 8 to Table 11 show full example translations (English to French) made for the experiment described in Section 5.3, translating sentences containing slang before and after applying slang interpretation.

## Appendix C Data Permissions

At the time when the research is performed, Online Slang Dictionary (OSD) explicitly forbids automated downloading of data from its website service. We therefore have obtained written permission from its owner to download and use the dataset for personal research use. We download data from the online version of the Oxford Dictionary (OD) under personal use. We cannot publically share the two datasets used above as a result. Readers interested in obtaining the exact datasets used in this work must first obtain relevant permission from the respective data owner before the authors of this work can share the data. The Urban Dictionary (UD) dataset is obtained from the authors of Ni and Wang (2017) under a research only license. We release entries relevant to our study with the original data license attached.

\begin{table}
\begin{tabular}{l r r} Model & Distinct negatives & Random negatives \\ \hline Dual Encoder, \(n\) = 5 & 0.604 & 0.598 \\  + SSI & **0.612** & **0.599** \\ Dual Encoder, \(n\) = 50 & 0.583 & 0.570 \\  + SSI & **0.627** & **0.633** \\ \end{tabular}
\end{table}
Table 6: Interpretation results on OSD measured in mean-reciprocal rank (MRR) when training the Dual Encoder without filtering out entries corresponding to words in the OSD testset.

 

[Example 1]

Query (target slang in _bold italic_):

Groundtruth definition of target slang:

Used to describe a girl's butt in a good way.

LM Infill baseline prediction:

LM Infill + SSI prediction:

Horse, dog, puppy.

Dual Encoder baseline prediction:

Penis.

Dual Encoder + SSI prediction:

Girl with big ass and big boobs.

\begin{table}
\begin{tabular}{l l} \hline [Example 2] & \\ Query: & I am an _onion_. \\ Groundtruth definition of target slang:

LM Infill baseline prediction:

LM Infill + SSI prediction:

Dual Encoder baseline prediction:

An idiot. \\ Dual Encoder + SSI prediction:

An asian person. \\ \hline [Example 3] & \\ Query: & In Blastem version 4, they really _nerf_ the EnemyToaster. \\ Groundtruth definition of target slang:

In an update or sequel to a video game, to make a \\ weapon weak or weaker, such that it’s like a Nerf gun. \\ LM Infill baseline prediction:

LM Infill + SSI prediction:

Dual Encoder baseline prediction:

To do something. \\ Dual Encoder + SSI prediction:

To beat someone in the face with your penis. \\ \hline [Example 4] & \\ Query: & I heard Steve was sent to the _cooler_ for breaking and entering. \\ Groundtruth definition of target slang:

LR Infill baseline prediction:

School, house, class. \\ LM Infill + SSI prediction:

Bathroom, kitchen, grounds. \\ Dual Encoder baseline prediction:

Dual Encoder + SSI prediction:

One of the most dangerous things in the world the best. \\ \hline [Example 5] & \\ Query: & Do you have any _sqafety_ \\ Groundtruth definition of target slang:

Marijuana. \\ LM Infill baseline prediction:

Money, friends, cash. \\ LM Infill + SSI prediction:

Self, shoes, money. \\ Dual Encoder baseline prediction:

Dual Encoder + SSI prediction:

Word that is used to describe something that is very good. \\ \hline \end{tabular}
\end{table}
Table 7: Additional examples: Example OSD slang entries with predicted definitions from both the language infill model (LM Infill) and the Dual Encoder model with \(n\) = 50, along with predictions from the corresponding semantically informed slang interpretation (SSI) models.

 

* [25] M. C. Bader, A. D. Krizhevsky, and S. I. Sutskever, "A simple model for the classification of deep neural networks with convolutional neural networks: a review," in _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_, pp. 1038-1049, 2016.
* [26] J. L. Friedman, R. G. Olshenkov, and E. Rabinovich, "Learning from incomplete data to improve recognition by reducing the dimensionality of the training set," _IEEE Transactions on Neural Networks_**7**(1), no. 2, 2015, pp. 1-12.
* [27] P. F. Donoho, N. Srivastava, and V. Vanhoucke, "Deep residual learning for image recognition," in _Advances in neural information processing systems_, vol. 10, no. 1, p. 113002, 2016.
* [28] T. H. Lee, Y. Li, and Z. Zhang, "ImageNet: A survey on machine learning," _arXiv preprint arXiv:1512.00158_ (2015).
* [29] W. Liu, X. Wang, and Q. Sun, "Improving deep neural network with convolutional neural networks," in _NIPS Workshop on Deep Learning Representations_, pp. 1-10.
* [30] W. Liu, X. Wang, and Q. Sun, "Deep residual learning for image recognition," in _CVPR 2015 International Joint Conference on Artificial Intelligence_, pp. 1-10.
* [31] W. Liu, X. Wang, and Q. Sun, "Deep residual learning for image recognition," in _ICML 2015 IEEE Conference on Acoustics and Speech_, pp. 1-1.

[MISSING_PAGE_POST]

 

[Example 1]

Query (target slang in _bold italic_):

Definition of target slang:

Let's smoke a _bowl_ of marijuana.

a marijuana smoking pipe. Most frequently bowls are made out of blown glass, but can be made of metal, wood, etc.

Let's smoke a _pipe_ of marijuana.

Arijuana smoking pipe. Most frequently bowls are made out of blown glass, but can be made of metal, wood, etc.

Let's smoke a _pipe_ of marijuana.

Arijuana smoking pipe. Most frequently bowls are made out of blown glass, but can be made of metal, wood, etc.

Let's smoke a _pipe_ of marijuana.

Arijuana smoking pipe. Most frequently bowls are made out of blown glass, but can be made of metal, wood, etc.

Let's smoke a _pipe_ of marijuana.

Arijuana smoking pipe. Most frequently bowls are made out of blown glass, but can be made of metal, wood, etc.

Let's smoke a _pipe_ of marijuana.

Arijuana smoking pipe. Most frequently bowls are made out of blown glass, but can be made of metal, wood, etc.

Let's smoke a _pipe_ of marijuana.

Arijuana smoking pipe. Most frequently bowls are made out of blown glass, but can be made of metal, wood, etc.

Let's smoke a _pipe_ of marijuana.

Arijuana smoking pipe. Most frequently bowls are made out of blown glass, but can be made of metal, wood, etc.

Let's smoke a _pipe_ of marijuana.

Arijuana smoking pipe. Most frequently bowls are made out of blown glass, but can be made of metal, wood, etc.

Let's smoke a _pipe_ of marijuana.

Arijuana smoking pipe. Most frequently bowls are made out of blown glass, but can be made of metal, wood, etc.

Let's smoke a _pipe_ of marijuana.

Arijuana smoking pipe. Most frequently bowls are made out of blown glass, but can be made of metal, wood, etc.

Let's smoke a _pipe_ of marijuana.

Arijuana smoking pipe. Most frequently bowls are made out of blown glass, but can be made of metal, wood, etc.

Let's smoke a _pipe_ of marijuana.

Arijuana smoking pipe. Most frequently bowls are made out of blown glass, but can be made of metal, wood, etc.

Let's smoke a _pipe_ of marijuana.

Arijuana smoking pipe. Most frequently bowls are made out of blown glass, but can be made of metal, wood, etc.

Let's smoke a _pipe_ of marijuana.

Arijuana smoking pipe. Most frequently bowls are made out of blown glass, but can be made of metal, wood, etc.

Let's smoke a _pipe_ of marijuana.

Arijuana smoking pipe. Most frequently bowls are made out of blown glass, but can be made of metal, wood, etc.

Let's smoke a _pipe_ of marijuana.

Arijuana smoking pipe. Most frequently bowls are made out of blown glass, but can be made of metal, wood, etc.

Let's smoke a _pipe_ of marijuana.

Arijuana smoking pipe. Most frequently bowls are made out of blown glass, but can be made of metal, wood, etc.

Let's smoke a _pipe_ of marijuana.

Arijuana smoking pipe. Most frequently bowls are made out of blown glass, but can be made of metal, wood, etc.

Let's smoke a _pipe_ of marijuana.

Arijuana smoking pipe. Most frequently bowls are made out of blown glass, but can be made of metal, wood, etc.

Let's smoke a _pipe_ of marijuana.

Arijuana smoking pipe. Most frequently bowls are made out of blown glass, but can be made of metal, wood, etc.

Let's smoke a _pipe_ of marijuana.

Arijuana smoking pipe. Most frequently bowls are made out of blown glass, but can be made of metal, wood, etc.

Let's smoke a _pipe_ of marijuana.

Arijuana smoking pipe. Most frequently bowls are made out of blown glass, but can be made of metal, wood, etc.

Let's smoke a _pipe_ of marijuana.

Arijuana smoking pipe. Most frequently bowls are made out of blown glass, but can be made of metal, wood, etc.

Let's smoke a _pipe_ of marijuana.

Arijuana smoking pipe. Most frequently bowls are made out of blown glass, but can be made of metal, wood, etc.

Let's smoke a _pipe_ of marijuana.

Arijuana smoking pipe. Most frequently bowls are made out of blown glass, but can be made of metal, wood, etc.

Let's smoke a _pipe_ of marijuana.

Arijuana smoking pipe. Most frequently bowls are made out of blown glass, but can be made of metal, wood, etc.

Let's smoke a _pipe_ of marijuana.

Arijuana smoking pipe. Most frequently bowls are made out of blown glass, but can be made of metal, wood, etc.

Let's smoke a _pipe_ of marijuana.

Arijuana smoking pipe. Most frequently bowls are made out of blown glass, but can be made of metal, wood, etc.

Let's smoke a _pipe_ of marijuana.

Arijuana smoking pipe. Most frequently bowls are made out of blown glass, but can be made of metal, wood, etc.

Let's smoke a _pipe_ of marijuana.

Arijuana smoking pipe. Most frequently bowls are made out of blown glass, but can be made of metal, wood, etc.

Let's smoke a _pipe_ of marijuana.

Arijuana smoking pipe. Most frequently bowls are made out of blown glass, but can be made of metal, wood, etc.

Let's smoke a _pipe_ of marijuana.

Arijuana smoking pipe. Most frequently bowls are made out of blown glass, but can be made of metal, wood, etc.

Let's smoke a _pipe_ of marijuana.

Arijuana smoking pipe. Most frequently bowls are made out of blown glass, but can be made of metal, wood, etc.

Let's smoke a _pipe_ of marijuana.

Arijuana smoking pipe. Most frequently bowls are made out of blown glass, but can be made of metal, wood, etc.

Let's smoke a _pipe_ of marijuana.

Arijuana smoking pipe. Most frequently bowls are made out of blown glass, but can be made of metal, wood, etc.

Let's smoke a _pipe_ of marijuana.

Arijuana smoking pipe. Most frequently bowls are made out of blown glass, but can be made of metal, wood, etc.

Let's smoke a _pipe_ of marijuana.

Arijuana smoking pipe. Most frequently bowls are made out of blown glass, but can be made of metal, wood, etc.

Let's smoke a _pipe_ of marijuana.

Arijuana smoking pipe. Most frequently bowls are made out of blown glass, but can be made of metal, wood, etc.

Let's smoke a _pipe_ of marijuana.

Arijuana smoking pipe. Most frequently bowls are made out of blown glass, but can be made of metal, wood, etc.

Let's smoke a _pipe_ of marijuana.

Arijuana smoking pipe. Most frequently bowls are made out of blown glass, but can be made of metal, wood, etc.

Let's smoke a _pipe_ of marijuana.

Arijuana smoking pipe. Most frequently bowls are made out of blown glass, but can be made of metal, wood, etc.

Let's smoke a _pipe_ of marijuana.

Arijuana smoking pipe. Most frequently bowls are made out of blown glass, but can be made of metal, wood, etc.

Let's smoke a _pipe_ of marijuana.

Arijuana smoking pipe. Most frequently bowls are made out of blown glass, but can be made of metal, wood, etc.

Let's smoke a _pipe_ of marijuana.

Arijuana smoking pipe. Most frequently bowls are made out of blown glass, but can be made of metal, wood, etc.

Let's smoke a _pipe_ of marijuana.

Arijuana smoking pipe. Most frequently bowls are made out of blown glass, but can be made of metal, wood, etc.

Let's smoke a _pipe_ of marijuana.

Arijuana smoking pipe. Most frequently bowls are made out of blown glass, but can be made of metal, wood, etc.

Let's smoke a _pipe_ of marijuana.

Arijuana smoking pipe. Most frequently bowls are made out of blown glass, but can be made of metal, wood, etc.

Let's smoke a _pipe_ of marijuana.

Arijuana smoking pipe. Most frequently bowls are made out of blown glass, but can be made of metal, wood, etc.

Let's smoke a _pipe_ of marijuana.

Arijuana smoking pipe. Most frequently bowls are made out of blown glass, but can be made of metal, wood, etc.

Let's smoke a _pipe_ of marijuana.

Arijuana smoking pipe. Most frequently bowls are made out of blown glass, but can be made of metal, wood, etc.

Let's smoke a _pipe_ of marijuana.

Arijuana smoking pipe. Most frequently bowls are made out of blown glass, but can be made of metal, wood, etc.

Let's smoke a _pipe_ of marijuana.

Arijuana smoking pipe. Most frequently bowls are made out of blown glass, but can be made of metal, wood, etc.

Let's smoke a _pipe_ of marijuana.

Arijuana smoking pipe. Most frequently bowls are made out of blown glass, but can be made of metal, wood, etc.

Let's smoke a _pipe_ of marijuana.

Arijuana smoking pipe. Most frequently bowls are made out of blown glass, but can be made of metal, wood, etc.

Let's smoke a _pipe_ of marijuana.

Arijuana smoking pipe. Most frequently bowls are made out of blown glass, but can be made of metal, wood, etc.

Let's smoke a _pipe_ of marijuana.

Arijuana smoking pipe. Most frequently bowls are made out of blown glass, but can be made of metal, wood, etc.

Let's smoke a _pipe_ of marijuana.

Arijuana smoking pipe. Most frequently bowls are made out of blown glass, but can be made of metal, wood, etc.

Let's smoke a _pipe_ of marijuana.

Arijuana smoking pipe. Most frequently bowls are made out of blown glass, but can be made of metal, wood, etc.

Let's smoke a _pipe_ of marijuana.

Arijuana smoking pipe. Most frequently bowls are made out of blown glass, but can be made of metal, wood, etc.

Let's smoke a _pipe_ of marijuana.

Arijuana smoking pipe. Most frequently bowls are made out of blown glass, but can be made of metal, wood, etc.

Let's smoke a _pipe_ of marijuana.

Arijuana smoking pipe. Most frequently bowls are made out of blown glass, but can be made of metal, wood, etc.

Let's smoke a _pipe_ of marijuana.

Arijuana smoking pipe. Most frequently bowls are made out of blown glass, but can be made of metal, wood, etc.

Let's smoke a _pipe_ of marijuana.

Arijuana smoking pipe. Most frequently bowls are made out of blown glass, but can be made of metal, wood, etc.

Let's smoke a _pipe_ of marijuana.

Arijuana smoking pipe. Most frequently bowls are made out of blown glass, but can be made of metal, wood, etc.

Let's smoke a _pipe_ of marijuana.

Arijuana smoking pipe. Most frequently bowls are made out of blown glass, but can be made of metal, wood, etc.

Let's smoke a _pipe_ of marijuana.

Arijuana smoking pipe. Most frequently bowls are made out of blown glass, but can be made of metal, wood, etc.

Let's smoke a _pipe_ of marijuana.

Arijuana smoking pipe. Most frequently bowls are made out of blown glass, but can be made of metal, wood, etc.

Let's smoke a _pipe_ of marijuana.

Arijuana smoking pipe. Most frequently bowls are made out of blown glass, but can be made of metal, wood, etc.

Let's smoke a _pipe_ of marijuana.

Arijuana smoking pipe. Most frequently bowls are made out of blown glass, but can be made of metal, wood, etc.

Let's smoke a _pipe_ of marijuana.

Arijuana smoking pipe. Most frequently bowls are made out of blown glass, but can be made of metal, wood, etc.

Let's smoke a _pipe_ of marijuana.

 

[Example 2]

Query:

Definition of target slang:

Groundtruth interpreted sentence:

Original query sentence translation:

\begin{table}
\begin{tabular}{l l}  & That band was so totally _vast_. \\  & Cool or anything good. \\ Groundtruth interpreted sentence: & That band was so totally _cool_. \\ Original query sentence translation: & Ce groupe etait si vaste. \\  & (_BLEU: 53.2, BLEURT: 32.9_, COMET: -0.59) \\ Gold-standard translation: & Ce groupe etait tellement cool. \\  & \(<=>\) 10 

[Example 3]

Query (target slang in _bold italic_): Definition of target slang: Groundtruth interpreted sentence: Original query sentence translation: Gold-standard translation: LM Infill interpretation & translation: (1) Man, I ain't been to that place in a _while_! (2) Man, I ain't been to that place in a _million_! (3) Man, I ain't been to that place in a _both_! (4) Man, I ain't been to that place in a _vanilla_! (5) Man, I ain't been to that place in a _ignment_! LM Infill + SSI interpretation & translation: (1) Man, I ain't been to that place in a _week_! (2) Man, I ain't been to that place in a _minute_! (3) Man, I ain't been to that place in a _hour_! (4) Man, I ain't been to that place in a _decade_! (5) Man, I ain't been to that place in a _day_! Jan, I ain't been to that place in a _Mant_; An unspecific, but long-ish length of time. Man, I ain't been to that place in a _long time_! Je ne suis pas alle a cet endroit en une quinzaine! (_BLEU: 36.1, BLEURT: 61.2, COMET: 0.57_) Je n'y suis pas alle depuis longtemps! Je ne suis pas alle a cet endroit depuis un _moment_! (_BLEU: 46.9, BLEURT: 76.5, COMET: 0.88_) Je ne suis pas alle a cet endroit dans un _million_! (_BLEU: 38.8, BLEURT: 25.1, COMET: -1.17_) Je ne suis pas alle a cet endroit dans les _deux_! (_BLEU: 42.2, BLEURT: 25.7, COMET: -0.98_) Mec, je n'ai pas ete a cet endroit dans une _vanille_! (_BLEU: 16.2, BLEURT: 7.3, COMET: 1.53_) Mec, je n'ai pas ete a cet endroit dans un _ignement_! (_BLEU: 16.2, BLEURT: 12.7, COMET: -1.31_) Je ne suis pas alle a cet endroit en une _semaine_! (_BLEU: 38.2, BLEURT: 49.8, COMET: 0.45_) Je ne suis pas alle a cet endroit en une _minute_! (_BLEU: 38.8, BLEURT: 42.5, COMET: -0.36_) Je ne suis pas alle a cet endroit en une _heure_! (_BLEU: 38.7, BLEURT: 35.8, COMET: -0.51_) Je n'y suis pas alle depuis une _decennie_ (_BLEU: 68.8, BLEURT: 81.8, COMET: 1.03_) Je ne suis pas alle a cet endroit en une _journee_! (_BLEU: 37.1, BLEURT: 49.7, COMET: -0.30_)

\begin{table}
\begin{tabular}{l l} \hline  & Man, I ain’t been to that place in a _fortnight_! \\  & Ann supecific, but long-ish length of time. \\  & Man, I am‘t been to that place in a _long time_! \\  & Ja re ne suis pas alle a cet endroit en une quinzaine! (_BLEU: 36.1, BLEURT: 61.2, COMET: 0.57_) Je n’y suis pas alle depuis longtemps! \\  & Je ne suis pas alle a cet endroit depuis un _moment_! (_BLEU: 46.9, BLEURT: 76.5, COMET: 0.88_) Je ne suis pas alle a cet endroit dans un _million_! (_BLEU: 38.8, BLEURT: 25.1, COMET: -1.17_) Je ne suis pas alle a cet endroit dans les _deux_! (_BLEU: 42.2, BLEURT: 25.7, COMET: -0.98_) Mcc, je n’ai pas éte à cet endroit dans une _vanille_! (_BLEU: 16.2, BLEURT: 7.3, COMET: 1.53_) Mec, je n’ai pas êê a cet endroit dans un _ignement_! (_BLEU: 16.2, BLEURT: 12.7, COMET: -1.31_) Je ne suis pas alle a cet endroit en une _semaine_! (_BLEU: 38.2, BLEURT: 49.8, COMET: 0.45_) Je ne suis pas alle a cet endroit en une _minute_! (_BLEU: 38.8, BLEURT: 42.5, COMET: -0.36_) Je ne suis pas alle a cet endroit en une _heure_! (_BLEU: 38.7, BLEURT: 35.8, COMET: -0.51_) Je n’y suis pas alle depuis une _decennie_ (_BLEU: 68.8, BLEURT: 81.8, COMET: 1.03_) Je ne suis pas alle a cet endroit en une _journee_! (_BLEU: 37.1, BLEURT: 49.7, COMET: -0.30_) 

* [Example 4]
* Query: Definition of target slang: Groundtruth interpreted sentence: Original query sentence translation: Gold-standard translation: LM Infill interpretation & translation: (1) I want to go get coffee but it's _raining_ outside.
* [2] I want to go get coffee but it's _closed_ outside.
* [3] I want to go get coffee but it's _pouring_ outside.
* [4] I want to go get coffee but it's _been_ outside.
* [5] I want to go get coffee but it's _starting_ outside.
* LM Infill + SSI interpretation & translation: (1) I want to go get coffee but it's _cold_ outside.
* [6] I want to go get coffee but it's _warm_ outside.
* [7] I want to go get coffee but it's _driving_ outside.
* [8] I want to go get coffee but it's _closing_ outside.
* [9] I want to go get coffee but it's _dark_ outside.

\begin{table}
\begin{tabular}{l l} \hline Example 4 \\ Query: & I want to go get coffee but it’s _bitter_ outside. \\ Definition of target slang: Groundtruth interpreted sentence: Original query sentence translation: Gold-standard translation: & Abbreviated form of bitterly cold. I want to go get coffee but it’s _bitterly cold_ outside.

\begin{tabular}{l l} Original query sentence translation: & Je veux aller prendre un cafe mais cl’est amer dehors. \\  & (_BLEU:_ 65.0, _BLEURT:_ 59.8, _COMET:_ 0.77) \\  & Je veux aller prendre un cafe, mais il fait tres froid \\  & debors. \\ \end{tabular} \\ \hline LM Infill interpretation & translation: (1) I want to go get coffee but it’s _training_ outside.
* [2] I want to go get coffee but it’s _closed_ outside.
* [3] I want to go get coffee but it’s _pouring_ outside.
* [4] I want to go get coffee but it’s _been_ outside.
* [5] I want to go get coffee but it’s _starting_ outside.
* LM Infill + SSI interpretation & translation: (1) I want to go get coffee but it’s _cold_ outside.
* [2] I want to go get coffee but it’s _warm_ outside.
* [3] I want to go get coffee but it’s _driving_ outside.
* [4] I want to go get coffee but it’s _closing_ outside.
* [5] I want to go get coffee but it’s _dark_ outside.
* Je veux aller prendre un cafe mais il _pleut_ dehors. _DEUU:_ 68.1, _BLEURT:_ 79.9, _COMET:_ 0.97)
* Je veux aller prendre un cafe mais il est _ferme_ dehors.
* Je veux aller prendre un cafe mais cl _ceule_ dehors.
* Je veux aller prendre un cafe mais cl a _ete_ dehors.
* Je veux aller prendre un cafe, mais cla a _ette_ dehors.
* Je veux aller prendre un cafe mais cla _et_ dehors.
* Je veux aller prendre un cafe, mais cla a _ette_ dehors.
* Je veux aller prendre un cafe mais cla _et_ dehors.
* Je veux aller prendre un cafe, mais cla _commence_ delays.
* Je veux aller prendre un cafe, mais cla _comhetc_ delays.
* Je veux aller prendre un cafe mais il fait _moir_ dehors.
* Je veux aller prendre un cafe mais il _conduit_ dehors.
* Je veux aller prendre un cafe mais il se _ferme_ dehors.
* Je veux aller prendre un cafe mais il feit _noir_ dehors.
* Je veux aller prendre un cafe mais il faint _noir_ dehors.
* Je veux aller prendre un cafe mais il _conduct_ delays.
* Je veux aller prendre un cafe mais il _conduit_ dehors.