SlowGuess/NAACL_2022_OCR huggingface dataset:
Adam (Kingma and Ba, 2015) with a learning rate of  $2^{-5}$ , resulting in 768-dimensional definition sense presentations. We reserve  $5\%$  of the training examples as a development set for hyperparameter tuning. The contrastive models are trained on a Nvidia Titan V GPU for 4 epochs. The OSD model took 85 minutes to train and the UD model took 8 hours. We follow the training procedure from Sun et al. (2021) to estimate the kernel width parameters ( $h_m$  in Equation 3 and  $h_{cf}$  in Equation 9) via generative training when it is computationally feasible to do so and otherwise use 0.1 as our default value.
We check the similarity between two expressions in Equation 9 by comparing their fastText (Bojanowski et al., 2017) embeddings. For collaborative filtering, the neighborhood of words  $L(S)$  in Equation 8 is defined as the 5 closest words (including the query word itself) in the dataset's slang expression vocabulary to the query word, measured in terms of cosine similarity between their respective fastText embeddings. We use the list of stopwords from NLTK (Bird et al., 2009) to check whether a word is a content word. We apply the simple_preprocess routine from Gensim (Rehurek and Sojka, 2011) before checking for the degree of content word overlap between two sentences.
B.1 Additional Interpretation Examples
Table 7 show additional example interpretations made by the models evaluated in Section 5.1. The first three examples illustrate cases where the semantically informed models were not able to predict the exact definitions, but came up with definitions that are more closely related to the groundtruth compared to the baseline. The latter two examples show cases where the semantically informed models fail to make an improvement.
B.2 Effect of Context Length
In the model evaluation described in Section 5.1, we control for the content-word length of the usage context sentence to examine its effect with respect to interpretation performance for both the baseline and the semantically informed models. Figure 4 shows the results partitioned by the number of content words in the example usage sentence excluding the slang expression, evaluated against four distinctively sampled candidates. To our surprise, we do
Model	Distinct negatives	Random negatives
Dual Encoder, n = 5	0.604	0.598
+ SSI	0.612	0.599
Dual Encoder, n = 50	0.583	0.570
+ SSI	0.627	0.633
Table 6: Interpretation results on OSD measured in mean-reciprocal rank (MRR) when training the Dual Encoder without filtering out entries corresponding to words in the OSD testset.
not observe any consistent trends when controlling for context length. Interpretation performance for both the context-based baseline models and their semantically informed variants is fairly consistent under different context length.
B.3 Finetuning Dual Encoder
We consider the case of finetuning the Dual Encoder by training it on all available UD data entries and test on the full OSD test set. Under this scenario, the Dual Encoder model would have seen examples of slang in the OSD test set, though the difference between the definition sentences and usage examples would not allow it to memorize the exact answer. While examining how much knowledge can be transferred from one dataset to another, we also apply the SSI reranker trained on OSD training data on the finetuned results to simulate a stronger baseline model. Table 6 shows the results. When compared to the zero-shot results in Table 2, finetuning on entries corresponding to the same slang, albeit coming from two very different resources, does noticeably improve interpretation accuracy. Moreover, applying SSI to the improved interpretation candidates from the finetuned Dual Encoder further increases interpretation accuracy. This finding suggests that the improvement brought by SSI can indeed generalize in cases where the baseline context-based interpretation model outputs better interpretation candidates.
B.4 Machine Translation Examples
Table 8 to Table 11 show full example translations (English to French) made for the experiment described in Section 5.3, translating sentences containing slang before and after applying slang interpretation.
C Data Permissions
At the time when the research is performed, Online Slang Dictionary (OSD) explicitly forbids automated downloading of data from its website service. We therefore have obtained written permission from its owner to download and use the dataset for personal research use. We download data from the online version of the Oxford Dictionary (OD) under personal use. We cannot publically share the two datasets used above as a result. Readers interested in obtaining the exact datasets used in this work must first obtain relevant permission from the respective data owner before the authors of this work can share the data. The Urban Dictionary (UD) dataset is obtained from the authors of Ni and Wang (2017) under a research only license. We release entries relevant to our study with the original data license attached.
[Example 1]
Query (target slang in bold italic):	That girl has a donkey.
Groundtruth definition of target slang:	Used to describe a girl's butt in a good way.
LM Infill baseline prediction:	Name, crush, boyfriend.
LM Infill + SSI prediction:	Horse, dog, puppy.
Dual Encoder baseline prediction:	Penis.
Dual Encoder + SSI prediction:	Girl with big ass and big boobs.
[Example 2]
Query:	I am an onion.
Groundtruth definition of target slang:	A native of Bermuda.
LM Infill baseline prediction:	Adult, man, athlete.
LM Infill + SSI prediction:	Ren, adult, guard.
Dual Encoder baseline prediction:	An idiot.
Dual Encoder + SSI prediction:	An asian person.
[Example 3]
Query:	In Blastem version 4, they really nerf the EnemyToaster.
Groundtruth definition of target slang:	In an update or sequel to a video game, to make a weapon weak or weaker, such that it's like a Nerf gun.
LM Infill baseline prediction:	Were, called, attack.
LM Infill + SSI prediction:	Made, hacked, came.
Dual Encoder baseline prediction:	To do something.
Dual Encoder + SSI prediction:	To beat someone in the face with your penis.
[Example 4]
Query:	I heard Steve was sent to the cooler for breaking and entering.
Groundtruth definition of target slang:	Reform school.
LM Infill baseline prediction:	School, house, class.
LM Infill + SSI prediction:	Bathroom, kitchen, grounds.
Dual Encoder baseline prediction:	Slang term for the police.
Dual Encoder + SSI prediction:	One of the most dangerous things in the world the best.
[Example 5]
Query:	Do you have any safety
Groundtruth definition of target slang:	Marijuana.
LM Infill baseline prediction:	Money, friends, cash.
LM Infill + SSI prediction:	Self, shoes, money.
Dual Encoder baseline prediction:	Marijuana.
Dual Encoder + SSI prediction:	Word that is used to describe something that is very good.
Table 7: Additional examples: Example OSD slang entries with predicted definitions from both the language infill model (LM Infill) and the Dual Encoder model with  $n = 50$ , along with predictions from the corresponding semantically informed slang interpretation (SSI) models.
(a) OSD
(b)UD
Figure 4: Evaluation of slang interpretation performance measured in mean-reciprocal rank (MRR) for all models with  $n = 50$ . Test entries are partitioned based on the number of content words (excluding the slang expression itself) found within the corresponding example usage sentence. Number of entries corresponding to each context length is shown in parenthesis on the x-axis legend.
[Example 1]
Query (target slang in bold italic):	Let's smoke a bowl of marijuana.
Definition of target slang:	a marijuana smoking pipe. Most frequently bowls are made out of blown glass, but can be made of metal, wood, etc.
Groundtruth interpreted sentence:	Let's smoke a pipe of marijuana.
Original query sentence translation:	Faisons fumer un bol de marijuana.
(BLEU: 78.1, BLEURT: 66.1,COMET: 1.05)
Gold-standard translation:	Faisons fumer une pipe de marijuana.
LM Infill interpretation & translation:
(1) Let's smoke a for of marijuana.	Fumons un pour de la marijuana.
(BLEU: 47.1,BLEURT: 20.6,COMET: -0.58)
(2) Let's smoke a in of marijuana.	On fume un peu (little) de marijuana.
(BLEU: 51.6,BLEURT: 64.8,COMET: 0.48)
(3) Let's smoke a myself of marijuana.	Nous allons fumer moi-même de la marijuana.
(BLEU: 51.8,BLEURT: 32.4,COMET: -0.55)
(4) Let's smoke a or of marijuana.	Fumons un ou de marijuana.
(BLEU: 45.4,BLEURT: 32.2,COMET: -1.04)
(5) Let's smoke a vapor of marijuana.	Fumons une vapeur de marijuana.
(BLEU: 56.4,BLEURT: 57.0,COMET: 0.40)
LM Infill + SSI interpretation & translation:
(1) Let's smoke a pot of marijuana.	Faisons fumer un pot de marijuana.
(BLEU: 79.5,BLEURT: 78.8,COMET: 1.15)
(2) Let's smoke a pipe of marijuana.	Faisons fumer une pipe de marijuana.
(BLEU: 100.0,BLEURT: 99.1,COMET: 1.32)
(3) Let's smoke a pack of marijuana.	Faisons fumer un paquet de marijuana.
(BLEU: 77.7,BLEURT: 68.3,COMET: 0.80)
(4) Let's smoke a leaf of marijuana.	Faisons fumer une feuille de marijuana.
(BLEU: 79.9,BLEURT: 48.2,COMET: 1.21)
(5) Let's smoke a cigarette of marijuana.	Faisons fumer une cigarette de marijuana.
(BLEU: 75.7,BLEURT: 81.7,COMET: 1.25)
Table 8: Additional examples of machine translation of slang, without or with the application of the SSI framework. The top 5 interpreted and translated sentences are shown for each model with BLEU, BLEURT, and COMET scores against the gold-standard translation shown in parentheses.
[Example 2]
Query:	That band was so totally vast.
Definition of target slang:	Cool or anything good.
Groundtruth interpreted sentence:	That band was so totally cool.
Original query sentence translation:	Ce groupe était si vaste.
(BLEU: 53.2, BLEURT: 32.9,COMET: -0.59)
Gold-standard translation:	Ce groupe était tellement cool.
LM Infill interpretation & translation:
(1) That band was so totally popular.	Ce groupe était tellement populaire.
(BLEU: 74.5,BLEURT: 78.7,COMET: 0.43)
(2) That band was so totally good.	Ce groupe était si bon.
(BLEU: 51.8,BLEURT: 77.0,COMET: 0.32)
(3) That band was so totally different.	Ce groupe était complètement différent.
(BLEU: 57.2,BLEURT: 50.3,COMET: -0.07)
(4) That band was so totally famous.	Ce groupe était si célibre.
(BLEU: 54.4,BLEURT: 66.2,COMET: -0.21)
(5) That band was so totally new.	Ce groupe était totalement nouveau.
(BLEU: 64.2,BLEURT: 50.2,COMET: -0.21)
LM Infill + SSI interpretation & translation:
(1) That band was so totally huge.	Ce groupe était tellement enormme.
(BLEU: 81.1,BLEURT: 56.0,COMET: 0.15)
(2) That band was so totally big.	Ce groupe était tellement grand.
(BLEU: 83.0,BLEURT: 50.7,COMET: -0.19)
(3) That band was so totally important.	Ce groupe était si important.
(BLEU: 55.9,BLEURT: 49.9,COMET: -0.58)
(4) That band was so totally cool.	Ce groupe était tellement cool.
(BLEU: 100.0,BLEURT: 97.9,COMET: 1.29)
(5) That band was so totally bad.	Ce groupe était si mauvais.
(BLEU: 52.3,BLEURT: 62.9,COMET: -0.48)
Table 9: Continuation of Table 8.
[Example 3]
Query (target slang in bold italic):	Man, I ain't been to that place in a fortnight!
Definition of target slang:	An unspecific, but long-ish length of time.
Groundtruth interpreted sentence:	Man, I ain't been to that place in a long time!
Original query sentence translation:	Je ne suis pas allé à cet endroit en une quinzaine!
(BLEU: 36.1, BLEURT: 61.2,COMET: 0.57)
Gold-standard translation:	Je n'y suis pas allé depuis longtemps!
LM Infill interpretation & translation:
(1) Man, I ain't been to that place in a while!	Je ne suis pas allé à cet endroit depuis un moment!
(BLEU: 46.9,BLEURT: 76.5,COMET: 0.88)
(2) Man, I ain't been to that place in a million!	Je ne suis pas allé à cet endroit dans un million!
(BLEU: 38.8,BLEURT: 25.1,COMET: -1.17)
(3) Man, I ain't been to that place in a both!	Je ne suis pas allé à cet endroit dans les deux!
(BLEU: 42.2,BLEURT: 25.7,COMET: -0.98)
(4) Man, I ain't been to that place in a vanilla!	Mec, je n'ai pas été à cet endroit dans une vanille!
(BLEU: 16.2,BLEURT: 7.3,COMET: 1.53)
(5) Man, I ain't been to that place in a ignment!	Mec, je n'ai pas été à cet endroit dans un ignement!
(BLEU: 16.2,BLEURT: 12.7,COMET: -1.31)
LM Infill + SSI interpretation & translation:
(1) Man, I ain't been to that place in a week!	Je ne suis pas allé à cet endroit en une semaine!
(BLEU: 38.2,BLEURT: 49.8,COMET: 0.45)
(2) Man, I ain't been to that place in a minute!	Je ne suis pas allé à cet endroit en une minute!
(BLEU: 38.8,BLEURT: 42.5,COMET: -0.36)
(3) Man, I ain't been to that place in a hour!	Je ne suis pas allé à cet endroit en une heures!
(BLEU: 38.7,BLEURT: 35.8,COMET: -0.51)
(4) Man, I ain't been to that place in a decade!	Je n'y suis pas allé depuis une décembre
(BLEU: 68.8,BLEURT: 81.8,COMET: 1.03)
(5) Man, I ain't been to that place in a day!	Je ne suis pas allé à cet endroit en une journée!
(BLEU: 37.1,BLEURT: 49.7,COMET: -0.30)
Table 10: Continuation of Table 9.
[Example 4]
Query:	I want to go get coffee but it's bitter outside.
Definition of target slang:	Abbreviated form of bitterly cold.
Groundtruth interpreted sentence:	I want to go get coffee but it's bitterly cold outside.
Original query sentence translation:	Je veux aller prendre un café mais c'est amer dehors. (BLEU: 65.0, BLEURT: 59.8,COMET: 0.77)
Gold-standard translation:	Je veux aller prendre un café, mais il fait très froid dehors.
LM Infill interpretation & translation:
(1) I want to go get coffee but it's raining outside.	Je veux aller prendre un café mais il pleut dehors. (BLEU: 68.1, BLEURT: 79.9,COMET: 0.97)
(2) I want to go get coffee but it's closed outside.	Je veux aller prendre un café mais il est fermé dehors. (BLEU: 70.7, BLEURT: 53.9,COMET: -0.15)
(3) I want to go get coffee but it's pouring outside.	Je veux aller chercher du café, maisça coule dehors. (BLEU: 51.9, BLEURT: 31.6,COMET: -0.38)
(4) I want to go get coffee but it's been outside.	Je veux aller prendre un café, maisça a été dehors. (BLEU: 68.4, BLEURT: 27.1,COMET: -0.88)
(5) I want to go get coffee but it's starting outside.	Je veux aller prendre un café, maisça commence dehors. (BLEU: 68.5, BLEURT: 31.0,COMET: -0.57)
LM Infill + SSI interpretation & translation:
(1) I want to go get coffee but it's cold outside.	Je veux aller prendre un café, mais il fait froid dehors. (BLEU: 90.3, BLEURT: 92.7,COMET: 1.20)
(2) I want to go get coffee but it's warm outside.	Je veux aller prendre un café mais il fait chaud dehors. (BLEU: 78.1, BLEURT: 79.1,COMET: 1.12)
(3) I want to go get coffee but it's driving outside.	Je veux aller prendre un café mais il conduit dehors. (BLEU: 70.4, BLEURT: 26.5,COMET: -0.69)
(4) I want to go get coffee but it's closing outside.	Je veux aller prendre un café mais il se ferme dehors. (BLEU: 69.8, BLEURT: 23.2,COMET: -0.81)
(5) I want to go get coffee but it's dark outside.	Je veux aller prendre un café, mais il fait noir dehors. (BLEU: 82.3, BLEURT: 73.7,COMET: 0.80)
Table 11: Continuation of Table 10.