C oming to T erms: A utomatic F ormation of N eologisms in H ebrew (2020)

Abstract: Spoken languages are ever-changing, with new words entering them all the time. However, coming up with new words (neologisms) today relies exclusively on human creativity. In this paper we propose a system to automatically suggest neologisms. We focus on the Hebrew language as a test case due to the unusual regularity of its noun formation. User studies comparing our algorithm to experts and non-experts demonstrate that our algorithm is capable of generating high-quality outputs, as well as enhance human creativity. More broadly, we seek to inspire more computational work around the topic of linguistic creativity, which we believe offers numerous unexplored opportunities.

## introduction human languages are always changing, evolving, and adapting to the needs of their speakers. new words regularly enter our vocabulary, while others disappear. for example, the word "selfie" (selfportrait digital photo, typically taken with a smartphone) has recently become part of everyday english, even spawning variations such as helfie (a selfie of one's hair), welfie (a selfie taken during a workout), and drelfie (a selfie taken while being drunk) @xcite . newly coined words or expressions are termed neologisms. there are many neologism formation mechanisms; common ones include loanwords borrowed from another language (kindergarten), morphological derivation (socialize, simplify), compounding (football, breakwater), blending (spoon + @xmath0, and acronyms (laser). importantly, the coining of novel words relies on human creativity, with the new terms often conveying a lot of information in an inventive way. in this work, we set out to explore the possibility of automating some of this inherently-human, creative linguistic process. in other words, we ask whether computers can generate high-quality, novel words on their own, or alternatively help inspire people to find better words. we focus on automatic generation of neologisms in the hebrew language. hebrew has several properties which make it particularly interesting for our goal: first, modern hebrew was revived after a long period of time @xcite , which is unique. there are no other cases of a natural language without any native speakers subsequently acquiring millions of native speakers. for this reason, foreign words are very common in hebrew, and many terms need to be coined. another reason for focusing on hebrew is its unusual regularity of noun formation. while portmanteaus (word blends), word combinations and other formation mechanisms do exist in hebrew, most words are created by combination of root and pattern. to the best of our knowledge, this method of word generation was not explored before in a computational context. our contributions are: beyond the specific task of generating hebrew neologisms, we hope this work would inspire further research towards automating and supporting creative tasks. ## background hebrew is classified as an afroasiatic, semitic language. like arabic, hebrew is written right to left. vowels are indicated by diacritic marks representing the syllabic onset, or by matres lectionis (consonantal letters used as vowels). everyday printed hebrew often omits the diacritic marks, resulting in a highly ambiguous text. for example, ‫בצל‬ can be diacritized as "onion", "in a shadow" or "in the shadow" @xcite . hebrew morphology. hebrew follows nonconcatenative morphology. it is based on roots, consisting of a sequence of consonants (usually three), from which nouns, adjectives and verbs are formed. thus, different words composed of the same root often have semantically related meanings. for example, the words ‫ֶת‬ ‫ֹר‬ ‫ְמ‬ ‫ז‬ ‫תּ‬ (tizmoret),‫ר‬ ‫ַמּ‬ ‫ז‬ (zamar), and ‫ר‬ ‫ֶמ‬ ‫ז‬ (zemer) all have the root ‫זמר‬ (sing), and stand respectively for an orchestra, a singer, and a song. while in english words are usually formed by adding prefixes and suffixes, in hebrew the root letters are combined into patterns, called mishkalim. the patterns are commonly represented by using the arbitrary placeholder letters ‫קטל‬ (k-t-l) for root consonants. patterns usually include diacritics, vowel letters and sometimes prefixes and suffixes. for example, to form the hebrew word ‫ֶת‬ ‫ֹר‬ ‫ְמ‬ ‫ז‬ ‫תּ‬ (orchestra), the placeholder letters ‫קטל‬ of the pattern ‫ֶת‬ ‫ל‬ ‫ְט‬ ‫ִק‬ ‫תּ‬ are replaced with the root letters ‫.זמר‬ even though this concept is simple, there is a significant amount of special cases requiring modifications to the form of the final word. from a sample of the even-shoshan dictionary (even-shoshan and azar, 2003), we estimate that ∼ 2/3 of the roots require some modification. for example, combining the root ‫רפא‬ with the pattern ‫ָה‬ ‫ֵל‬ ‫ְט‬ ‫ַק‬ ‫תּ‬ should have resulted in ‫ָה‬ ‫ֵא‬ ‫ְפּ‬ ‫ר‬ ‫תּ‬ (tarpe'a). however, since ‫רפא‬ is a special root (ends with ‫,)א‬ it becomes ‫ָה‬ ‫רוּפ‬ ‫תּ‬ (trufa). importantly, many patterns denote specific semantic categories. for example, the pattern ‫ָל‬ ‫ַטּ‬ ‫ק‬ (katal) is commonly used to describe professions, as in ‫ר‬ ‫ַמּ‬ ‫ז‬ (singer), ‫ָח‬ ‫ַבּ‬ ‫ט‬ (cook), and ‫ָב‬ ‫ַתּ‬ ‫כּ‬ (reporter). however, not every category has its matching patterns, and some patterns can denote multiple different categories. for example, the pattern ‫ֶת‬ ‫ֶל‬ ‫ַטּ‬ ‫ק‬ (katelet) can be used for professions in feminine form, but is also a very common pattern for illnesses. formation of hebrew words. many world languages have official language regulators, often referred to as language academies (e.g., the royal spanish academy, l'académie franc ¸aise, the council for german orthography). the regulating body for hebrew is the academy of the hebrew language. one of the academy's most important roles is creating new words to replace loanwords derived from other languages @xcite . the initiative tends to come from the public, seeking hebrew alternatives for foreign words common in everyday speech. a committee of scholars of language, linguistics, judaic studies, and bible discusses the word and suggests a hebrew replacement. most new words are built using the rootpattern system (aca, 2020), although compound nouns and portmanteaus (blends) are also used. we note that even with decades of experience, it is difficult to predict whether the new terms will be picked up by the public. some words catch on immediately, some take years, and some never do. ## methodology in this section we present our algorithm, eliezer bot-yehuda (eby), named after eliezer ben-yehuda, a lexicographer who was the driving force behind the revival of the hebrew language in the modern era. we follow the three main ways of forming words used by the academy of the hebrew language: root-pattern, compounds, and portmanteaus. the input to the algorithm is a source word in english, for which we wish to find a hebrew word. we used english as a mediating language due to the variety of linguistic resources available for it, but the algorithm can work with any other language (see section 3.3). figure 1 shows the process for the input word "palette". ## evaluation of individual components our pipeline (depicted in figure 1 ) is composed of several components. in this section we evaluate the contribution of the three main components: important words (tf-idf), combining roots and patterns (seq2seq model) and ranking and filtering (language model). for these evaluations, we used student annotators who are native speakers of hebrew. ## evaluating the algorithm's output after evaluating the main parts of the algorithm, we continue to evaluate its suggestions (including root and pattern, compound and portmanteaus). we address two main questions: (1) how do the words our algorithm generated compare to those generated by humans? (2) can our algorithm's output boost creativity in humans generating new words? we note that we do not expect our algorithm to beat human performance. rather, we set out to test whether it can generate plausible suggestions, and whether it can inspire people to suggest better words. we considered the following baselines: the officially chosen hebrew words, as well as runner-up suggestions discussed by the committee. 2. non-expert suggestions: new word suggestions by human participants (non experts). 3. non-expert + eby. new word suggestions by non experts, after being exposed to the algorithm's output. ## error analysis we analyzed the algorithm's errors to understand where it is lacking and where to focus future work efforts. we identified two main issues. limited resources. in many of the cases in which our algorithm failed to generate appropriate alternatives, it appears to be due to a lack of resourcesabsent / inaccurate hebrew translations, or a lack of root / pattern information. for example, consider the word "leggings". one of the important words identified was "fitting", which was inaccurately translated to "appropriate". another word, "tight", was accurately translated to both ‫דוּק‬ ‫ה‬ (haduk) and ַ ‫ָתוּח‬ ‫מ‬ (matuakh), but our dictionaries did not have their roots. we believe that better hebrew resources will significantly improve our algorithm. connotations. some of eby's suggestion received low likability scores. one such word, which was highly disliked, is ‫ָה‬ ‫ֵע‬ ‫ָל-ז‬ ‫ָכּ‬ ‫ס‬ (sakal ze'a) for "deodorant". literally, this is a combination of "to thwart" and "sweat". even though the meaning is well-represented here, both words have a negative connotation. describing deodorant by the word "sweat" is not appealing, and the hebrew word for "thwart" also carries negative connotations. another example is "periphery", where suggestions focused on roots with meanings of "margin" and "out". this can be offensive for people who live there. in fact, even the hebrew academy was unable to reach a decision for this word. after discussing suggestions based on "margin", it was taken off of the agenda following public outrage foot_4 . we believe a better understanding of connotations can help the algorithm produce more appealing results. computational neologism. much previous computational work on neologisms focused on automatic recognition of neologisms and their meanings @xcite @xcite @xcite . work on computational generation of neologisms mostly focused on creating compounds and word blends from source words @xcite @xcite @xcite . although our algorithm supports these word formations, the main focus of our work is on word generation via root and pattern combination, unexplored in a computational context before. in addition to providing an algorithm for the generation of the neologisms themselves, we also show its potential in enhancing human creativity. ## discussion and future work coming up with new words (neologisms) is a hallmark of human creativity. in this paper we proposed a system to automatically suggest neologisms, using the hebrew language as a test case. given a source word, the system identifies related words, roots and patterns and uses them to suggest new terms. we evaluated the system through a user study, comparing it to experts and non-experts, and showed that while humans still perform better, our algorithm is capable of generating high-quality outputs, as well as enhance human creativity. in the future, we plan to explore more word formation strategies, such as associations; for exam-ple, by using the eat database @xcite . another exciting avenue is researching the factors influencing the acceptance of new words by the public. a better understanding of successful neologisms, adopted by speakers of the language, can potentially help in their creation. beyond the somewhat-niche nature of hebrew neologisms, we seek more broadly to inspire more work on automating and supporting creative tasks (such as authoring), especially in human-computer collaborative frameworks. we believe more nlp should be applied to tackle psychological phenomena, and that the intersection of the fields opens up many intriguing research questions.