J - U ni M orph: J apanese Morphological Annotation through the Universal Feature Schema (2024)

Abstract: We introduce a Japanese Morphology dataset, J-UniMorph, developed based on the UniMorph feature schema. This dataset addresses the unique and rich verb forms characteristic of the language’s agglutinative nature. J-UniMorph distinguishes itself from the existing Japanese subset of UniMorph, which is automatically extracted from Wiktionary. On average, the Wiktionary Edition features around 12 inflected forms for each word and is primarily dominated by denominal verbs (i.e., [noun] + suru (do-PRS)). Morphologically, this inflection pattern is same as the verb suru (do). In contrast, J-UniMorph explores a much broader and more frequently used range of verb forms, offering 118 inflected forms for each word on average. It includes honorifics, a range of politeness levels, and other linguistic nuances, emphasizing the distinctive characteristics of the Japanese language. This paper presents detailed statistics and characteristics of J-UniMorph, comparing it with the Wiktionary Edition. We will release J-UniMorph and its interactive visualizer publicly available, aiming to support cross-linguistic research and various applications.

## introduction universal morphology (unimorph) is a collaborative project that delivers a wide-ranging collection of standardized morphological features for over 170 languages in the world (sylak-glassman, 2016; mccarthy et al., 2020). unimorph feature schema comprises over 212 feature labels across 23 dimensions of meaning labels, such as tense, aspect, and mood. more concretely, unimorph dataset consists of a lemma coupled with a set of morphological features that correspond to a specific inflected form, as illustrated by the following example: 走る/ hashi-ru 走った/ hashi-tta v;pst;ipfv where the original form (lemma) "hashi-ru" (走 る, run-prs) is inflected to "hashi-tta" (走った, run-pst) to indicate the past tense (pst) and imperfective aspect (ipfv) as morphological features. the challenge of morphological (re)inflection, which started with the sigmorphon 2016 shared task (cotterell et al., 2016), involves generating an inflected form from a given form and its corresponding morphological feature. this effort has continued over years, covering multiple shared tasks (cotterell et al., 2017, 2018; mccarthy et al., 2019; vylomova et al., 2020; pimentel et al., 2021; kodner et al., 2022; goldman et al., 2023). the sigmorphon-unimorph 2023 shared task 0 (goldman et al., 2023) released a japanese morphology dataset, foot_0 which was automatically extracted from wiktionary. this wiktionary edition, on average, highlights 12 inflected forms for each word. it mainly consists of denominal verbs, which are formed by combining a noun with a light verb, and their inflection patterns are morphologically same as the verb "suru" (do-prs). we propose j-unimorph. it aims to focus on basic verbs found at the n5 level of the japanese language proficiency test (jlpt), and it excludes denominal verbs with identical inflection patterns. our aim was to incorporate a diverse range of expression forms, resulting in an average of 118 inflected forms per word. it includes honorifics, varying levels of politeness, and imperatives with fine-grained distinctions, showcasing the distinctive features of the japanese language. while only a few languages have manually curated uni-morph resources that extend beyond wiktionary, j-unimorph has been carefully designed and created, sharing the same motivation as the project for korean (jo et al., 2023). this paper begins with a brief overview of japanese verbs, detailing the criteria for labeling j-unimorph ( §2). we then explain the data creation process ( §3). as illustrated in figure 1 , this output hashir-anai hashir-imasu hashir-asemasu : watar-emasen table a: conjugation and politeness type inflected form hashir-anai hashir-anai-desu hashir-imasu o-hashir-i-ninarimasu hashir-asemasu hashir-ase-raremashita : watar-emasen labels prefix and suffix v;prs;ipfv;… -anai v;prs;ipfv;… -anai-desu v;prs;ipfv;… -imasu v;form;elev;… o--ininarimasu v;prs;ipfv;… -asemasu v;pst;pfv;… -aseraremashita : : v;prs;ipfv;… -emasen table d: proposed unimorph labels labels inflected form v;prs;… hashir-anai v;prs;… hashir-anai-desu v;prs;… hashir-imasu v;form;… o-hashir-i-ninari… v;prs;… hashir-asemasu v;pst;… hashir-ase-rare… : : v;prs;… watar-emasen verb inflection tool: kamiya-codec conj. type politeness type seed verb regular i basic hashiru regular i basic au regular ii basic taberu regular i respectful (taberu) meshiagaru regular i humble (taberu) itadaku : : : regular i basic wataru r_add l_add r_strip -desu ---ninarimasu o--masu -raremashita --masu : : : ---table c: modification for extra inf. labels inflected form lemma v;prs;ipfv;neg hashir-anai hashiru v;prs;ipfv;pol;neg;col hashir-anai-desu hashiru v;prs;ipfv;pol;foreg hashir-imasu hashiru v;form;elev;prs;ipfv;pol;foreg o-hashir-i-ninarimasu hashiru v;prs;ipfv;caus;pol;foreg hashir-asemasu hashiru v;pst;pfv;caus;pass;pol;foreg hashir-ase-raremashita hashiru : : : v;prs;ipfv;pot;pol;foreg;neg watar-emasen wataru j-unimorph (our dataset) hits inflected form 5,480,000 hashir-anai 497,000 hashir-anai-desu 22,700,000 hashir-imasu 8 o-hashir-i-ninari… 1,170,000 hashir-asemasu 266,000 hashir-ase-rare… : : 230,000 watar-emasen table e: number of google search hits generate manually annotated product copy row replace add row add column discard if @xmath0: -emasen table b: inflection suffix figure 1: overview of the j-unimorph creation process: first, we generate inflected forms from seed verbs (table a, detailed in §3.1) and inflection suffix (table b, detailed in §3.2) using the verb inflection tool, kamiya-codec. this is followed by modifying and adding inflected forms that the tool does not cover (table c, detailed in §3.2). second, japanese native speakers annotate unimorph labels to each form (table d, detailed in §2). finally, we apply a frequency filter to discard infrequent inflected forms (table e, detailed in §3.3). we have released j-unimorph and its interactive visualizer, aiming to provide a useful resource for cross-linguistic studies, japanese language learning support, and various applications. ## how to generate inflected forms the previous section outlined how we matched inflected forms with their unimorph labels. in this section, we will walk through our process for generating all the inflected forms and how we filter out the less common forms, yielding a total of 12,687. ## conclusion we introduced j-unimorph, a japanese morphology dataset based on the unimorph schema. j-unimorph covers a wide range of verb inflection forms, including honorifics, politeness levels, and other linguistic nuances, reflecting the language's agglutinative nature. unlike the wiktionary edition, which is automatically extracted from wiktionary, j-unimorph has been carefully designed by native speakers, featuring an average of 118 inflected forms per word (with a total of 12,687 instances), compared to wiktionary edition's 12 inflected forms per word (12,000 instances in total). j-unimorph, along with its interactive visualizer, has been released to facilitate cross-linguistic research and applications, offering a more comprehensive resource than previously available. omer goldman and reut tsarfaty. 2022. morphology without borders: clause-level morphology. transactions of the association for computational linguistics, 10:1455-1472. david guriel, omer goldman, and reut tsarfaty. 2022. morphological reinflection with multiple arguments: an extended annotation schema and a georgian case study. in proceedings of the 60th annual meeting of the association for computational linguistics (volume 2: short papers), pages 196-202, dublin, ireland. association for computational linguistics. yoshisuke hirabayashi and yumiko hama. 1988. keigo (honorific speech). aratake publishers. eunkyul jo, kim kyuwon, xihan wu, kyungtae lim, jungyeul park, and chulwoo park. 2023. k-unimorph: korean universal morphology and its feature schema. in findings of the association for b examples of contraction form of causative conj. type base ordinary contraction reg. i 書く 書かせる 書かす kak-u kak-ase-ru kak-as-u reg. ii 見る 見させる 見さす mi-ru mi-sase-ru mi-sas-u irreg. 来る 来させる 来さす ku-ru ko-sase-ru ko-sas-u irreg. する させる さす su-ru s-ase-ru sas-u