Joint Word and Morpheme Segmentation with B ayesian Non-Parametric Models (2023)

Abstract: Language documentation often requires segmenting transcriptions of utterances collected on the field into words and morphemes. While these two tasks are typically performed in succession, we study here Bayesian models for simultaneously segmenting utterances at these two levels. Our aim is twofold: (a) to study the effect of explicitly introducing a hierarchy of units in joint segmentation models; (b) to further assess whether these two levels can be better identified through weak supervision. For this, we first consider a deterministic coupling between independent models; then design and evaluate hierarchical Bayesian models. Experiments with two under-resourced languages (Japhug and Tsez) allow us to better understand the value of various types of weak supervision. In our analysis, we use these results to revisit the distributional hypotheses behind Bayesian segmentation models and evaluate their validity for language documentation data.

## introduction in computational language documentation, unsupervised segmentation into words or morphemes foot_0 aims to identify boundaries between units in sequences of symbols, typically corresponding to a phonetic or orthographic transcription of an unsegmented utterance. these tasks are fundamental, as they help to identify and analyse possible dictionary entries. there is a long tradition to handle these tasks with generative probabilistic models @xcite initially designed to model the acquisition of speech by children. the most successful approaches to date rely on nonparametric bayesian models based on dirichlet processes @xcite @xcite and adaptor grammars @xcite @xcite . an interesting property of these generative models is their ability to accommodate existing resources (e.g. partial list of word types) @xcite , which are often available in actual documentation settings @xcite . we study here a scenario where we automatically generate a two-level segmentation, foot_1 identifying simultaneously both word and morpheme boundaries. figure 1 illustrates such a segmentation, where whitespaces separate words, while morphemes are joined with hyphens. our main task is thus to identify two types of boundaries from the unsegmented stream of symbols (first line) to form the two-level segmented sentence (penultimate line). in this work, we only focus on surface segmentation (e.g. eat+ing) as opposed to canonical segmentation (e.g. hike+ing for hiking) @xcite . ## related work word segmentation and morphological segmentation are related tasks; however, both segment only at a single level. we focus here on methods or objectives of approaches comparable to ours. word segmentation with bayesian nonparametric models, on the one hand, benefits from models based on dirichlet processes @xcite , extended with the more general pitman-yor processes and a hierarchical structure @xcite . in language documentation settings, unsupervised methods are applied @xcite . morphological segmentation, on the other hand, usually focuses on the surface segmentation of word types @xcite , with models such as morfessor @xcite . @xcite extensively survey the task for supervised conditions. in low-resource settings, recent works include @xcite @xcite . for both tasks, the adaptor grammar (ag) @xcite , capable of modelling hierarchical structure in sequences with trees, often yields strong results @xcite @xcite , especially thanks to its flexibility in incorporating minimal supervision. for instance, @xcite explicitly model words as a compound of one or more morphemes in their ag. ## conclusion by extending a bayesian non-parametric segmentation model, dpseg, we have proposed two models to simultaneously segment into words and morphemes: one segmenting in parallel and the other in a hierarchical manner. using corpora of two lowresource, morphologically complex languages, we have observed improved performance with respect to the baselines. these two approaches have been contrasted in various ways, leading us to favour the hierarchical approach when supervision is available. the observed improvements are, however, modest, partly due to modelling assumptions that are not fully matched in our data. it remains that sorting words from morphemes based solely on distributional cues is difficult, if possible at all, even with the supervision considered in this work. further studies will need to consider other signals of 'wordness'. some can be extracted from the way units combine with their neighbours, using contextual word models; some will require new sources of supervision, e.g. at the phonological level. another extension will be to distinguish between lexical and grammatical morphemes, which tend to occur and behave differently.