<figure><img src="image_1.png" /></figure>

HAL
open science

# Joint Word and Morpheme Segmentation with Bayesian Non-Parametric Models

Shu Okabe, François Yvon

► To cite this version:

Shu Okabe, François Yvon. Joint Word and Morpheme Segmentation with Bayesian Non-Parametric Models. 17th Conference of the European Chapter of the Association for Computational Linguistics (EACL 2023), Association for Computational Linguistics, May 2023, Dubrovnik, Croatia. pp.628-642. hal-04086368

HAL Id: hal-04086368

https://hal.science/hal-04086368

Submitted on 2 May 2023

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution - ShareAlike 4.0 International License
# Joint Word and Morpheme Segmentation with Bayesian Non-Parametric Models

## Shu Okabe

Univ. Paris-Saclay & CNRS
LISN, rue du Belvédère
91405 Orsay, France
shu.okabe@lisn.fr

## François Yvon

Univ. Paris-Saclay & CNRS
LISN, rue du Belvédère
91405 Orsay, France
francois.yvon@limsi.fr

## Abstract

Language documentation often requires segmenting transcriptions of utterances collected on the field into words and morphemes. While these two tasks are typically performed in succession, we study here Bayesian models for simultaneously segmenting utterances at these two levels. Our aim is twofold: (a) to study the effect of explicitly introducing a hierarchy of units in joint segmentation models; (b) to further assess whether these two levels can be better identified through weak supervision. For this, we first consider a deterministic coupling between independent models; then design and evaluate hierarchical Bayesian models. Experiments with two under-resourced languages (Japhug and Tsez) allow us to better understand the value of various types of weak supervision. In our analysis, we use these results to revisit the distributional hypotheses behind Bayesian segmentation models and evaluate their validity for language documentation data.

## 1 Introduction

In computational language documentation, unsupervised segmentation into words or morphemes<fnref n="1" /> aims to identify boundaries between units in sequences of symbols, typically corresponding to a phonetic or orthographic transcription of an unsegmented utterance. These tasks are fundamental, as they help to identify and analyse possible dictionary entries. There is a long tradition to handle these tasks with generative probabilistic models (Brent, 1999; Venkataraman, 2001) initially designed to model the acquisition of speech by children. The most successful approaches to date rely on non-parametric Bayesian models based on Dirichlet Processes (Goldwater et al., 2006, 2009; Godard et al., 2016) and Adaptor Grammars (Johnson et al., 2007; Eskander et al., 2016; Godard et al., 2018;

Eskander et al., 2019). An interesting property of these generative models is their ability to accommodate existing resources (e.g. partial list of word types) (Sirts and Goldwater, 2013; Ruokolainen et al., 2016), which are often available in actual documentation settings (Bird, 2020).

We study here a scenario where we automatically generate a two-level segmentation,<fnref n="2" /> identifying simultaneously both word and morpheme boundaries. Figure 1 illustrates such a segmentation, where whitespaces separate words, while morphemes are joined with hyphens. Our main task is thus to identify two types of boundaries from the unsegmented stream of symbols (first line) to form the two-level segmented sentence (penultimate line). In this work, we only focus on surface segmentation (e.g. eat+ing) as opposed to canonical segmentation (e.g. hike+ing for hiking) (Cotterell et al., 2016).

<table><caption>Figure 1: Example of two segmentation levels in Japhug: words are separated by whitespaces (‘ ’) and morphemes by hyphens (‘-’). Extract from (Jacques, 2021)</caption><thead><tr><td>Segmentation</td><td>Sentence</td></tr></thead><tbody><tr><td>Unsegmented</td><td>u̯zokuatcuipuwwysumtoa</td></tr><tr><td>Word</td><td>u̯zo ku̯ atcu puwwysumtoa</td></tr><tr><td>Morpheme</td><td>u̯zo-ku̯-atcu-pu̯-wy-su̯-mto-a</td></tr><tr><td>Two-level</td><td>u̯zo ku̯ a-tcu pu̯-wy-su̯-mto-a</td></tr><tr><td>Translation</td><td>He let me see my son.</td></tr></tbody></table>

The motivation for this task is two-fold: (a) to evaluate our ability to obtain annotations such as Figure 1 in an unsupervised way; (b) to see how much the two-level model can disambiguate word from morpheme boundaries, thus improving word segmentations. Note that in actual documentation settings, the annotation of morpheme boundaries

<footnote n="1">¹In this paper, our position regarding the notions of ‘words’ and ‘morphemes’ is entirely empirical, as we mainly try to reproduce annotations performed by field linguists.</footnote>

<footnote n="2">²This ‘two-level segmentation’ is unrelated to the ‘two-level morphology’ (Koskenniemi, 1983), which describes the association between surface forms and underlying representations using the formalism of extended rational expressions.</footnote>
is usually performed on utterances that are already segmented into words, hence the need to optimise this step.

A baseline for this task is a two-pass approach: first, identify putative word boundaries, then iterate the segmentation procedure on the corresponding set of word types. As we discuss below, unsupervised word segmentation procedures tend to generate units that are often halfway between morphemes and words (see e.g. (Goldwater et al., 2009) or (Godard et al., 2016) who report oversegmentation for words). This means that the first pass often delivers units that are too short and inadequate for the latter processing step. This remains true even with partial supervision information at the word level (Okabe et al., 2022).

We therefore study models that explicitly distinguish between words and morphemes, considering both the fully unsupervised and the minimally supervised settings. The research questions that we address are the following:

* RQ1: Bayesian segmentation models identify units based solely on distributional properties, identifying units that are often in between words and morphemes. Can we improve both segmentations through an explicit modelling of these two levels?

* RQ2: a simple baseline is to first segment sentences into words, then to segment each word type<fnref n="3" /> identified in the first step into morphemes. A second question is how much a single joint segmentation model can mitigate the error propagation of this two-step baseline.

* RQ3: there are multiple ways to implement and supervise joint segmentation models, an important distinction being between linear (flat) and hierarchical segmentation models. A third question relates to the strengths and weaknesses of these approaches, both in the presence and absence of supervision.

* RQ4: Bayesian segmentation models primarily rely on distributional properties of characters in morphemes and words, and embed specific assumptions regarding these distributions. We last question the validity of these assumptions in a low-resource language documentation context.

More generally, our main goal in this study is to assess whether statistical cues alone are sufficient to identify two distinct segmentation levels. To answer this question, we analyse several simple joint segmentation models introduced in Section 2 and experiment with two under-resourced languages, briefly presented in Section 3. Our main results and analyses are in Section 4. From a practical perspective, our objective is not to devise directly-usable models for field work but to observe the effect of introducing a subword level of segmentation in Bayesian non-parametric models, especially in very low-resource situations as in language documentation: will it improve the (original) word-level segmentation quality? How can additional resources help?

# 2 Segmentation models

## 2.1 One-level segmentation

For this work, we use our own Python implementation<fnref n="4" /> of the unigram version of Goldwater et al.'s (2009) model: dpseg. This model relies on Dirichlet Processes to evaluate the probability of a word sequence, as we briefly recall below. In dpseg, the probability of a new occurrence $w$, based on the observed past words, is expressed through Equation (1) where $w$ denotes a word $w = c_1...c_L$ comprising $L$ characters:

$$P(w|h^{-}; \alpha) = \frac{n_{w}^{(h^{-})} + \alpha P_{0}(w|h^{-})}{n^{-} + \alpha} \quad (1)$$

Here, $h^{-}$ denotes the rest of the text ($w$ excluded), $n_{w}^{(h^{-})}$ the frequency of word $w$ in the text, and $n^{-}$ the total number of words. $\alpha$ is the concentration parameter and $P_0$, the base distribution, is defined by Equation (2):

$$P_0(w) = p_{\#}(1 - p_{\#})^{(L-1)} * \prod_{l=1}^{L} P_c(c_l), \quad (2)$$

with $p_{\#}$ the probability to terminate a word and $P_c$ a distribution over the set of characters, assumed uniform in the dpseg model.

Observing an unsegmented character string $c_1...c_T$, word segmentation can be formalised with a latent variable model, introducing unobserved boundary variables $b_1...b_T$, where $b_t = 1$ (resp. $b_t = 0$) respectively denotes presence or absence of a boundary after $c_t$. The inference is typi-

<footnote n="3">³Types denote unique words, as opposed to tokens, which encompass all running occurrences of types in a corpus.</footnote>

<footnote n="4">⁴Available at https://github.com/shuokabe/pyseg.</footnote>
cally performed with Gibbs sampling, using Equation (1) to iteratively resample the latent boundary variables values. To speed up convergence, Goldwater et al. (2009) additionally use simulated annealing.

We chose to explore dpseg over alternative segmentation models such as SentencePiece (Kudo and Richardson, 2018) or Morfessor (Creutz and Lagus, 2002; Smit et al., 2014) because of its better performance in similar language documentation contexts. It is also well suited to small data conditions and enables weak supervision (Okabe et al., 2022). Furthermore, preliminary experiments showed no major difference between using a Dirichlet process (DP), as we do, and a variant based on a Pitman-Yor process (PYP), known to better capture the underlying power-law distribution. Overall, we believe that using more sophisticated variants or faster implementations of dpseg would not substantially alter our main observations.

## 2.2 Pipeline model: two-step segmentation

We now turn to models computing a segmentation in words and morphemes. Our baseline two-level model combines in a pipeline two dpseg models: the first inputs unsegmented text and yields a word-level segmentation. The *word types* in this segmentation are then collected and processed by a second dpseg to get the morpheme-level segmentation. By design, in this approach, a word type is always associated with a unique morphological analysis.<fnref n="5" />

## 2.3 Flat segmentations with coupling

Two-level segmentation can also be formalised with latent variables, using two sets of variables, denoted as {$b_1^w ... b_T^w$} (resp. {$b_1^m ... b_T^m$}) for word (resp. morpheme) boundaries. Obviously, using the same dpseg model to independently sample these variables will produce indistinguishable segmentations. It is, however, possible to get two-level segmentations by introducing interactions between these two models, so that the values of variables $b_t^w$ and $b_t^m$ are no longer independent. Deterministic interactions can be introduced in two ways which both ensure that word and morpheme segmentation hypotheses always remain consistent: by imposing either i) that word boundaries also correspond to morpheme boundaries, or ii) that morpheme internal positions are also considered word internal.

In strategy i), we first sample boundary variables for words and then for morphemes, yielding the parallel-w approach. If a word boundary is detected ($b_t^w = 1$), then we deterministically identify a morpheme boundary at that position ($b_t^m = 1$). Otherwise ($b_t^w = 0$), we sample the value for $b_t^m$ as usual. The net effect is to make morpheme boundaries more likely than in an independent model and generate shorter units at the morpheme level; no change is expected at the word level. In strategy ii), denoted parallel-m, morpheme variables are sampled first: if a boundary is detected ($b_t^m = 1$), an extra sample decides the value of $b_t^w$; else, we readily assign $b_t^w = 0$. Here, the effect is reversed and makes word boundaries less likely, forcing the word model to generate longer units; the morpheme-level segmentation remains unchanged.

## 2.4 Hierarchical segmentations

Inspired by (Mochihashi et al., 2009), we also implement hierarchical segmentation models for the two-level segmentation task. These models aim to explicitly represent the structured aspect of the double segmentation process. Here, the word model is nearly identical to the basic version of dpseg, with a change in the base distribution $P_0$ of Equation (1). The character model ($P_c$) is replaced by a second non-parametric model for morphemes (hence the hierarchical nature of the model), also based on dpseg. This morpheme model has a base distribution that is, as for the original dpseg, a unigram character model.

Considering a word $w$ (of length $L$) made of $K$ morphemes, $w = m_1...m_K$, by analogy to Equation (2), $P_0$ is therefore changed to:

$$P_0^w(w|h^-) = p_\#(1-p_\#)^{(L-1)} * \prod_{k=1}^{K} P^m(m_k|h^-), \quad (3)$$

where $P^m(m_k)$ is the probability of morpheme $m_k$ according to the morpheme model (the standard dpseg model), which is written as follows:

$$P^m(m_k|h^-; \alpha^m) = \frac{n_{m_k}^{(h^-)} + \alpha^m P_0^m(m_k)}{n_m^- + \alpha_m}, \quad (4)$$

where $\alpha^m$ and $P_0^m$ are, respectively, the concentration parameter and the base distribution for morphemes—the latter being a uniform character model. Sampling in this model is implemented as follows: each time a new word is hypothesised, a morpheme segmentation is obtained from the morpheme model; for words that are actually retained,

<footnote n="5">⁵This hypothesis corresponds to what we observe in our corpora, where only a few dozen words occur with more than one segmentation in morphemes.</footnote>
this segmentation is recorded and used for further
occurrences of the same word form. A basic ver-
sion of this approach (denoted hier-type) thus
samples boundary variables for morphemes only
once for each word type.<fnref n="6" /> Two variants are consid-
ered: in the hier-iter model, morpheme bound-
aries are iteratively resampled for all existing word
types every *k* iterations of the word-level Gibbs
sampler; in hier-final, this process is only per-
formed once after convergence of the word model,
to make a fair comparison with the pipeline model
of Section 2.2. As in the pipeline model, these
approaches ensure that all occurrences of a given
word type will have the same morphological de-
composition.

## 2.5 Unsupervised Adaptor Grammars

Another hierarchical baseline is based on the Adap-
tor Grammar (AG) model of (Johnson et al., 2007).
This is a strong unsupervised word segmentation
model that can also capture morphological struc-
ture to some extent. We use the colloc grammar
in (Johnson, 2008), which considers the follow-
ing levels: a Sentence is made of Collocations,
which are made of Words, themselves composed
of Characters. In the same manner as (Johnson,
2008, Section 3.2), we considered the Collocation
tier to correspond to words and the Word tier to
morphemes.⁷

## 2.6 Weak supervision

Following (Okabe et al., 2022), we further consider
two types of realistically available resources that
can supervise the segmentation process. The first
takes the form of a small number of segmented sen-
tences (e.g. from previously annotated texts), where
the corresponding boundary variables are observed.
During Gibbs sampling, we skip these positions
and simply use the observed values (0 or 1). This
type of supervision, which gives information at the
*token* level, is denoted *sentence*.

A second type of resource corresponds to lists
of lexical units (words and/or morphemes). We
use them to replace in $P_0$ the uniform model with
a bigram model, thus increasing the likelihood of
known units. This supervision method which uses
knowledge about *types* is denoted *dictionary*.

Observed word segmentation or the word dic-
tionaries will be used to compute $P^w(w|h^{-}, \alpha)$

(Equation (1)). Likewise, observed morpheme
boundaries or morpheme lists will be taken into
account at the morpheme level (e.g. in Equation (4)
for the hierarchical model). In all our experiments,
we assume that weak supervision is available si-
multaneously at the word and morpheme levels.

## 2.7 Full supervision

An even more favourable situation is when bound-
aries are fully observed for a sufficiently large set of
sentences, warranting the use of supervised learn-
ing techniques such as Conditional Random Fields
(Lafferty et al., 2001). This situation is studied
notably by (Moeller and Hulden, 2018; Kann et al.,
2018). Our experiments with this setting show
that this procedure is sample efficient. It is, how-
ever, also subject to the same confusion between
word and morpheme boundaries and does not sig-
nificantly outperform the weak supervision setting.
Full results are reported in Appendix D.

# 3 Experimental protocol and material

## 3.1 Evaluation metrics

Following (Goldwater et al., 2006), the segmenta-
tion outputs are evaluated with F-scores on the two
levels of segmentation (word and morpheme) at
three tiers: BF at the boundary level obtained by
comparing predicted and actual boundary values
(0 or 1), WF for the token level, which focuses on
the correspondence between each unit in the sen-
tences, and LF for the lexicon level, counting the
matches between unit types collected on the whole
text. For finer analyses, we also report the precision
and recall for all three levels in Appendix D.

In addition, some basic statistics regarding the
texts will be presented for both segmentation levels.
$N_{utt}$, $N_{type}$, and $N_{token}$ respectively denote the
number of utterances, unit types, and tokens in the
text. We also report the inferred average token
(WL) and type (TL) lengths.

## 3.2 Linguistic material

This work studies two low-resource languages: Jap-
phug and Tsez.

Japhug is a Sino-Tibetan language from the Gyal-
rong family spoken in the Sichuan province in
China. It notably has a rich morphology for both
nouns and verbs. For example, verbs can use sev-
eral prefixes to express tense or aspect features on
top of suffixes. Japhug is currently being docu-
mented: recordings, annotated corpora, and dictio-

<footnote n="6">⁶This is slightly more subtle, as the same word can be
created then deleted during the Gibbs sampling iterations.

⁷Appendix C details the hyperparameter values.</footnote>
naries are available in Pangloss.<fnref n="8" /> Jacques (2021) comprehensively describes the language. The corpus is composed of all the Japhug examples from the LATEX source files of this grammar book.<fnref n="9" /> The extraction of those sentences is made easier thanks to the \gll command before the Japhug sentences.

Tsez is a Caucasian language part of the Nakh-Daghestanian language family, spoken in the Republic of Dagestan in Russia. It is officially an unwritten language, transcribed and transliterated through the Avar writing system (Comrie and Polinsky, forthcoming). Nouns and verbs are mainly inflected with a variety of combined suffixes. Moreover, Tsez features a set of clitics that are merged with the words. The latest grammar is currently in the process of being published. The only substantial dictionary of the language contains around 7,500 entries. The Tsez corpus contains sentences from the Tsez Annotated Corpus of (Abdulaev and Abdulaev, 2010), used in (Zhao et al., 2020) to study the generation of interlinear glosses.

<table><caption>Table 1: Statistics for the Japhug and Tsez corpora. Both are segmented into words and morphemes (morph.).</caption><thead><tr><th rowspan="2">language segment</th><th colspan="2">Japhug</th><th colspan="2">Tsez</th></tr><tr><th>word</th><th>morph.</th><th>word</th><th>morph.</th></tr></thead><tbody><tr><td>N<sub>utt</sub></td><td>3628</td><td>3628</td><td>2000</td><td>2000</td></tr><tr><td>WL</td><td>4.73</td><td>2.90</td><td>5.61</td><td>2.81</td></tr><tr><td>TL</td><td>7.30</td><td>5.41</td><td>6.93</td><td>5.21</td></tr><tr><td>N<sub>type</sub></td><td>6739</td><td>2731</td><td>5732</td><td>1603</td></tr><tr><td>N<sub>token</sub></td><td>28579</td><td>46632</td><td>20153</td><td>40229</td></tr><tr><td>N<sub>super.</sub></td><td>664</td><td>493</td><td>867</td><td>455</td></tr></tbody></table>

Table 1 describes the two language corpora, reporting statistics at the two segmentation levels. Japhug word types have an average number of 2.48 morphemes, while in Tsez, that value is 2.37.

For weak supervision, the first 200 sentences of each corpus are selected as training material, used as is for boundary supervision (sentence method) or as a list of unique (word or morpheme) types for lexical supervision (dictionary method). N<sub>super.</sub> above summarises the number of supervision units.

### 3.3 Experimental settings

In our experiments, the results are obtained after 20,000 iterations of Gibbs sampling, with 10 increments of simulated annealing, for quicker convergence as detailed in (Goldwater et al., 2009). The last iteration of a run returns the final boundary prediction that is considered to be the model output. To account for the variability of the sampler, we report below the average of three runs. We find that this segmentation procedure is stable with an average standard deviation of less than 1 for all metrics.

We use the default values of the base dpseg for hyperparameters: p<sub>#</sub> = 0.5 and α = α<sup>m</sup> = 20. We set the same initial value of the concentration parameter for the two levels in both categories of model. Following Teh (2006) and Mochihashi et al. (2009), the two concentration parameters, which both have a Gamma posterior distribution, are resampled after each iteration on the corpus—thus, upon convergence, we observe α ≠ α<sup>m</sup>.

For the hierarchical models, hier-final re-segments word types into morphemes with 1,000 iterations of Gibbs sampling, while hier-iter carries out 5 iterations of morphological segmentation every 100 iterations of word segmentation.

## 4 Experimental results

### 4.1 RQ1: unsupervised two-level models

Table 2 reports segmentation results for the Japhug corpus. The corresponding results for Tsez are in Table 6 in Appendix D. As briefly stated in the introduction, the one-level dpseg segments into units that are too short for words (cf. average unit lengths WL and TL) and seems to segment units that are closer to morphemes, with higher morpheme F-scores for all three evaluation tiers. This motivated our work on two-level segmentation models, which, contrarily to the basic dpseg, make a distinction between the two types of boundaries. The more sophisticated AG model shows a similar trend, out-putting words and morphemes that are too short, insufficiently diverse (low LF), and result in too many tokens (excessive N<sub>token</sub>).

The pipeline approach only differs from the one-level dpseg at the morpheme level, where we see worse F-scores, with a massive drop in LF score. For the ‘parallel’ models, the expected improvements are observed: better morpheme boundaries for parallel-w, better word boundaries for parallel-m. However, these two models deliver units that remain quite close in average length, and the F-score improvements remain rather limited in magnitude. In those experiments, the hierarchical

<footnote n="8">⁸https://pangloss.cnrs.fr/corpus/Japhug.</footnote>

<footnote n="9">⁹https://github.com/langsci/295/.</footnote>
<table><caption>Table 2: Results on the Japhug corpus for unsupervised one-level (*) and two-level dpseg models. The reference contains 6,739 words and 2,731 morphemes (N<sub>type</sub>). **Bold** numbers represent the best result per metrics.</caption><thead><tr><th rowspan="2">model level</th><th colspan="2">AG</th><th colspan="2">dpseg*</th><th colspan="2">pipeline</th><th colspan="2">parallel-w</th><th colspan="2">parallel-m</th><th colspan="2">hier-type</th><th colspan="2">-final</th><th colspan="2">hier-iter</th></tr><tr><th>word</th><th>morph.</th><th>word</th><th>morph.</th><th>word</th><th>morph.</th><th>word</th><th>morph.</th><th>word</th><th>morph.</th><th>word</th><th>morph.</th><th>word</th><th>morph.</th><th>word</th><th>morph.</th></tr></thead><tbody><tr><td>BF</td><td>71.0</td><td>83.4</td><td>73.1</td><td>81.0</td><td>73.1</td><td>80.6</td><td>73.3</td><td><strong>83.6</strong></td><td>73.2</td><td>80.8</td><td><strong>74.7</strong></td><td>62.5</td><td>82.3</td><td>73.5</td><td>81.4</td></tr><tr><td>WF</td><td>45.8</td><td><strong>62.5</strong></td><td>46.2</td><td>55.1</td><td>46.2</td><td>57.8</td><td>46.5</td><td>61.3</td><td>46.0</td><td>54.7</td><td><strong>48.7</strong></td><td>24.5</td><td>60.9</td><td>47.2</td><td>59.1</td></tr><tr><td>LF</td><td>31.1</td><td>28.9</td><td>20.4</td><td><strong>41.4</strong></td><td>20.4</td><td>17.5</td><td>20.6</td><td>40.5</td><td>23.8</td><td>41.4</td><td>28.8</td><td>17.0</td><td>23.4</td><td><strong>31.7</strong></td><td>24.7</td></tr><tr><td>WL</td><td>4.72</td><td>2.51</td><td>3.34</td><td></td><td>3.34</td><td>2.13</td><td>3.34</td><td>2.98</td><td>3.73</td><td>3.35</td><td>3.93</td><td>1.65</td><td>2.32</td><td>4.29</td><td>2.37</td></tr><tr><td>TL</td><td>6.60</td><td>3.27</td><td>4.22</td><td></td><td>4.22</td><td>2.64</td><td>4.23</td><td>3.99</td><td>4.77</td><td>4.21</td><td>4.78</td><td>2.83</td><td>2.87</td><td>5.12</td><td>2.88</td></tr><tr><td>N<sub>type</sub></td><td>5582</td><td>1113</td><td>2260</td><td></td><td>2260</td><td>694</td><td>2257</td><td>1834</td><td>2921</td><td>2281</td><td>3806</td><td>1013</td><td>911</td><td>4925</td><td>956</td></tr><tr><td>N<sub>token</sub></td><td>28.6k</td><td>53.9k</td><td>40.5k</td><td></td><td>40.5k</td><td>63.4k</td><td>40.5k</td><td>45.4k</td><td>36.3k</td><td>40.4k</td><td>34.4k</td><td>82.1k</td><td>58.2k</td><td>31.5k</td><td>57.0k</td></tr></tbody></table>

models make a stronger distinction between the two types of units, yielding well-differentiated average lengths (WL and TL). Overall, almost all two-level models but the simple-minded pipeline improve the baseline scores for at least one level of segmentation, with the unsupervised parallel-w flat model delivering the best results on average.

While our answer to RQ1 is positive, we note that the score differences between approaches are often small and that all models keep oversegmenting words, leading to a too low number of word types and yielding poor LF scores. The same trend is observed for the hierarchical models at the morpheme level: they find too few morphemes (cf. N<sub>type</sub>) and result in poor type-level scores.

## 4.2 RQ2: error propagation

Compared to the baseline, the unsupervised pipeline approach obtains poor LF score at the morpheme level (Table 2). As the two approaches have almost identical BF and WF scores, this means that pipeline performance is mostly due to its ability to detect frequent morphemes at the expense of rarer ones. This is also reflected by the very small number of morpheme types found by this model.

This is because the pipeline model uses the word types computed by the regular dpseg to detect morpheme boundaries. As this first step obtains poor results (WF ≈ 20), cascading errors accumulate. Wrong detections at the word level are thus counted twice: once at the word level, once at the morpheme level. The use of joint models slightly remeDES this state of play, yielding improvements in the word dictionary, which then turn into improved morpheme dictionaries. This allows us to answer RQ2 positively, even though the recall for morpheme types still remains far from satisfactory. To progress on that front, the surest way seems to improve word segmentation, if only because many word types are made of one single morpheme.

## 4.3 RQ3: flat and hierarchical models

This section compares the flat (parallel) and hierarchical models, first analysing the differences between variants of the same family, before comparing these two approaches.<fnref n="10" />

**Parallel models** As explained in § 2.3, each ‘parallel’ model only improves the baseline for one type of unit: morpheme boundaries for parallel-w and word boundaries for parallel-m (Table 2). This remains true when using weak supervision. A first comparison is between the parallel models, where we see better scores for parallel-w, which outperforms parallel-m on almost all accounts and all weak supervision settings. In fact, even with the help of supervision, parallel-m obtains lower BF and WF scores at the word level than parallel-w: more word types are generated, the average length is increased, but these hypotheses are often wrong. We do not see the reverse for parallel-w, which generates fewer morphemes: the decrease in recall is almost balanced by the increase in precision, with little negative impact on the morpheme segmentation quality.

**Hierarchical models** First, for all three F-scores at the morpheme level, in any experimental situation, the hier-type model is consistently worse than the hier-final model, which carries out additional Gibbs sampling steps for the morpheme variables once the word boundaries have stabilised. This model finds longer units (cf. WL) with the additional iterations, which leads to significant improvements (+20 points in WF).

The hier-iter variant achieves a fair trade-off between the boundary and token F-scores on the one hand, and the type F-score on the other hand: this model is better when evaluated at the type level, while hier-final reaches better scores on the other two levels. As the hier-final model

<footnote n="10">¹⁰Full results in Appendix D.</footnote>
<figure><img src="image_3.png" /><figcaption>Figure 2: Results in Japhug without supervision on the left, with sentence supervision in the middle, and dictionary supervision on the right of each subplot (row: segmentation level, column: F-score). We use a different y-axis scale for each F-score.</figcaption></figure>

attains a similar but higher aggregated F-score on average, we chose it to represent the hierarchical models for the following sections. This model is also slightly less computationally involved than *hier-iter*, another reason for choosing it in practical settings.

**Comparing two types of models** Figure 2 displays the results for the baselines and best performing flat and hierarchical models on Japhug with and without supervision, illustrating the impact of resources. Four models are compared: *pipeline*, *parallel-w*, *hier-final*, and *independent*. The latter corresponds to *two distinct* dpseg models, one trained for word boundaries, the other for morpheme boundaries, each supervised and evaluated at the corresponding level. It can produce inconsistent segmentations.

By design, *independent*, *pipeline*, and *parallel-w* generate similar word-level segmentations and improve a lot from *dictionary supervision*. At the morpheme level, the latter model strongly improves its LF score, equally benefiting from both weak supervision strategies.

The hierarchical model has the best results for word-level scores with sentence supervision, whereas, with *dictionary* supervision, it lags behind the other methods. At the morpheme level, results are less clear. When unsupervised,

*hier-final* is better than the baselines but worse than *parallel-w*; however, it always gets a strong boost from supervision, more so than its contenders. In short, sentence is more beneficial for the hierarchical model, while *dictionary* rather improves the others. Still, these increments remain small; we conclude that weak supervision does not seem to help the models better differentiate the two types of units. Overall, when aggregating F-scores across settings and languages, models rank as follows, from worst to best: *independent*, *pipeline*, *parallel-w*, and *hier-final*. This answers RQ3.

## 4.4 RQ4: distributional assumptions

### 4.4.1 Word distributions in CLD

The parallel and hierarchical models both rely on the same fundamental assumption: the distribution of word tokens in a natural corpus follows a power law, which was a motivation for using Dirichlet processes in (Goldwater et al., 2006). As described in (Goldwater et al., 2011), such distributions derive from the use of a two-stage model: a *generator* which focuses on creating word types (this is $P_0$ in the dpseg model) and an *adaptor* that produces the ‘rich-get-richer’ effect (Equation (1)).

To check how well our data matches this assumption, in Figure 3, we look at type/token curves, which display the number of word types in texts
of increasing lengths. We deem this ratio to be a
reasonable proxy to observe the ‘rich-get-richer’
effect on word types. We compare the Japhug and
Tsez texts, their automatic segmentations (‘dp-’),
as well as their English translation (‘-en’) (as in
(Godard et al., 2016)), with five languages of vary-
ing morphological complexity: English, French,
Finnish, German, and Turkish. For these, we use
the 2020 news data from the Leipzig corpus (Gold-
hahn et al., 2012), keeping only the first 2,000 sen-
tences for comparison.

<figure><img src="image_4.png" /><figcaption>Figure 3: Type-token curves for several languages</figcaption></figure>

We see that the curves for Japhug and Tsez fol-
low the French and English trends, reflecting a
lesser lexical variation than for German, Turkish,
and Finnish. Looking at their English translations
confirms this trend and hints that the number of
word types in our corpora does not correctly mir-
ror the actual morphological complexity of these
languages. Indeed, corpora collected for language
documentation may present distributional biases:
sentences are often chosen to illustrate relevant lin-
guistic properties, as in our Japhug corpus extracted
from a grammar book. This reduced lexical vari-
ety is amplified in automatically segmented texts,
where we fail to identify most rare words. For ex-
ample, 97% of the words occurring only once are
not found by the unsupervised *hier-final* model.
See Appendix A for another view of the same phe-
nomena.

4.4.2 Modelling morpheme distributions

Where the parallel and hierarchical versions dif-
fer is how they estimate morpheme models:
parallel-w assumes a power law of morphemes
in running texts, while hier-final assumes it on
word types. We see the impact of these assump-
tions in Figure 4. This graph is based on an esti-

mation of the parameter of the Zipf distributions
of words in the Tsez corpus and of morphemes in
the Tsez word types (see details in Appendix A).
While these parameters strongly depend on the cor-
pus size, they are typically in the range [-1, -1.2]
(Baayen, 2001) — the lower value computed for
the reference Tsez word distribution again hints
at the peculiarity of this distribution, whereas the
corresponding parameters for morpheme are in the
right ballpark.

All inferred segmentations at the word level be-
have similarly, with values steeper than for the
reference, reflecting the effect of using a power-
law model. Once more, we see that supervision is
hardly helping. We observe sharper differences at
the morpheme level, where the hierarchical model
gets much closer to the reference, further boosted
by sentence supervision. This is in line with (Vir-
pioja et al., 2011), which notes the better mor-
pheme segmentations obtained when modelling
types rather than tokens.

<figure><img src="image_5.png" /><figcaption>Figure 4: Zipfianity of various segmentations of Tsez. par is based on parallel-w, htl on hier-final.</figcaption></figure>

5 Related work

Word segmentation and morphological segmenta-
tion are related tasks; however, both segment only
at a single level. We focus here on methods or
objectives of approaches comparable to ours.

Word segmentation with Bayesian non-
parametric models, on the one hand, benefits from
models based on Dirichlet Processes (Goldwater
et al., 2006, 2009), extended with the more general
Pitman-Yor Processes and a hierarchical structure
(Teh, 2006; Mochihashi et al., 2009). In language
documentation settings, unsupervised methods
are applied (Godard et al., 2016). Morphological
segmentation, on the other hand, usually focuses
on the *surface* segmentation of word types (Cot-
terell et al., 2016), with models such as Morfessor
(Creutz and Lagus, 2002). Ruokolainen et al.
(2016) extensively survey the task for supervised
conditions. In low-resource settings, recent works
include (Kann et al., 2018; Liu et al., 2021; Moeng
et al., 2021).

For both tasks, the Adaptor Grammar (AG)
(Johnson et al., 2007), capable of modelling hi-
erarchical structure in sequences with trees, of-
ten yields strong results (Johnson, 2008; Eskander
et al., 2016; Godard et al., 2018), especially thanks
to its flexibility in incorporating minimal super-
vision. For instance, Sirts and Goldwater (2013)
explicitly model words as a compound of one or
more morphemes in their AG.

6 Conclusion

By extending a Bayesian non-parametric segmenta-
tion model, dpseg, we have proposed two models
to simultaneously segment into words and mor-
phemes: one segmenting in parallel and the other
in a hierarchical manner. Using corpora of two low-
resource, morphologically complex languages, we
have observed improved performance with respect
to the baselines. These two approaches have been
contrasted in various ways, leading us to favour the
hierarchical approach when supervision is avail-
able. The observed improvements are, however,
modest, partly due to modelling assumptions that
are not fully matched in our data. It remains that
sorting words from morphemes based solely on dis-
tributional cues is difficult, if possible at all, even
with the supervision considered in this work.

Further studies will need to consider other sig-
nals of ‘wordness’. Some can be extracted from
the way units combine with their neighbours, using
contextual word models; some will require new
sources of supervision, e.g. at the phonological
level. Another extension will be to distinguish be-
tween lexical and grammatical morphemes, which
tend to occur and behave differently.

Limitations

The main limitation comes from the use of the un-
igram dpseg model. Although it has strong and
stable performance on the word-level segmenta-
tion task, comparable to its bigram version in our
settings (Godard et al., 2016), some weaknesses
inherent to the unigram assumption appear as in
Appendix B. Moreover, such an assumption at the

morpheme level means that, for example, adding a
distinction between lexical and grammatical mor-
phemes, as suggested in conclusion, will be of little
use since the probability of a morpheme does not
affect that of others in the word for unigram mod-
els. Nevertheless, in our language documentation
setting, we deem this unigram assumption to have
a small impact on the overall results due to data
size.

For some of our two-level models (pipeline and
hierarchical), we also relied on the assumption that
a word can only have a single morphological de-
composition, as stated in Sections 2.2 and 2.4. Al-
though it may not apply in other situations, this
reasonably holds in our two corpora (as briefly ex-
plained in footnote 5) since we found 51 word types
with several morphological analyses in Japhug and
14 in Tsez.

Besides, our work and observation only rely on
two languages. However, the two-level segmen-
tation for very low-resourced languages, as we
displayed, needs a reference text segmented with
distinct boundaries for words and morphemes for
evaluation in particular. Since word segmentation
usually focuses on tokens in sentences and mor-
pheme segmentation on word types, texts explicitly
segmented in two levels are difficult to obtain, even
so of good quality.

Finally, we reckon that our current implementa-
tion of the Gibbs sampler is not particularly op-
timised. For actual deployment, these models
should be designed and implemented in a more
computationally-efficient way or even another lan-
guage than Python.

Acknowledgements

This work was partly funded by French ANR and
German DFG under grant ANR-19-CE38-0015
(CLD 2025). The authors wish to thank the anony-
mous reviewers, Laurent Besacier for his feedback,
Guillaume Jacques for the Japhug corpus, and An-
tonios Anastasopoulos for the Tsez corpus.

References

Asen' K. Abdulaev and I. K. Abdulaev. 2010. *Cezjas* *fol'klor*: (gíurus mecrek° iorno butirno) = *Dido* (Tsez) *folklore* = *Didojskij* (cezskij) *fol'klor*. Lotos, Leipzig.

R. Harald Baayen. 2001. *Word Frequency Distributions*, volume 18 of *Text, Speech and Language Technology*. Springer Netherlands, Dordrecht.
Steven Bird. 2020. Decolonising Speech and Language Technology. In *Proceedings of the 28th International Conference on Computational Linguistics*, pages 3504–3519, Barcelona, Spain (Online). International Committee on Computational Linguistics.

Michael R. Brent. 1999. An efficient, probabilistically sound algorithm for segmentation and word discovery. *Machine Learning*, 34(1-3):71–105.

Bernard Comrie and Maria Polinsky. forthcoming. Tsez. In Yuri Koryakov, Yury Lander and Timur Maisak (eds.) The Caucasian Languages. An International Handbook. Mouton. HSK series.

Ryan Cotterell, Tim Vieira, and Hinrich Schütze. 2016. A joint model of orthography and morphological segmentation. In *Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies*, pages 664–669, San Diego, California. Association for Computational Linguistics.

Mathias Creutz and Krista Lagus. 2002. Unsupervised discovery of morphemes. In *Proceedings of the ACL-02 Workshop on Morphological and Phonological Learning*, pages 21–30. Association for Computational Linguistics.

Ramy Eskander, Francesca Callejas, Elizabeth Nichols, Judith Klavans, and Smaranda Muresan. 2020. MorphAGram, evaluation and framework for unsupervised morphological segmentation. In *Proceedings of the 12th Language Resources and Evaluation Conference*, pages 7112–7122, Marseille, France. European Language Resources Association.

Ramy Eskander, Judith Klavans, and Smaranda Muresan. 2019. Unsupervised morphological segmentation for low-resource polysynthetic languages. In *Proceedings of the 16th Workshop on Computational Research in Phonetics, Phonology, and Morphology*, pages 189–195, Florence, Italy. Association for Computational Linguistics.

Ramy Eskander, Owen Rambow, and Tianchun Yang. 2016. Extending the use of Adaptor Grammars for unsupervised morphological segmentation of unseen languages. In *Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers*, pages 900–910, Osaka, Japan. The COLING 2016 Organizing Committee.

Pierre Godard, Gilles Adda, Martine Adda-Decker, Alexandre Allauzen, Laurent Besacier, Hélène Bonneau-Maynard, Guy-Noël Kouarata, Kevin Löser, Annie Rialland, and François Yvon. 2016. Preliminary Experiments on Unsupervised Word Discovery in Mboshi. In *Proceedings of Interspeech 2016*, pages 3539–3543.

Pierre Godard, Laurent Besacier, François Yvon, Martine Adda-Decker, Gilles Adda, Hélène Maynard, and Annie Rialland. 2018. Adaptor Grammars for the linguist: Word segmentation experiments for very

low-resource languages. In *Proceedings of the Fifteenth Workshop on Computational Research in Phonetics, Phonology, and Morphology*, pages 32–42, Brussels, Belgium. Association for Computational Linguistics.

Dirk Goldhahn, Thomas Eckart, and Uwe Quasthoff. 2012. Building large monolingual dictionaries at the Leipzig corpora collection: From 100 to 200 languages. In *Proceedings of the Eighth International Conference on Language Resources and Evaluation (LREC’12)*, pages 759–765, Istanbul, Turkey. European Language Resources Association (ELRA).

Sharon Goldwater, Thomas L. Griffiths, and Mark Johnson. 2006. Contextual dependencies in unsupervised word segmentation. In *Proceedings of the 21st International Conference on Computational Linguistics and 44th Annual Meeting of the Association for Computational Linguistics*, pages 673–680, Sydney, Australia. Association for Computational Linguistics.

Sharon Goldwater, Thomas L. Griffiths, and Mark Johnson. 2009. A Bayesian framework for word segmentation: Exploring the effects of context. *Cognition*, 112(1):21–54.

Sharon Goldwater, Thomas L. Griffiths, and Mark Johnson. 2011. Producing power-law distributions and damping word frequencies with two-stage language models. *Journal of Machine Learning Research*, 12:2335–2382.

Guillaume Jacques. 2021. *A grammar of Japhug*. Number 1 in Comprehensive Grammar Library. Language Science Press, Berlin.

Mark Johnson. 2008. Unsupervised word segmentation for Sesotho using adaptor grammars. In *Proceedings of the Tenth Meeting of ACL Special Interest Group on Computational Morphology and Phonology*, pages 20–27, Columbus, Ohio. Association for Computational Linguistics.

Mark Johnson, Thomas L. Griffiths, and Sharon Goldwater. 2007. Adaptor Grammars: a Framework for Specifying Compositional Nonparametric Bayesian Models. In *Advances in Neural Information Processing Systems 19*, pages 641–648, Cambridge, MA. MIT Press.

Katharina Kann, Jesus Manuel Mager Hois, Ivan Vladimir Meza-Ruiz, and Hinrich Schütze. 2018. Fortification of neural morphological segmentation models for polysynthetic minimal-resource languages. In *Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)*, pages 47–57, New Orleans, Louisiana. Association for Computational Linguistics.

Kimmo Koskenniemi. 1983. *Two-level morphology: A general computational model for word-form recognition and production*, volume 11. University
of Helsinki, Department of General Linguistics
Helsinki, Finland.

Taku Kudo and John Richardson. 2018. SentencePiece: A simple and language independent subword tokenizer and detokenizer for neural text processing. In *Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing: System Demonstrations*, pages 66–71, Brussels, Belgium. Association for Computational Linguistics.

John D. Lafferty, Andrew McCallum, and Fernando C. N. Pereira. 2001. Conditional random fields: Probabilistic models for segmenting and labeling sequence data. In *Proceedings of the Eighteenth International Conference on Machine Learning*, ICML '01, page 282–289, San Francisco, CA, USA. Morgan Kaufmann Publishers Inc.

Thomas Lavergne, Olivier Cappé, and François Yvon. 2010. Practical very large scale CRFs. In *Proceedings the 48th Annual Meeting of the Association for Computational Linguistics (ACL)*, pages 504–513. Association for Computational Linguistics.

Zoey Liu, Robert Jimerson, and Emily Prud’hommeaux. 2021. Morphological segmentation for Seneca. In *Proceedings of the First Workshop on Natural Language Processing for Indigenous Languages of the Americas*, pages 90–101, Online. Association for Computational Linguistics.

Daichi Mochihashi, Takeshi Yamada, and Naonori Ueda. 2009. Bayesian unsupervised word segmentation with nested Pitman-Yor language modeling. In *Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP*, pages 100–108, Suntec, Singapore. Association for Computational Linguistics.

Sarah Moeller and Mans Hulden. 2018. Automatic glossing in a low-resource setting for language documentation. In *Proceedings of the Workshop on Computational Modeling of Polysynthetic Languages*, pages 84–93, Santa Fe, New Mexico, USA. Association for Computational Linguistics.

Tumi Moeng, Sheldon Reay, Aaron Daniels, and Jan Buys. 2021. Canonical and surface morphological segmentation for Nguni languages. *CoRR*, abs/2104.00767.

Shu Okabe, Laurent Besacier, and François Yvon. 2022. Weakly supervised word segmentation for computational language documentation. In *Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)*, pages 7385–7398, Dublin, Ireland. Association for Computational Linguistics.

Teemu Ruokolainen, Oskar Kohonen, Kairit Sirts, Stig-Arne Grönroos, Mikko Kurimo, and Sami Virpioja. 2016. A comparative study of minimally supervised morphological segmentation. *Computational Linguistics*, 42(1):91–120.

Kairit Sirts and Sharon Goldwater. 2013. Minimally-supervised morphological segmentation using Adaptor Grammars. *Transactions of the Association for Computational Linguistics*, 1:255–266.

Peter Smit, Sami Virpioja, Stig-Arne Grönroos, and Mikko Kurimo. 2014. Morfessor 2.0: Toolkit for statistical morphological segmentation. In *Proceedings of the Demonstrations at the 14th Conference of the European Chapter of the Association for Computational Linguistics*, pages 21–24, Gothenburg, Sweden. Association for Computational Linguistics.

Yee Whye Teh. 2006. A Bayesian interpretation of interpolated Kneser-Ney. Technical Report TRA2/06, School of Computing, National University of Singapore.

Anand Venkataraman. 2001. A statistical model for word discovery in transcribed speech. *Computational Linguistics*, 27(3):352–372.

Sami Virpioja, Oskar Kohonen, and Krista Lagus. 2011. Evaluating the effect of word frequencies in a probabilistic generative model of morphology. In *Proceedings of the 18th Nordic Conference of Computational Linguistics (NODALIDA 2011)*, pages 230–237, Riga, Latvia. Northern European Association for Language Technology (NEALT).

Xingyuan Zhao, Satoru Ozaki, Antonios Anastasopoulos, Graham Neubig, and Lori Levin. 2020. Automatic interlinear glossing for under-resourced languages leveraging translations. In *Proceedings of the 28th International Conference on Computational Linguistics*, pages 5397–5408, Barcelona, Spain (Online). International Committee on Computational Linguistics.

## A Word and morpheme distributions

According to Zipf's law, for a unit of rank *R*, its normalised frequency *f* ($f = \frac{F}{N}$) with *F* the frequency of the unit and *N* the total number of units in the corpus) is computed as follows in Equation (5):

$$f = \frac{c}{R^a}, \qquad (5)$$

with *c* a normalising constant and *a* the parameter of the distribution (Baayen, 2001). Hence, the relationship between the log-(normalised) frequency and the log-rank is:

$$\log(f) = -a \log(R) + \log(c) \qquad (6)$$

To visualise the linear relationship shown in Equation (6), we hence fit a (least square) linear regression. Thus, Figure 4 plots the value of the slope $-a$ for words (x-axis) and morphemes (y-axis).
<figure><img src="image_6.png" /><figcaption>Figure 5: Log-normalised frequency of words according to their log-rank for several languages.</figcaption></figure>

In Figure 5, we compare the Japhug text with three languages of varying morphological complexity (as in Section 4.4.1).

The Japhug curve lies between the English and Finnish ones, two languages with a clear contrast in morphological complexity. If for the most frequent words (i.e. low Zipf rank, on the left), the Japhug words follow the English or German trend, for rare words (i.e. high Zipf rank, on the right), it joins the Finnish trend.

## B Output analysis

<table><caption>Figure 6: An example Japhug sentence segmented by various models, with and without supervision.</caption><thead><tr><th></th><th>supervision</th><th>sentence</th></tr></thead><tbody><tr><td>dpseg</td><td>/</td><td>a mbroujme zui kxzo</td></tr><tr><td>parallel-w</td><td>/</td><td>a mbro-ujme z ukx zo</td></tr><tr><td>parallel-w</td><td>sentence</td><td>a-mbro ujme zui kx-zo</td></tr><tr><td>reference</td><td></td><td>a-mbro ujme zui kx-zo<br>Land on my horse's tail</td></tr></tbody></table>

The example in Figure 6 displays a Japhug sentence segmented by two models. First, without supervision, dpseg fuses two units that should be separated by a word boundary ('mbroujme'), and so does parallel-w with 'ukx'. Apart from diminishing the three F-scores, this kind of error creates meaningless units. Besides, some reference morpheme boundaries are not identified: no boundary at all for 'ujme' and a word boundary in 'a-mbro'.

Once supervised, the parallel-w model corrects its initial error ('ukx' is segmented) and finds morpheme boundaries. Indeed, the model seems to benefit from the supervision data, which contained

the words 'a-mbro' and 'kx-zo'. The remaining error ('ujme') can be explained by the fact that in the corpus, all occurrences of the morpheme 'jme' are always preceded by 'uj-'. The model thus does not identify nor recreate 'jme' as a unit but keeps 'ujme'. The negative effect of collocations constitutes an inherent limit of the unigram dpseg model, already discussed by Goldwater et al. (2006).

## C Reproducibility

All presented experiments have been obtained with the same three random seeds (42, 142, and 1234) for a fair comparison. Details about the hyperparameters are in Section 3.3.

The Adaptor Grammar was run with the hyper-parameter values indicated for MorphAGram<fnref n="11" /> (Es-kander et al., 2020).

For reference, a processor of 6 cores and 12 threads takes around two days for a hierarchical model on the Tsez 2K corpus (20,000 iterations of Gibbs sampling). With the same setting, a parallel model takes approximately one day.

## D Complete results

This section displays the full results for all our experimental settings: each model will be unsupervised or supervised with the sentence or dictionary supervision and will segment the Japhug and Tsez corpora. The tables also report the precision and recall for each evaluation tier (BP and BR for Boundary Precision and Boundary Recall; WP, WR, and LP, LR, respectively for token and type evaluation). Bold values are the best score in a given experimental situation.

### D.1 Japhug

Tables 3, 4, and 5 display the full results for the Japhug text.

### D.2 Tsez

Similarly, Tables 6, 7, and 8 display the full results for the Tsez text.

### D.3 Fully supervised model

For the sentence supervision method of Section 2.7, we also report the results of a CRF (Conditional Random Field, Lafferty et al. 2001), mainly inspired by the methodology of Moeller and Hulden (2018). Each training sentence is labelled as in Figure 7.

<footnote n="11">¹¹https://github.com/rnd2110/MorphAGram.</footnote>
<table><caption>Figure 7: Example of Japhug sentence labelled for CRF</caption><tr><td>Original sentence</td><td colspan="5">χpum w–pw</td></tr><tr><td>Translation (EN)</td><td colspan="5">little monk</td></tr><tr><td>χ</td><td>p</td><td>w</td><td>n</td><td>w</td><td>p</td><td>w</td></tr><tr><td>B-w</td><td>I</td><td>I</td><td>I</td><td>B-w</td><td>B-m</td><td>I</td></tr></table>

The ‘B-w’ label indicates the beginning of a word, while ‘B-m’ marks the start of a morpheme inside a word. The ‘I’ label is used for all other characters (inside a morpheme). We use Wapiti<fnref n="12" /> (Lavergne et al., 2010) for the CRF implementation. Our feature set only includes basic unigram and bigram features.

The results in Table 4 and Table 7 show that, on average, full supervision yields better segmentation scores than weakly supervised models at the word level; contrarily, we observe worse scores at the morpheme level for both languages.

We also note that the CRF model identifies more than 4,000 morpheme types in both languages (i.e. much more than what exist in the reference or our models), which results in less than 36 in F-score on morpheme types (LF). This suggests that morphemes are difficult to distinguish from words, even in this favourable setting, confirming one of our main conclusions: statistical cues alone do not seem to be enough to correctly separate these two types of units.

<footnote n="12">¹²https://github.com/Jekub/Wapiti.</footnote>
<table><caption>Table 3: Results on the Japhug corpus for unsupervised dpseg and its two-level versions. **Bold** numbers denote the best results per metrics. Reference $N_{type}$: 6,739 for words and 2,731 for morphemes.</caption><thead><tr><th rowspan="2">model level</th><th colspan="2">AG</th><th colspan="2">dpseg</th><th colspan="2">pipeline</th><th colspan="2">parallel-w</th><th colspan="2">parallel-m</th><th colspan="2">hier-type</th><th colspan="2">-final</th><th colspan="2">hier-iter</th></tr><tr><th>word</th><th>morph.</th><th>word</th><th>morph.</th><th>word</th><th>morph.</th><th>word</th><th>morph.</th><th>word</th><th>morph.</th><th>word</th><th>morph.</th><th>word</th><th>morph.</th><th>word</th><th>morph.</th></tr></thead><tbody><tr><td>BP</td><td><strong>70.9</strong></td><td>77.3</td><td>61.3</td><td><strong>87.8</strong></td><td>61.3</td><td>69.3</td><td>61.5</td><td>84.9</td><td>64.5</td><td>87.6</td><td>67.6</td><td>48.4</td><td>73.6</td><td>69.6</td><td>73.5</td></tr><tr><td>BR</td><td>71.1</td><td>90.4</td><td>90.6</td><td>75.2</td><td>90.6</td><td><strong>96.3</strong></td><td><strong>90.8</strong></td><td>82.4</td><td>84.5</td><td>75.0</td><td>83.4</td><td>88.2</td><td>93.4</td><td>77.8</td><td>91.2</td></tr><tr><td>BF</td><td>71.0</td><td>83.4</td><td>73.1</td><td>81.0</td><td>73.1</td><td>80.6</td><td>73.3</td><td><strong>83.6</strong></td><td>73.2</td><td>80.8</td><td><strong>74.7</strong></td><td>62.5</td><td>82.3</td><td>73.5</td><td>81.4</td></tr><tr><td>WP</td><td><strong>45.8</strong></td><td>58.3</td><td>39.4</td><td>59.3</td><td>39.4</td><td>50.1</td><td>39.7</td><td><strong>62.1</strong></td><td>41.1</td><td>58.9</td><td>44.6</td><td>19.2</td><td>54.9</td><td>44.9</td><td>53.7</td></tr><tr><td>WR</td><td>45.9</td><td>67.3</td><td>55.9</td><td>51.5</td><td>55.9</td><td>68.2</td><td><strong>56.2</strong></td><td>60.4</td><td>52.3</td><td>51.1</td><td>53.7</td><td>33.8</td><td><strong>68.5</strong></td><td>49.6</td><td>65.7</td></tr><tr><td>WF</td><td>45.8</td><td><strong>62.5</strong></td><td>46.2</td><td>55.1</td><td>46.2</td><td>57.8</td><td>46.5</td><td>61.3</td><td>46.0</td><td>54.7</td><td><strong>48.7</strong></td><td>24.5</td><td>60.9</td><td>47.2</td><td>59.1</td></tr><tr><td>LP</td><td>34.3</td><td>49.9</td><td>40.6</td><td>45.7</td><td>40.6</td><td>43.3</td><td><strong>41.1</strong></td><td><strong>50.5</strong></td><td>39.4</td><td>45.4</td><td>40.0</td><td>31.5</td><td>46.7</td><td>37.5</td><td>47.6</td></tr><tr><td>LR</td><td><strong>28.4</strong></td><td>20.3</td><td>13.6</td><td>37.8</td><td>13.6</td><td>11.0</td><td>13.8</td><td>33.9</td><td>17.1</td><td><strong>37.9</strong></td><td>22.6</td><td>11.7</td><td>15.6</td><td>27.4</td><td>16.7</td></tr><tr><td>LF</td><td>31.1</td><td>28.9</td><td>20.4</td><td><strong>41.4</strong></td><td>20.4</td><td>17.5</td><td>20.6</td><td>40.5</td><td>23.8</td><td>41.4</td><td>28.8</td><td>17.0</td><td>23.4</td><td><strong>31.7</strong></td><td>24.7</td></tr><tr><td>WL</td><td>4.72</td><td>2.51</td><td>3.34</td><td>3.34</td><td>2.13</td><td>3.34</td><td>2.98</td><td>3.73</td><td>3.35</td><td>3.93</td><td>1.65</td><td>2.32</td><td>4.29</td><td>2.37</td><td></td></tr><tr><td>TL</td><td>6.60</td><td>3.27</td><td>4.22</td><td>4.22</td><td>2.64</td><td>4.23</td><td>3.99</td><td>4.77</td><td>4.21</td><td>4.78</td><td>2.83</td><td>2.87</td><td>5.12</td><td>2.88</td><td></td></tr><tr><td>N<sub>type</sub></td><td>5582</td><td>1113</td><td>2260</td><td>2260</td><td>694</td><td>2257</td><td>1834</td><td>2921</td><td>2281</td><td>3806</td><td>1013</td><td>911</td><td>4925</td><td>956</td><td></td></tr><tr><td>N<sub>token</sub></td><td>28.6k</td><td>53.9k</td><td>40.5k</td><td>40.5k</td><td>63.4k</td><td>40.5k</td><td>45.4k</td><td>36.3k</td><td>40.4k</td><td>34.4k</td><td>82.1k</td><td>58.2k</td><td>31.5k</td><td>57.0k</td><td></td></tr></tbody></table>

<table><caption>Table 4: Results on the Japhug corpus for dpseg and its two-level versions, supervised with dense annotations (sentence). 200 sentences are used as supervision data.</caption><thead><tr><th rowspan="2">model level</th><th colspan="2">CRF</th><th colspan="2">dpseg</th><th colspan="2">pipe.</th><th colspan="2">parallel-w</th><th colspan="2">parallel-m</th><th colspan="2">hier-type</th><th colspan="2">-final</th><th colspan="2">hier-iter</th></tr><tr><th>word</th><th>morph.</th><th>word</th><th>morph.</th><th>morph.</th><th>word</th><th>morph.</th><th>word</th><th>morph.</th><th>word</th><th>morph.</th><th>word</th><th>morph.</th><th>word</th><th>morph.</th><th>word</th><th>morph.</th></tr></thead><tbody><tr><td>BP</td><td><strong>73.5</strong></td><td>83.2</td><td>63.8</td><td>88.1</td><td>79.2</td><td>64.0</td><td>86.4</td><td>66.4</td><td><strong>88.9</strong></td><td>70.9</td><td>63.7</td><td>80.9</td><td>72.4</td><td>80.1</td><td></td><td></td></tr><tr><td>BR</td><td>80.8</td><td>85.2</td><td>91.4</td><td>79.6</td><td><strong>97.4</strong></td><td><strong>91.4</strong></td><td>83.7</td><td>86.3</td><td>77.7</td><td>84.0</td><td>92.2</td><td>96.0</td><td>80.2</td><td>94.5</td><td></td><td></td></tr><tr><td>BF</td><td><strong>77.0</strong></td><td>84.2</td><td>75.1</td><td>83.6</td><td>87.3</td><td>75.3</td><td>85.1</td><td>75.0</td><td>82.9</td><td>76.9</td><td>75.4</td><td><strong>87.8</strong></td><td>76.1</td><td>86.7</td><td></td><td></td></tr><tr><td>WP</td><td><strong>52.6</strong></td><td>66.4</td><td>43.7</td><td>64.3</td><td>67.2</td><td>43.9</td><td>65.6</td><td>44.8</td><td>63.6</td><td>49.6</td><td>43.8</td><td><strong>68.5</strong></td><td>49.8</td><td>66.9</td><td></td><td></td></tr><tr><td>WR</td><td>57.3</td><td>67.8</td><td>60.2</td><td>58.7</td><td><strong>81.5</strong></td><td><strong>60.3</strong></td><td>63.7</td><td>56.5</td><td>56.3</td><td>57.5</td><td>61.9</td><td>80.3</td><td>54.5</td><td>78.0</td><td></td><td></td></tr><tr><td>WF</td><td><strong>54.9</strong></td><td>67.1</td><td>50.6</td><td>61.4</td><td>73.6</td><td>50.8</td><td>64.7</td><td>50.0</td><td>59.7</td><td>53.3</td><td>51.3</td><td><strong>74.0</strong></td><td>52.1</td><td>72.0</td><td></td><td></td></tr><tr><td>LP</td><td>39.4</td><td>27.5</td><td>50.7</td><td>53.9</td><td><strong>61.6</strong></td><td><strong>51.2</strong></td><td>55.3</td><td>47.2</td><td>51.1</td><td>46.3</td><td>49.4</td><td>59.6</td><td>43.2</td><td>60.8</td><td></td><td></td></tr><tr><td>LR</td><td><strong>49.5</strong></td><td><strong>50.3</strong></td><td>19.6</td><td>40.2</td><td>23.5</td><td>19.9</td><td>39.4</td><td>22.3</td><td>42.7</td><td>30.0</td><td>23.2</td><td>25.9</td><td>33.4</td><td>26.4</td><td></td><td></td></tr><tr><td>LF</td><td><strong>43.9</strong></td><td>35.5</td><td>28.3</td><td>45.8</td><td>34.0</td><td>28.7</td><td>46.0</td><td>30.3</td><td><strong>46.5</strong></td><td>36.4</td><td>31.6</td><td>36.1</td><td>37.6</td><td>36.8</td><td></td><td></td></tr><tr><td>WL</td><td>4.35</td><td>2.84</td><td>3.44</td><td>3.19</td><td>2.39</td><td>3.45</td><td>2.99</td><td>3.75</td><td>3.28</td><td>4.08</td><td>2.05</td><td>2.48</td><td>4.32</td><td>2.49</td><td></td><td></td></tr><tr><td>TL</td><td>6.67</td><td>5.09</td><td>4.66</td><td>4.25</td><td>3.44</td><td>4.66</td><td>4.12</td><td>5.04</td><td>4.30</td><td>5.13</td><td>3.36</td><td>3.47</td><td>5.33</td><td>3.46</td><td></td><td></td></tr><tr><td>N<sub>type</sub></td><td>8453</td><td>4999</td><td>2610</td><td>2061</td><td>1040</td><td>2627</td><td>1946</td><td>3182</td><td>2283</td><td>4363</td><td>1285</td><td>1186</td><td>5208</td><td>1185</td><td></td><td></td></tr><tr><td>N<sub>token</sub></td><td>31.1k</td><td>47.6k</td><td>39.4k</td><td>42.5k</td><td>56.5k</td><td>39.2k</td><td>45.3k</td><td>36.0k</td><td>41.2k</td><td>33.2k</td><td>65.9k</td><td>54.6k</td><td>31.3k</td><td>54.4k</td><td></td><td></td></tr></tbody></table>

<table><caption>Table 5: Results on the Japhug corpus for dpseg and its two-level versions, supervised with a dictionary (dictionary). 200 sentences are used as supervision data.</caption><thead><tr><th rowspan="2">model level</th><th colspan="2">dpseg</th><th colspan="2">pipe.</th><th colspan="2">parallel-w</th><th colspan="2">parallel-m</th><th colspan="2">hier-type</th><th colspan="2">-final</th><th colspan="2">hier-iter</th></tr><tr><th>word</th><th>morph.</th><th>morph.</th><th>word</th><th>morph.</th><th>word</th><th>morph.</th><th>word</th><th>morph.</th><th>word</th><th>morph.</th><th>word</th><th>morph.</th><th>word</th><th>morph.</th></tr></thead><tbody><tr><td>BP</td><td><strong>76.6</strong></td><td><strong>93.2</strong></td><td>87.0</td><td>76.6</td><td>91.0</td><td>76.4</td><td>93.0</td><td>66.4</td><td>58.4</td><td>83.6</td><td>66.6</td><td>84.3</td><td></td><td></td></tr><tr><td>BR</td><td>81.0</td><td>64.3</td><td>83.1</td><td>81.2</td><td>71.3</td><td>74.9</td><td>64.2</td><td>89.6</td><td>89.9</td><td>90.2</td><td><strong>90.1</strong></td><td><strong>90.8</strong></td><td></td><td></td></tr><tr><td>BF</td><td>78.7</td><td>76.1</td><td>85.0</td><td><strong>78.8</strong></td><td>80.0</td><td>75.6</td><td>76.0</td><td>76.2</td><td>70.8</td><td>86.8</td><td>76.6</td><td><strong>87.4</strong></td><td></td><td></td></tr><tr><td>WP</td><td>54.4</td><td>54.8</td><td>65.9</td><td><strong>54.5</strong></td><td>60.1</td><td>51.6</td><td>54.4</td><td>45.6</td><td>30.4</td><td>67.2</td><td>46.0</td><td><strong>68.7</strong></td><td></td><td></td></tr><tr><td>WR</td><td>57.1</td><td>39.2</td><td>63.2</td><td>57.4</td><td>48.1</td><td>50.7</td><td>38.9</td><td>59.6</td><td>45.6</td><td>72.1</td><td><strong>60.1</strong></td><td><strong>73.6</strong></td><td></td><td></td></tr><tr><td>WF</td><td>55.7</td><td>45.7</td><td>64.5</td><td><strong>55.9</strong></td><td>53.5</td><td>51.1</td><td>45.4</td><td>51.7</td><td>36.5</td><td>69.6</td><td>52.1</td><td><strong>71.1</strong></td><td></td><td></td></tr><tr><td>LP</td><td>49.9</td><td>37.0</td><td>47.0</td><td><strong>50.5</strong></td><td>40.9</td><td>46.4</td><td>37.2</td><td>46.4</td><td>51.1</td><td>56.0</td><td>47.3</td><td><strong>57.9</strong></td><td></td><td></td></tr><tr><td>LR</td><td>37.3</td><td>54.8</td><td>43.5</td><td><strong>37.8</strong></td><td>52.3</td><td>36.8</td><td><strong>54.8</strong></td><td>21.6</td><td>30.2</td><td>34.3</td><td>21.9</td><td>34.9</td><td></td><td></td></tr><tr><td>LF</td><td>42.7</td><td>44.2</td><td>45.2</td><td><strong>43.2</strong></td><td><strong>45.9</strong></td><td>41.1</td><td>44.3</td><td>29.5</td><td>38.0</td><td>42.5</td><td>29.9</td><td>43.6</td><td></td><td></td></tr><tr><td>WL</td><td>4.51</td><td>4.06</td><td>3.03</td><td>4.49</td><td>3.62</td><td>4.81</td><td>4.06</td><td>3.63</td><td>1.94</td><td>2.71</td><td>3.62</td><td>2.71</td><td></td><td></td></tr><tr><td>TL</td><td>6.18</td><td>5.40</td><td>4.45</td><td>6.16</td><td>5.14</td><td>6.49</td><td>5.38</td><td>4.46</td><td>3.77</td><td>3.84</td><td>4.52</td><td>3.86</td><td></td><td></td></tr><tr><td>N<sub>type</sub></td><td>5041</td><td>4044</td><td>2524</td><td>5040</td><td>3492</td><td>5356</td><td>4027</td><td>3141</td><td>1618</td><td>1671</td><td>3116</td><td>1646</td><td></td><td></td></tr><tr><td>N<sub>token</sub></td><td>30.0k</td><td>33.3k</td><td>44.7k</td><td>30.1k</td><td>37.3k</td><td>28.1k</td><td>33.3k</td><td>37.3k</td><td>69.9k</td><td>50.0k</td><td>37.4k</td><td>49.9k</td><td></td><td></td></tr></tbody></table>
<table><caption>Table 6: Results on the Tsez corpus for unsupervised dpseg and its two-level versions. **Bold** numbers denote the best results per metrics. Reference $N_{type}$: 5,732 for words and 1,603 for morphemes.</caption><thead><tr><th rowspan="2">model level</th><th colspan="2">AG</th><th colspan="2">dpseg</th><th colspan="2">pipeline</th><th colspan="2">parallel-w</th><th colspan="2">hier-type</th><th colspan="2">-final</th><th colspan="2">hier-iter</th></tr><tr><th>word</th><th>morph.</th><th>word</th><th>morph.</th><th>word</th><th>morph.</th><th>word</th><th>morph.</th><th>word</th><th>morph.</th><th>morph.</th><th>word</th><th>morph.</th></tr></thead><tbody><tr><td>BP</td><td><b>67.3</b></td><td>78.1</td><td>59.9</td><td><b>91.8</b></td><td>59.9</td><td>69.9</td><td>59.6</td><td>89.3</td><td>64.0</td><td>47.8</td><td>74.4</td><td>64.7</td><td>74.7</td></tr><tr><td>BR</td><td>76.6</td><td>85.5</td><td><b>87.9</b></td><td>63.9</td><td><b>87.9</b></td><td><b>88.8</b></td><td>87.4</td><td>71.6</td><td>83.0</td><td>81.7</td><td>86.1</td><td>77.6</td><td>85.1</td></tr><tr><td>BF</td><td>71.6</td><td><b>81.6</b></td><td>71.3</td><td>75.3</td><td>71.3</td><td>78.2</td><td>70.9</td><td>79.5</td><td><b>72.2</b></td><td>60.3</td><td>79.8</td><td>70.6</td><td>79.6</td></tr><tr><td>WP</td><td><b>41.6</b></td><td>55.6</td><td>33.3</td><td>52.1</td><td>33.3</td><td>46.0</td><td>32.8</td><td><b>57.7</b></td><td>38.2</td><td>19.0</td><td>50.8</td><td>38.8</td><td>51.4</td></tr><tr><td>WR</td><td>46.7</td><td><b>60.5</b></td><td>47.4</td><td>37.1</td><td>47.4</td><td>57.8</td><td>46.6</td><td>46.8</td><td><b>48.4</b></td><td>31.9</td><td>58.3</td><td>45.7</td><td>58.2</td></tr><tr><td>WF</td><td><b>44.0</b></td><td><b>57.9</b></td><td>39.1</td><td>43.4</td><td>39.1</td><td>51.2</td><td>38.5</td><td>51.7</td><td>42.7</td><td>23.8</td><td>54.3</td><td>42.0</td><td>54.6</td></tr><tr><td>LP</td><td>45.9</td><td><b>51.3</b></td><td><b>49.6</b></td><td>41.4</td><td><b>49.6</b></td><td>41.2</td><td>49.0</td><td>47.7</td><td>47.3</td><td>24.2</td><td>41.1</td><td>42.3</td><td>43.1</td></tr><tr><td>LR</td><td><b>28.8</b></td><td>28.0</td><td>16.9</td><td><b>50.4</b></td><td>16.9</td><td>16.6</td><td>16.7</td><td>47.6</td><td>22.6</td><td>13.1</td><td>20.1</td><td>25.5</td><td>21.8</td></tr><tr><td>LF</td><td><b>35.4</b></td><td>36.2</td><td>25.2</td><td>45.5</td><td>25.2</td><td>23.6</td><td>25.0</td><td><b>47.7</b></td><td>30.5</td><td>17.0</td><td>27.0</td><td>31.8</td><td>29.0</td></tr><tr><td>WL</td><td>4.99</td><td>2.58</td><td>3.95</td><td></td><td>3.95</td><td>2.24</td><td>3.95</td><td>3.46</td><td>4.43</td><td>1.68</td><td>2.45</td><td>4.76</td><td>2.48</td></tr><tr><td>TL</td><td>6.52</td><td>3.52</td><td>4.53</td><td></td><td>4.53</td><td>2.89</td><td>4.52</td><td>4.32</td><td>4.95</td><td>2.87</td><td>3.07</td><td>5.35</td><td>3.08</td></tr><tr><td>N<sub>type</sub></td><td>3597</td><td>875</td><td>1950</td><td></td><td>1950</td><td>646</td><td>1958</td><td>1600</td><td>2732</td><td>867</td><td>786</td><td>3456</td><td>812</td></tr><tr><td>N<sub>token</sub></td><td>22.7k</td><td>43.8k</td><td>28.6k</td><td></td><td>28.6k</td><td>50.5k</td><td>28.6k</td><td>32.7k</td><td>25.6k</td><td>67.4k</td><td>46.2k</td><td>23.8k</td><td>45.5k</td></tr></tbody></table>

<table><caption>Table 7: Results on the Tsez corpus for dpseg and its two-level versions, supervised with dense annotations (sentence). 200 sentences are used as supervision data.</caption><thead><tr><th rowspan="2">model level</th><th colspan="2">CRF</th><th colspan="2">dpseg</th><th colspan="2">pipe.</th><th colspan="2">parallel-w</th><th colspan="2">hier-type</th><th colspan="2">-final</th><th colspan="2">hier-iter</th></tr><tr><th>word</th><th>morph.</th><th>word</th><th>morph.</th><th>morph.</th><th>word</th><th>morph.</th><th>word</th><th>morph.</th><th>word</th><th>morph.</th><th>morph.</th><th>word</th><th>morph.</th></tr></thead><tbody><tr><td>BP</td><td><b>83.3</b></td><td>85.9</td><td>65.4</td><td><b>93.3</b></td><td>83.2</td><td>65.3</td><td>90.6</td><td>69.1</td><td>65.6</td><td>85.0</td><td>69.5</td><td>84.0</td></tr><tr><td>BR</td><td>78.3</td><td>82.5</td><td><b>90.7</b></td><td>69.3</td><td><b>95.9</b></td><td>90.6</td><td>74.7</td><td>83.6</td><td>88.7</td><td>92.9</td><td>80.7</td><td>91.7</td></tr><tr><td>BF</td><td><b>80.7</b></td><td>84.2</td><td>76.0</td><td>79.5</td><td><b>89.1</b></td><td>75.9</td><td>81.9</td><td>75.7</td><td>75.4</td><td>88.8</td><td>74.7</td><td>87.7</td></tr><tr><td>WP</td><td><b>64.5</b></td><td>67.8</td><td>42.5</td><td>61.8</td><td>71.9</td><td>42.2</td><td>63.2</td><td>46.6</td><td>46.4</td><td><b>72.5</b></td><td>46.9</td><td>70.6</td></tr><tr><td>WR</td><td><b>60.9</b></td><td>65.3</td><td>57.3</td><td>46.7</td><td><b>82.4</b></td><td>56.9</td><td>52.6</td><td>55.4</td><td>62.0</td><td>79.0</td><td>53.8</td><td>76.8</td></tr><tr><td>WF</td><td><b>62.6</b></td><td>66.6</td><td>48.8</td><td>53.2</td><td><b>76.8</b></td><td>48.4</td><td>57.4</td><td>50.6</td><td>53.1</td><td>75.6</td><td>50.1</td><td>73.6</td></tr><tr><td>LP</td><td>47.6</td><td>21.9</td><td><b>62.7</b></td><td>49.1</td><td><b>61.9</b></td><td>62.4</td><td>53.4</td><td>53.8</td><td>46.5</td><td>59.8</td><td>50.6</td><td>59.3</td></tr><tr><td>LR</td><td><b>61.0</b></td><td><b>62.0</b></td><td>26.9</td><td>57.5</td><td>36.7</td><td>26.7</td><td>54.6</td><td>32.7</td><td>33.8</td><td>38.9</td><td>34.4</td><td>38.5</td></tr><tr><td>LF</td><td><b>53.5</b></td><td>32.4</td><td>37.6</td><td>53.0</td><td>46.1</td><td>37.3</td><td><b>54.0</b></td><td>40.6</td><td>39.1</td><td>47.1</td><td>41.0</td><td>46.7</td></tr><tr><td>WL</td><td>5.94</td><td>2.92</td><td>4.16</td><td>3.72</td><td>2.46</td><td>4.16</td><td>3.38</td><td>4.72</td><td>2.11</td><td>2.58</td><td>4.90</td><td>2.59</td></tr><tr><td>TL</td><td>7.83</td><td>5.98</td><td>5.02</td><td>4.58</td><td>3.67</td><td>5.03</td><td>4.40</td><td>5.43</td><td>3.49</td><td>3.70</td><td>5.61</td><td>3.67</td></tr><tr><td>N<sub>type</sub></td><td>7343</td><td>4537</td><td>2458</td><td>1877</td><td>950</td><td>2450</td><td>1639</td><td>3479</td><td>1165</td><td>1043</td><td>3902</td><td>1041</td></tr><tr><td>N<sub>token</sub></td><td>19.0k</td><td>38.7k</td><td>27.2k</td><td>30.4k</td><td>46.1k</td><td>27.2k</td><td>33.5k</td><td>24.0k</td><td>53.7k</td><td>43.8k</td><td>23.1k</td><td>43.7k</td></tr></tbody></table>

<table><caption>Table 8: Results on the Tsez corpus for dpseg and its two-level versions, supervised with a dictionary (dictionary). 200 sentences are used as supervision data. Reference $N_{type}$: 5,732 for words and 1,603 for morphemes.</caption><thead><tr><th rowspan="2">model level</th><th colspan="2">dpseg</th><th colspan="2">pipe.</th><th colspan="2">parallel-w</th><th colspan="2">hier-type</th><th colspan="2">-final</th><th colspan="2">hier-iter</th></tr><tr><th>word</th><th>morph.</th><th>morph.</th><th>word</th><th>morph.</th><th>word</th><th>morph.</th><th>word</th><th>morph.</th><th>morph.</th><th>word</th><th>morph.</th></tr></thead><tbody><tr><td>BP</td><td>73.2</td><td><b>95.8</b></td><td>90.6</td><td><b>73.4</b></td><td>94.3</td><td>66.0</td><td>58.0</td><td>87.1</td><td>66.6</td><td>87.7</td></tr><tr><td>BR</td><td>84.9</td><td>58.5</td><td>79.6</td><td>84.9</td><td>63.1</td><td><b>91.2</b></td><td>82.0</td><td>84.5</td><td>90.5</td><td><b>85.4</b></td></tr><tr><td>BF</td><td>78.6</td><td>72.6</td><td>84.7</td><td><b>78.7</b></td><td>75.6</td><td>76.6</td><td>67.9</td><td>85.8</td><td>76.7</td><td><b>86.5</b></td></tr><tr><td>WP</td><td>50.3</td><td>49.7</td><td>66.1</td><td><b>50.5</b></td><td>53.1</td><td>43.0</td><td>29.1</td><td>65.8</td><td>43.6</td><td><b>67.7</b></td></tr><tr><td>WR</td><td>57.6</td><td>31.3</td><td>58.4</td><td>57.7</td><td>36.4</td><td><b>57.7</b></td><td>40.5</td><td>64.0</td><td>57.7</td><td><b>66.1</b></td></tr><tr><td>WF</td><td>53.7</td><td>38.4</td><td>62.0</td><td><b>53.9</b></td><td>43.2</td><td>49.3</td><td>33.8</td><td>64.9</td><td>49.7</td><td><b>66.9</b></td></tr><tr><td>LP</td><td>62.0</td><td>38.0</td><td>49.8</td><td><b>62.1</b></td><td>41.5</td><td>59.9</td><td>43.2</td><td>53.7</td><td>60.4</td><td><b>55.2</b></td></tr><tr><td>LR</td><td>37.2</td><td><b>64.6</b></td><td>54.1</td><td><b>37.3</b></td><td>63.1</td><td>26.9</td><td>36.2</td><td>44.9</td><td>27.7</td><td>45.9</td></tr><tr><td>LF</td><td>46.5</td><td>47.9</td><td><b>51.9</b></td><td><b>46.6</b></td><td>50.0</td><td>37.1</td><td>39.4</td><td>48.9</td><td>37.9</td><td>50.1</td></tr><tr><td>WL</td><td>4.91</td><td>4.47</td><td>3.18</td><td>4.92</td><td>4.10</td><td>4.18</td><td>2.02</td><td>2.89</td><td>4.24</td><td>2.88</td></tr><tr><td>TL</td><td>5.86</td><td>5.39</td><td>4.38</td><td>5.88</td><td>5.11</td><td>4.82</td><td>3.73</td><td>3.94</td><td>4.88</td><td>3.94</td></tr><tr><td>N<sub>type</sub></td><td>3442</td><td>2725</td><td>1744</td><td>3449</td><td>2441</td><td>2571</td><td>1342</td><td>1339</td><td>2624</td><td>1332</td></tr><tr><td>N<sub>token</sub></td><td>23.1k</td><td>25.3k</td><td>35.6k</td><td>23.0k</td><td>27.6k</td><td>27.1k</td><td>56.1k</td><td>39.1k</td><td>26.7k</td><td>39.2k</td></tr></tbody></table>