A daptor G rammars for Unsupervised Paradigm Clustering (2021)

Abstract: This work describes the Edinburgh submission to the SIGMORPHON 2021 Shared Task 2 on unsupervised morphological paradigm clustering. Given raw text input, the task was to assign each token to a cluster with other tokens from the same paradigm. We use Adaptor Grammar segmentations combined with frequency-based heuristics to predict paradigm clusters. Our system achieved the highest average F1 score across 9 test languages, placing first out of 15 submissions.

## introduction while the task of supervised morphological inflection has seen dramatic gains in accuracy over recent years (e.g. @xcite @xcite , unsupervised morphological analysis remains an open challenge. this is evident in the results of the 2020 sigmorphon shared task 2 on unsupervised morphological paradigm completion, in which no submission consistently outperformed the baseline @xcite . the 2021 shared task 2 @xcite focuses on a subproblem from the 2020 task: given raw text input, cluster tokens together based on membership in the same morphological paradigm. for example, given the sentence "my dog met some other dogs", a successful system would assign "dog" and "dogs" to the same paradigm because they are two inflected forms of the same lemma "dog", while each other word would occupy its own cluster. furthermore, a successful system needs to cluster typologically diverse, morphologically rich languages such as finnish and navajo, with inflectional paradigms which are much larger than english paradigms. ## adaptor grammars our approach is based upon adaptor grammars, a framework which achieves state-of-the-art results on the related task of unsupervised morphological segmentation @xcite . ## results and discussion performance was evaluated using the script provided by the shared task organizers. ## conclusion the adaptor grammar framework has previously been applied to unsupervised morphological segmentation. in this paper, we demonstrate that ag segmentations can be used for the related task of unsupervised paradigm clustering with successful results, as shown by our system's performance in the 2021 sigmorphon shared task. we note that there is still considerable room for improvement in our clustering procedure. two key directions for future development are more sophisticated treatment of nonconcatenative morphology, and incorporation of additional sources of information beyond the word form alone.