CLUZH at SIGMORPHON 2022 Shared Tasks on Morpheme Segmentation and Inflection Generation (2022)

Abstract: This paper describes the submissions of the team of the Department of Computational Linguistics, University of Zurich, to the SIGMORPHON 2022 Shared Tasks on Morpheme Segmentation and Inflection Generation. Our submissions use a character-level neural transducer that operates over traditional edit actions. While this model has been found particularly wellsuited for low-resource settings, using it with large data quantities has been difficult. Existing implementations could not fully profit from GPU acceleration and did not efficiently implement mini-batch training, which could be tricky for a transition-based system. For this year’s submission, we have ported the neural transducer to PyTorch and implemented true mini-batch training. This has allowed us to successfully scale the approach to large data quantities and conduct extensive experimentation. We report competitive results for morpheme segmentation (including sharing first place in part 2 of the challenge). We also demonstrate that reducing sentence-level morpheme segmentation to a word-level problem is a simple yet effective strategy. Additionally, we report strong results in inflection generation (the overall best result for large training sets in part 1, the best results in low-resource learning trajectories in part 2). Our code is publicly available.

## introduction this paper describes our submissions to the following sigmorphon 2022 shared tasks: segm morpheme segmentation @xcite 1. typologically diverse morphological inflection @xcite 2. morphological acquisition trajectories @xcite all our submissions rely on the same neural hardattention transducer architecture that has shown strong language-independent performance in a variety of character-level transduction tasks in morphology, grapheme-to-phoneme conversion, and text normalization @xcite . ## model description as a basis for all our submissions, we use a neural character-level transducer that edits the input string into the output string by a sequence of traditional edit actions: substitutions, insertions, deletion, and copy. the specific version of this approach was developed for grapheme-to-phoneme conversion @xcite . such neural transducers have typically performed well in morphological and related character-level transduction tasks in low to medium training data settings. although they can be competitive in large-data regimes @xcite , their successful application to large data settings with appropriately large parameter sizes (cf. the transformerbased models of @xcite have over 7m parameters) may also be limited by a specific implementation. in this year's submission, we scale the approach to large datasets by porting it to a different framework and making algorithmic improvements to training. true mini-batch training. the training procedure for transition-based systems could be difficult to batch @xcite , which is why many systems are trained by gradient accumulation over individual samples (and possibly relying on library optimizations such as dynet autobatch @xcite ). this results in slow training for large data sets. in our im- ## submission details for both tasks, we train separate models for each language and use the development set exclusively for model selection. ## conclusion this paper presents the submissions of the department of computational linguistics, university of zurich, to the sigmoprhon 2022 morpheme segmentation and inflection generation shared tasks. we build on the previous architecture, the neural transducer over edit actions, porting it to a new deep learning framework and implementing gpuoptimized mini-batch training. this permits scaling the system to large training datasets, as demonstrated by strong performance in both shared tasks. we show that reducing sentence-level morpheme segmentation to a word-level problem is a viable strategy. conditioning on pos tags brings further improvements. we leave it to future work to explore more powerful representations of context. we experimented with a transformer-based encoder for morpheme segmentation, and while the initial results were not satisfactory, we intent to pursue this further. in inflection generation, we note problems with capturing unseen lemmas, despite otherwise strong performance across data regimes.