MRL Parsing Without Tears: The Case of H ebrew (2024)

Abstract: Syntactic parsing remains a critical tool for relation extraction and information extraction, especially in resource-scarce languages where LLMs are lacking. Yet in morphologically rich languages (MRLs), where parsers need to identify multiple lexical units in each token, existing systems suffer in latency and setup complexity. Some use a pipeline to peel away the layers: first segmentation, then morphology tagging, and then syntax parsing; however, errors in earlier layers are then propagated forward. Others use a joint architecture to evaluate all permutations at once; while this improves accuracy, it is notoriously slow. In contrast, and taking Hebrew as a test case, we present a new “flipped pipeline”: decisions are made directly on the whole-token units by expert classifiers, each one dedicated to one specific task. The classifier predictions are independent of one another, and only at the end do we synthesize their predictions. This blazingly fast approach requires only a single huggingface call, without the need for recourse to lexicons or linguistic resources. When trained on the same training set used in previous studies, our model achieves near-SOTA performance on a wide array of Hebrew NLP tasks. Furthermore, when trained on a newly enlarged training corpus, our model achieves a new SOTA for Hebrew POS tagging and dependency parsing. We release this new SOTA model to the community. Because our architecture does not rely on any language-specific resources, it can serve as a model to develop similar parsers for other MRLs.

## a new mrl parsing proposal as detailed in the previous section, existing mrl parsing systems suffer from several primary shortcomings. pipeline architectures suffer from the compounded of errors from one level to the next; joint architectures entail slow computations of lattices covering all permutations; and almost all systems rely on external lexicons and other components which are an obstacle to straightforward online integration, and which may lead to uninformed predictions when faced with oov (out of vocabulary) words. is it possible to overcome these issues in an mrl parsing system? we believe it is, by utilizing whole-token prediction, a flipped pipeline, and eliminating the lexicon. we elaborate on each of these elements in the following sections. ## model implementation the overarching task is defined as follows: given an input sequence of whole tokens x 1 ...x n , we aim to predict a tree structure connecting s 1 ...s m segments with a predicted set of respective feature sets f 1 ...f m for each segment. notably, for mrls it is often the case that @xmath0. the model starts off by feeding the sequence of whole tokens x 1 ...x n into a pre-trained llm encoder, and then feeds each contextualized embedding c 1 ...c n into individual expert classifiers for each of the following tasks: foot_5 dependency tree parsing, lemmatization, morphological functions disambiguation, morphological form segmentation, and named entity recognition. we then synthesize the outputs of the expert classifiers into a unified ud analysis. in what follows, we elaborate first on the implementation details of each of these classifiers, and then we describe our synthesis procedure (see section 3.6). ## conclusion we proposed a new solution for mrl parsing to overcome the deficiencies of current approaches (pipelines with error propagation, joint parsers with long latencies, and cumbersome setup and integration due external lexicons and resources). our key innovation is a "flipped pipeline", in which the multiple layers involved in mrl parsing are predicted independently on a whole-token basis and then synthesized. our approach has the substantial advantage of providing a feasible approach to parsing mrl text without the need to rely on lexicons or any other hand-crafted linguistic resources. these aspects of our architecture translate into a substantial boost in practical usability over previous parsers, both in terms of speed and in terms of ease of installation and integration. when training on the same training corpus as previous studies, our architecture provides near-sota scores on hebrew nlp tasks; when training on our enlarged training corpora, we set a new sota for hebrew pos tagging and dependency parsing. we release our new sota hebrew parsing models to the nlp community on huggingface, under a cc by 4.0 license. finally, because our architecture does not rely on any hebrew-specific resources, it is readily adaptable to other mrls as well.