Abstract: When parsing morphologically-rich languages with neural models, it is beneficial to model input at the character level, and it has been claimed that this is because character-level models learn morphology. We test these claims by comparing character-level models to an oracle with access to explicit morphological analysis on twelve languages with varying morphological typologies. Our results highlight many strengths of character-level models, but also show that they are poor at disambiguating some words, particularly in the face of case syncretism. We then demonstrate that explicitly modeling morphological case improves our best model, showing that character-level models can benefit from targeted forms of explicit morphological modeling.

Introduction: Modeling language input at the character level BIBREF0 , BIBREF1 is effective for many NLP tasks, and often produces better results than modeling at the word level. For parsing, ballesteros-dyer-smith:2015:EMNLP have shown that character-level input modeling is highly effective on morphologically-rich languages, and the three best systems on the 45 languages of the CoNLL 2017 shared task on universal dependency parsing all use character-level models BIBREF2 , BIBREF3 , BIBREF4 , BIBREF5 , showing that they are effective across many typologies. The effectiveness of character-level models in morphologically-rich languages has raised a question and indeed debate about explicit modeling of morphology in NLP. BIBREF0 propose that “prior information regarding morphology ... among others, should be incorporated” into character-level models, while BIBREF6 counter that it is “unnecessary to consider these prior information” when modeling characters. Whether we need to explicitly model morphology is a question whose answer has a real cost: as ballesteros-dyer-smith:2015:EMNLP note, morphological annotation is expensive, and this expense could be reinvested elsewhere if the predictive aspects of morphology are learnable from strings. Do character-level models learn morphology? We view this as an empirical claim requiring empirical evidence. The claim has been tested implicitly by comparing character-level models to word lookup models BIBREF7 , BIBREF8 . In this paper, we test it explicitly, asking how character-level models compare with an oracle model with access to morphological annotations. This extends experiments showing that character-aware language models in Czech and Russian benefit substantially from oracle morphology BIBREF9 , but here we focus on dependency parsing (§ "Dependency parsing model" )—a task that benefits substantially from morphological knowledge—and we experiment with twelve languages using a variety of techniques to probe our models. Our summary finding is that character-level models lag the oracle in nearly all languages (§ "Experiments" ). The difference is small, but suggests that there is value in modeling morphology. When we tease apart the results by part of speech and dependency type, we trace the difference back to the character-level model's inability to disambiguate words even when encoded with arbitrary context (§ "Analysis" ). Specifically, it struggles with case syncretism, in which noun case—and thus syntactic function—is ambiguous. We show that the oracle relies on morphological case, and that a character-level model provided only with morphological case rivals the oracle, even when case is provided by another predictive model (§ "Characters and case syncretism" ). Finally, we show that the crucial morphological features vary by language (§ "Understanding head selection" ).

Question: Who made the stated claim (that "this is because character-level models learn morphology")?
Answer: Chung et al. (2016)