Common tasks involving orthographic words include spellchecking, stemming, morphological analysis, and morphological synthesis. To enable significant reuse of the language-specific resources across all such tasks, we have extended the functionality of the open source spellchecker MySpell, yielding a generic word analysis library, the runtime layer of the hunmorph toolkit. We added an offline resource management component, hunlex, which complements the efficiency of our runtime layer with a high-level description language and a configurable precompiler. Word-level analysis and synthesis problems range from strict recognition and approximate matching to full morphological analysis and generation. Our technology is predicated on the observation that all of these problems are, when viewed algorithmically, very similar: the central problem is to dynamically analyze complex structures derived from some lexicon of base forms. Viewing word analysis routines as a unified problem means sharing the same codebase for a wider range of tasks, a design goal carried out by finding the parameters which optimize each of the analysis modes independently of the language-specific resources. The C/C++ runtime layer of our toolkit, called hunmorph, was developed by extending the codebase of MySpell, a reimplementation of the wellknown Ispell spellchecker. Our technology, like the Ispell family of spellcheckers it descends from, enforces a strict separation between the language-specific resources (known as dictionary and affix files), and the runtime environment, which is independent of the target natural language. Compiling accurate wide coverage machinereadable dictionaries and coding the morphology of a language can be an extremely labor-intensive task, so the benefit expected from reusing the language-specific input database across tasks can hardly be overestimated. To facilitate this resource sharing and to enable systematic task-dependent optimizations from a central lexical knowledge base, we designed and implemented a powerful offline layer we call hunlex. Hunlex offers an easy
In an effort most similar to our own, the hunmorph project extended the functionality of the spellchecker MySpell to use its resources for stemming and morphological analysis, applied to Hungarian and several related languages REF .
Hunmorph: Open Source Word Analysis
Proceedings of ACL SIGLEX Workshop on Software
Computer Science