This paper describes a method using morphological rules and heuristics , for the automatic extraction of large-coverage lexicons of stems and root word-forms from a raw text corpus . We cast the problem of high-coverage lexicon extraction as one of stemming followed by root word-form selection . We examine the use of POS tagging to improve precision and recall of stemming and thereby the coverage of the lexicon . We present accuracy , precision and recall scores for the system on a Hindi corpus . Large-coverage morphological lexicons are an essential component of morphological analysers . Morphological analysers find application in language processing systems for tasks like tagging , parsing and machine translation . While raw text is an abundant and easily accessible linguistic resource , high-coverage morphological lexicons are scarce or unavailable in Hindi as in many other languages ( Clément et al . , 2004 ) . Thus , the development of better algorithms for the extraction of morphological lexicons from raw text corpora is a task of considerable importance . A root word-form lexicon is an intermediate stage in the creation of a morphological lexicon . In this paper , we consider the problem of extracting a large-coverage root word-form lexicon for the Hindi language , a highly inflectional and moderately agglutinative Indo-European language spoken widely in South Asia . Since a POS tagger , another basic tool , was available along with POS tagged data to train it , and since the error patterns indicated that POS tagging could greatly improve the accuracy of the lexicon , we used the POS tagger in our experiments on lexicon extraction . Previous work in morphological lexicon extraction from a raw corpus often does not achieve very high precision and recall ( de Lima , 1998 ; Oliver and Tadić , 2004 ) . In some previous work the process of lexicon extraction involves incremental or post-construction manual validation of the entire lexicon ( Clément et al . , 2004 ; Sagot , 2005 ; Forsberg et al . , 2006 ; Sagot et al . , 2006 ; Sagot , 2007 ) . Our method attempts to improve on and extend the previous work by increasing the precision and recall of the system to such a point that manual validation might even be rendered unnecessary . Yet another difference , to our knowledge , is that in our method we cast the problem of lexicon extraction as two subproblems : that of stemming and following it , that of root word-form selection . The input resources for our system are as follows : a ) raw text corpus , b ) morphological rules , c ) POS tagger and d ) word-segmentation labelled data . We output a stem lexicon and a root wordform lexicon . We take as input a raw text corpus and a set of morphological rules . We first run a stemming algorithm that uses the morphological rules and some heuristics to obtain a stem dictionary . We then create a root dictionary from the stem dictionary . The last two input resources are optional but when a POS tagger is utilized , the F-score ( harmonic mean of precision and recall ) of the root lexicon can be as high as 94.6 % . In the rest of the paper , we provide a brief overview of the morphological features of the Hindi language , followed by a description of our method including the specification of rules , the corpora and the heuristics for stemming and root word-form selection . We then evaluate the system with and without the POS tagger . We have described a system for automatically constructing a root word-form lexicon from a raw text corpus . The system is rule-based and utilizes a POS tagger . Though preliminary , our results demonstrate that it is possible , using this method , to extract a high-precision and high-recall root word-form lexicon . Specifically , we show that with a POS tagger capable of labelling wordforms with POS categories at an accuracy of about 88 % , we can extract root word-forms with an accuracy of about 87 % and a precision and recall of 94.1 % and 95.3 % respectively . Though the system has been evaluated on Hindi , the techniques described herein can probably be applied to other inflectional languages . The rules selected by the system and applied to the wordforms also contain information that can be used to determine the paradigm membership of each root word-form . Further work could evaluate the accuracy with which we can accomplish this task .

Challenge: High-coverage morphological lexicons are scarce or unavailable in Hindi calling algorithms that extract morphological lexicons from raw corpora with high recall and precision.
Approach: They propose to use morphological rules and heuristics to automatically extract lexicons of stems and root word forms from a raw corpus.
Outcome: They show that high-precision and high-recall root word-form lexicon can be extracted from raw corpora in Hindi, and a POS tagger improves results.