Patent ID: 8090571

Claim:
Within a system comprising a central processing unit and a memory, a method for building and contracting a linguistic dictionary, the linguistic dictionary comprising a list of surface forms and a list of normalized forms, each normalized form being associated with a surface form, the method comprising the steps of: stemming the surface forms to derive a normalized form; applying a pruning algorithm to a trie-based data structure by populating the trie-based data structure with a surface form and generating a generic rule for applying to the surface form, wherein the generic rule describes how the surface form is to be processed, wherein the pruning step comprises traversing the trie-based data structure from node to node until the generic rule is found, comparing the generic rule with a most frequently occurring rule and if the generic rule is identical to the most frequently occurring rule removing the nodes associated with the generic rule; comparing each character of a surface form with each character of the surface form's normalized form, wherein the normalized form specifies a first form of a word and the surface form specifies a second and different form of the same word; in response to the comparing step, determining an edit operation for each character compared; generating, via the central processing unit, a transform code from the set of the edit operations specifying how to transform the surface form to its normalized form; and storing the transform code within the memory.