Patent ID: 6606620
Filing Date: 2003-08-12
Classification: G06F,Y10S

Abstract:
A classifier, for use on a computer readable medium, for dynamically classifying a semi-structured document with a schema, comprising:a vectorization module for parsing the document into a structured vector model, wherein the structured vector model is divided into a tree of sub-vectors to reflect a plurality of hierarchical levels beginning with a root and ending with a plurality of leaves; a sorting module for searching the, document and for counting the occurrences of individual terms in the document; the sorting module further accounting for the frequency of occurrence of the terms at each hierarchical level to achieve a high contextual sensitivity; a testing module for assigning a class to the document by using a statistical model based on probability calculation to create a classification model; and wherein the classification model assigns a class to the document that maximizes a posteriori class probability Pr[c|d,Fk] according to the following expression: Pr[c&RightBracketingBar;&it;d,Fk]=Ï€&it;â€ƒ&it;(c)&it;â€ƒ&it;&Product;â€ƒ&it;â€‰t&it;â€ƒ&it;&epsi;&it;â€ƒ&it;pd&it;â€ƒ&it;(i,j),t&it;â€ƒ&it;&epsi;&it;â€ƒ&it;d&Intersection;Fk&it;â€ƒ&it;(ed&it;â€ƒ&it;(i,j))&it;f&it;â€ƒ&it;(c,pd&it;â€ƒ&it;(i,j),t)n&af;(d,pd&it;â€ƒ&it;(i,j),t)â€ƒ&Sum;â€ƒ&it;â€‰câ€²&it;Ï€&it;â€ƒ&it;(câ€²)&it;â€ƒ&it;&Product;â€ƒ&it;â€‰t&it;â€ƒ&it;&epsi;&it;â€ƒ&it;pd&it;â€ƒ&it;(i&CenterDot;),t&it;â€ƒ&it;&epsi;&it;â€ƒ&it;d&Intersection;Fk&it;â€ƒ&it;(ed&it;â€ƒ&it;(i,j))&it;f&it;â€ƒ&it;(c,pd&it;â€ƒ&it;(i,j),t)n&af;(d,pd&it;â€ƒ&it;(i,j),t)â€ƒwhere d is the document, p(c) is a prior distribution on the class c; câ€² is a class in a set of documents; pd is a path to a structure node ed from a root; n is a number of occurrences of term t in pd, f is a maximum likelihood estimation; Fk is a set of selected terms; F is a Fisher index defined by the following equation, where c1 and c2 are children of an internal class c0, and m is an average number of an occurrence of term t in class c: F&it;â€ƒ&it;(t)=&Sum;â€ƒ&it;(Î¼&it;â€ƒ&it;(c1,t)-Î¼&it;â€ƒ&it;(c2,t))2c1,c2&Sum;â€ƒ&it;â€‰c&it;1&LeftBracketingBar;c&RightBracketingBar;&it;â€ƒ&it;&Sum;â€ƒ&it;(f&it;â€ƒ&it;(t,d,c)-Î¼&it;â€ƒ&it;(c,t))2d&it;â€ƒ&it;&epsi;&it;â€ƒ&it;c.