Patent Document ID: 7580921
Application ID: 10900021

Base Claim:
1. A computer implemented method for identifying phrases in a document collection, the method comprising: collecting possible phrases from documents in the document collection; classifying individual possible phrases as either a good phrase or a bad phrase according to a frequency of occurrence of the individual possible phrase; determining, for a pair of good phrases g j and g k in the document collection, an information gain of g k with respect to g j as a function of an actual co-occurrence rate of g j and g k and an expected co-occurrence rate of g j and g k in the document collection; selectively retaining as valid phrases those good phrases that, predict the occurrence of at least one other good phrase in the document collection, where a good phrase g j predicts the occurrence of another good phrase g k in the document collection when the determined information gain of g k in the presence of g j exceeds a first predetermined threshold; identifying, for a plurality of selectively retained valid phrases g x , a phrase g y as a related phrase of g x where the information gain of g y in the presence of g x exceeds a second predetermined threshold that is more restrictive than the first predetermined threshold; and storing the valid phrases and identified related phrases on a computer-readable storage medium.

---

Claim 17:
17. The method of claim 1 , further comprising: for each phrase g x , identifying a set R including a plurality of related phrases; determining for each pair of related phrases in set R an information gain of the pair of related phrases; and identifying as a cluster of related phrases of g x , phrase g x , and each related phrase in set R that has non-zero information gain with respect to each other phrase in set R.