Patent Document ID: 8892420
Application ID: 13298941

Base Claim:
1. A method of text processing, comprising: training, using a processor, a classifier for classifying text, wherein: the training is based on a plurality of training sample entries; a training sample entry in the plurality of training sample entries includes: a character count; an independent use rate; a phrase structure rule value indicating whether the training sample entry complies with phrase structure rules; a semantic attribute value indicating an inclusion state of the training sample entry in a predetermined set of enumerated entries; an overlap attribute value indicating overlap of the training sample entry with another entry in the predetermined set of enumerated entries; and a classification result indicating whether the training sample entry is a compound semantic unit or a smallest semantic unit; building, using the processor, a lexicon of smallest semantic units, comprising: receiving an entry to be classified; using the trained classifier to determine whether the entry to be classified is a smallest semantic unit or a compound semantic unit; and in the event that the entry is determined to be a smallest semantic unit, adding the entry to the lexicon of smallest semantic units; segmenting, using the processor, received text based on the lexicon of smallest semantic units to obtain medium-grained segmentation results; merging, using the processor, the medium-grained segmentation results to obtain coarse-grained segmentation results, the coarse-grained segmentation results having coarser granularity than the medium-grained segmentation results; looking up, using the processor, in the lexicon of smallest semantic units respective search elements that correspond to segments in the medium-grained segmentation results; and forming, using the processor, fine-grained segmentation results based on the respective search elements, the fine-grained segmentation results having finer granularity than the medium-grained segmentation results.

---

Claim 4:
4. The method of claim 1 , wherein using the trained classifier to determine whether the entry is a smallest semantic unit or a compound semantic unit includes inputting into the trained classifier: a character count of the entry, an independent use rate of the entry, a phrase structure rule indicator indicating whether the entry complies with phrase structure rules, a semantic attribute indicating an inclusion state of the entry in the predetermined set of enumerated entries, and an overlap attribute of the entry.