Patent Document ID: 7676358
Application ID: 10670675

Base Claim:
1. A method to process a document, comprising: partitioning, with a tokenizer operating on at least one computer, document text separated by spaces into a plurality of tokens based on the spaces; identifying, with a token processing unit operating on at least one computer, tokens to be ignored and not considered; determining, with the token processing unit, that a first token considered of the plurality of tokens comprises a chemical name fragment, wherein determining comprises: examining syntax of the first token, examining context of the first token with respect to at least one adjacent token of the plurality of tokens, and taking into account the syntax and the context, applying to the first token a plurality of regular expressions, rules, and a plurality of dictionaries comprised of a prefix dictionary, and a suffix dictionary to recognize the chemical name fragments; adding, with the token processing unit, the recognized chemical name fragment to a vector of chemical name fragments, where the chemical name fragment is identified by a vector index variable; combining, with the token processing unit, the recognized chemical name fragment with at least one of the adjacent tokens that are determined to be a chemical name fragment into a complete chemical name, where combining comprises: initializing the chemical name fragment vector index variable, incrementing the chemical name fragment vector index variable, where the incrementing continues at least until no chemical name fragments remain; setting a string combination to include the chemical name fragments identified by the initialized and incremented chemical name fragment vector index variables, and adding the string combination to a vector c as the complete chemical name; assigning, with a sentence parser unit operating on at least one computer, the complete chemical name with one part of speech; and storing in a memory the complete chemical name assigned with the one part of speech.

---

Claim 5:
5. A method as in claim 1 , where application of said regular expressions and rules results in punctuation characters being one of maintained or removed between chemical name fragments as a function of context.