Patent Document ID: 8311806
Application ID: 12240671

Base Claim:
1. A method of processing a sequence of tokens to detect predetermined data, comprising: receiving the sequence of tokens, wherein each token has at least one token type and the predetermined data has a structure that comprises at least one required token of first predetermined token type and further includes at least one other optional token of a second predetermined token type; comparing, by a processor in a starting state of a decision tree, a token type of a first token in the sequence with one or more conditions provided for the starting state; if one of the starting state conditions is not met, determining the predetermined data cannot be detected; and if one of said starting state conditions is met, updating the sequence by consuming the first token and proceeding to a next state of the decision tree corresponding to the met condition, if there is a token remaining in the updated sequence, comparing the token type of the token in the updated sequence with one or more conditions provided for the now current state, if one of said next state conditions is met, updating the sequence by consuming the token and proceeding to the next state corresponding to the met condition, iterating said comparing of the updated sequence and said updating of the sequence until there is no token remaining in the sequence matching a condition of the current state, determining whether a reduction is associated with the current state and, if so, using the reduction, and if no reduction is associated with the current state, determining that the predetermined data cannot be detected.

---

Claim 5:
5. A method according to claim 1 further comprising: providing the decision tree for detecting the predetermined data, wherein the predetermined data has a structure that comprises a predetermined sequence of token types, at least one token type being optional, wherein the tree comprises a plurality of states, each of said plurality of states being linked with at least one other state by a respective condition, and an arrangement of linked states forms a plurality of paths; and comparing token types of the sequence of tokens to respective conditions in the decision tree to match the sequence of tokens to one or more paths in the decision tree, wherein a path is provided in the decision tree for the predetermined sequence including the at least one optional token of second predetermined token type and a separate path is provided for the predetermined sequence without the at least one optional token of the second predetermined token type.