Patent Document ID: 20090327304
Application ID: 12163898
Patent Status: 0

Claim One:
1. A method for normalizing Uniform Resource Locators (URLs) having delimiters, comprising: (a) obtaining a plurality of related URLs; (b) parsing the URLs with standardized delimiters to obtain a current token from each URL to be further normalized, each current token being a member of a set of current tokens; (c) identifying a set of deep tokens, each appearing in at least one of the current tokens; (d) determining anchors from the set of deep tokens; (e) forming patterns for the current tokens based on the anchors, the patterns comprising the anchors separated by wildcard subtokens comprising one or more deep tokens of the set; (f) organizing each pattern formed in a tree of nodes, where each pattern formed is a sibling node to each other; (g) selecting a wildcard subtoken from a pattern to represent a new set of current tokens, and providing a child node of the node corresponding to that pattern; (h) repeating (d)-(g) with the new set of current tokens until it is determined that no more subtokenization is necessary; and (i) tagging deep tokens as either keys or values based on organization information from the tree, and initial tag assignments.