Patent Document ID: 9734147
Application ID: 14499348
Patent Status: 1

Claim One:
1. A computer program product for identifying redundant structure of asynchronous web content, the computer program product comprising: a computer readable storage medium having program instructions embodied therewith, wherein the program instructions are executable by a computer processor to cause the computer processor to perform a method comprising: determining, by a processor, whether a document object model (DOM) includes a repetitive pattern of a combination, the combination including a tag of a leaf node and a tag of a parent node of the leaf node; responsive to determining the repetitive pattern of the combination, identifying, by the processor, a first inner cluster of a plurality of inner clusters, based on collapsing a set of multiple instances of the repetitive pattern into a first single instance of the repetitive pattern at a level of hierarchy within the DOM, determining, by a processor, whether one or more additional repetitive patterns of a combination is included within the DOM, based on collapsing one or more additional sets of multiple instances of a repetitive pattern of a combination into one or more additional single instances of the repetitive pattern, respectively, at levels of hierarchy within the DOM, until reaching a root node of the DOM; responsive to determining the one or more additional single instances of the one or more additional repetitive patterns, respectively, the processor identifying one or more additional inner clusters of the plurality of inner clusters; generating, by the processor, an LSH (locality sensitive hash) signature for the first inner cluster and an LSH signature for the one or more additional inner clusters, based on the first single instance and the one or more additional single instances of the repetitive pattern, respectively; determining, by the processor, a first outer cluster based on grouping a first set of inner clusters, including the first inner cluster and at least one of the one or more additional inner clusters, into a section rooted at a source node of the DOM, wherein the source node is a parent node of the first set of inner clusters; determining, by the processor, a second outer cluster, based on grouping a second set of inner clusters from the one or more additional inner clusters, into a section rooted at a distant node of the DOM, wherein the distant node is a parent node of the second set of inner clusters, and is neither a sibling nor a descendent of a sibling of the source node; responsive to determining that the first outer cluster and the second outer cluster, as a pair of outer clusters, are near repetitive, identifying, by the processor, one outer cluster of the pair of outer clusters that are near repetitive, as redundant content; wherein: the pair of outer clusters are near repetitive based on comparing an LSH signature of the source node and an LSH signature of a sub-tree of the source node, to an LSH signature of the distant node of the DOM and an LSH signature of a sub-tree of the distant node.