Patent ID: 7873901

Claim:
A method comprising: analyzing, on a computer, a markup language tag tree of a web page to identify peripheral regions around periphery of the web page and any body regions between the peripheral regions, wherein the peripheral regions comprise header, footer, left, and right regions, the left and right regions are defined, respectively, as being the left and right one third of the web page; and the header and footer regions are defined, respectively, as being in an upper N and lower N pixels of the web page, wherein: each of the header and footer regions contains one or more markup language tag regions; and each markup language tag region has a height and width; N is a dynamic threshold for the header region determined by a base threshold added to a function of the height of the tag region divided by the width of the tag region; within the markup language tag tree defining each peripheral and body region: identifying a first set of visual boundaries from properties of the tags of the markup language tag tree; and detecting patterns in leaf markup language tags of the markup language tag tree to find one or more basic semantic units each having a shape; projecting the shape of each basic semantic unit normal to perpendicular axes; identifying a second set of visual boundaries of the web page from the projection values on each perpendicular axis; within the markup language tag tree that defines each peripheral and body region, identifying each logical block that is adjacent to any of the first set or the second set of identified visual boundaries; and storing, using an annotation mechanism, a representation of the web page including: the logical blocks within each peripheral and body region; the peripheral regions; the body regions; and the visual boundaries.