Patent ID: 7529761

Claim:
A method in a computer system with a processor for labeling observations, the observations representing object elements of a web object of a web page, a web object of a web page being a collection of information that relates to a single object, the method comprising: receiving an identification of a web object of a web page; dividing by the processor the identified web object into a two-dimensional grid of object elements of the identified web object according to position and size of the object elements based on visual features of the web page, each object element representing an attribute value for the single object; representing the grid as a graph with vertices and edges, each vertex representing an object element and each edge representing a positional relationship between the object elements; representing diagonals of the graph as sequences of states; and determining by the processor a labeling for the object elements using a conditional random fields technique that factors in the relationships in two dimensions using the sequences of states representing the diagonals of the graphs; wherein when an object element represents multiple positions within the grid, representing the object element using a real state at one position and a virtual state at another position within the grid.