Patent ID: 11971936
Assignee: GOOGLE LLC
Field: Computer technology (Electrical engineering)
Classification: CPC G | IPC G

Claim 16:
17. A system comprising one or more processors and memory storing instructions that, in response to execution by the one or more processors, cause the one or more processors to:
obtain an unknown web page that includes one or more interactive elements, wherein the unknown web page is operable to carry out at least a portion of a procurement and the one or more interactive elements are operable to input one or more parameters for carrying out the at least portion of the procurement;
generate a first set of universal identifier matrices, wherein the generating includes, for each interactive element of the one or more interactive elements, wherein the instructions to generate include instructions to generate a universal identifier matrix comprising a two-dimensional grid of cells that semantically captures the interactive element and its surrounding visual context, wherein each cell of the two-dimensional grid has a predetermined width and height, and the generating includes:
positioning the two-dimensional grid over the interactive element to spatially align the underlying interactive element with a central cell of the two-dimensional grid such that other pieces of content that surround the interactive element are overlaid by one or more other cells of the two-dimensional grid that are spatially adjacent the central cell, and
including content that is captured spatially within each cell of the two-dimensional grid of cells in the universal identifier matrix;

identify, from a database of past user states, based at least in part on the one or more universal identifier matrices of the first set of universal identifier matrices, one or more candidate past user states,
wherein each respective candidate past user state of the one or more candidate past user states includes one or more candidate actions for interacting with one or more interactive elements of a known web page used to generate the respective candidate past user state, and
wherein a given candidate past user state of the one or more candidate past user states is identified based on matching one or more of the universal identifier matrices of the first set with one or more reference universal identifier matrices associated with the given candidate past user state, wherein the matching is based on a similarity measure between one or more of the universal identifier matrices and one or more of the reference universal identifier matrices;

attempt one or more candidate actions associated with one or more of the candidate past user states on one or more of the interactive elements of the unknown web page; and
based on one or more outcomes of the attempted one or more candidate actions, selectively classify the unknown web page as validated or invalidated;
wherein validated web pages are flagged as available to facilitate natural language-based automated navigation through websites that include the validated web pages.