Patent ID: 8612420

Claim:
A system for configuring web crawling, comprising: one or more processors to: obtain a webpage comprising a plurality of nodes; present the webpage to a user; receive a user selection of a node in the webpage, the node comprising at least one element; in response to the user selection of the node, present a web crawling configuration option pertaining to a web crawling action to be performed with respect to the node, the web crawling configuration option depending at least in part on a type of an element included in the node, wherein the presenting of the web crawling configuration option comprises determine whether the selected node includes text, an image, a link, a form element, or any combination thereof, wherein: in the event that the selected node includes text, the web crawling action comprises extract the text; in the event that the selected node includes a link, the web crawling action comprising download the link, extract alternative text, capture a target uniform resource locator (URL), extract text of the link, or any combination thereof; and in the event that the selected node includes an image, the web crawling action comprises capture the image, capture image name and image description, perform image processing, perform image transformation, extract alternative text from the image, or any combination thereof; receive a user input specifying the web crawling configuration options pertaining to the web crawling action to be performed with respect to the node; and store user specified web crawling configuration options, perform the web crawling action on the node according to the user input, or both; and one or more memories coupled to the one or more processors, to provide the one or more processors with instructions.