Patent ID: 7257589

Claim:
A computer-implemented method of providing information to a user based upon contents of a first document displayed to the user, the method comprising: identifying at least a first section of the first document; extracting a first set of one or more information objects from the first section of the first document, the first set of information objects comprising at least a first information object comprising information of a first type, wherein the first information object is one of an audio object, a video object, or an image object; processing information objects in the first set of information objects and information objects in a second set of information objects to determine a degree of relevancy information indicating the relevancy of information objects in the second set of information objects to information objects in the first set of information objects, wherein the second set of information objects comprises at least one information object comprising information of a second type, wherein the second type is different from the first type, and wherein determining the degree of relevancy information comprises determining a degree of relevancy of the at least one information object to the first information object from the first set of information objects; and selecting a third set of one or more information objects from information objects in the second set of information objects based upon the degree of relevancy information determined for information objects in the second set of information objects, wherein information objects in the third set of information objects store information to be output to the user when the first document is being displayed to the user.