Source: http://www.google.com/patents/US7599985?dq=7,599,985&hl=en&sa=X&ei=8O5UUJf6HsvyyAH_mIHgAQ&sqi=2&pjf=1&ved=0CC4Q6AEwAA
Timestamp: 2017-01-21 04:48:43
Document Index: 99529457

Matched Legal Cases: ['art 1', 'art 2', 'art 3', 'art 4', 'art 5', 'art 6', 'art 7', 'art 8', 'art 9', 'art 10', 'art 11', 'art 12', 'art 13', 'art 1', 'art 2', 'art 3', 'art 4', 'art 5', 'art 6', 'art 7', 'art 8', 'art 9']

Patent US7599985 - Distributed hypermedia method and system for automatically invoking external ... - Google PatentsSearch Images Maps Play YouTube News Gmail Drive More »Sign inPatentsA system allowing a user of a browser program on a computer connected to an open distributed hypermedia system to access and execute an embedded program object. The program object is embedded into a hypermedia document much like data objects. The user may select the program object from the screen. Once...http://www.google.com/patents/US7599985?utm_source=gb-gplus-sharePatent US7599985 - Distributed hypermedia method and system for automatically invoking external application providing interaction and display of embedded objects within a hypermedia documentAdvanced Patent SearchTry the new Google Patents, with machine-classified Google Scholar results, and Japanese and South Korean patents.Publication numberUS7599985 B2Publication typeGrantApplication numberUS 10/217,955Publication dateOct 6, 2009Filing dateAug 9, 2002Priority dateOct 17, 1994Fee statusPaidAlso published asUS5838906, US8082293, US8086662, US9195507, US20030154261Publication number10217955, 217955, US 7599985 B2, US 7599985B2, US-B2-7599985, US7599985 B2, US7599985B2InventorsMichael Doyle, David Martin, Cheong AngOriginal AssigneeRegents Of The University Of CaliforniaExport CitationBiBTeX, EndNote, RefManPatent Citations (33), Non-Patent Citations (121), Referenced by (26), Classifications (35), Legal Events (4) External Links: USPTO, USPTO Assignment, EspacenetDistributed hypermedia method and system for automatically invoking external application providing interaction and display of embedded objects within a hypermedia document
US 7599985 B2Abstract
1. A method for running an application program in a distributed hypermedia network environment, wherein the network environment comprises at least one client workstation and one network server coupled to the network environment, the method comprising:
receiving, at the client workstation from the network server over the network environment, at least one file containing information to enable a browser application to display at least a portion of a distributed hypermedia document within a browser-controlled window;
executing the browser application on the client workstation, with the browser application:
responding to text formats to initiate processing specified by the text formats;
displaying at least a portion of the document within the browser-controlled window;
identifying an embed text format which corresponds to a first location in the document, where the embed text format specifies the location of at least a portion of an object external to the file, where the object has type information associated with it;
utilizing the type information to identify and locate an executable application external to the file; and
automatically invoking the executable application, in response to the identifying of the embed text format, to execute on the client workstation in order to display the object and enable an end-user to directly interact with the object while the object is being displayed within a display area created at the first location within the portion of the hypermedia document being displayed in the browser-controlled window.
the information to enable comprises text formats.
3. The method of claim 2 where the text formats are HTML tags.
4. The method of claim 1 where the information contained in the file received comprises at least one embed text format.
5. The method of claim 1 where the step of identifying an embed text format comprises:
parsing the received file to identify text formats included in the received file.
6. The method of claim 5 where the parsing is by a parser in the browser.
7. The method of claim 1 where the processing specified by the text formats is specified directly.
8. The method of claim 1 where the correspondence is implied by the order of the text format in a set of all of the text formats.
9. The method of claim 1 where the embed text format specifies the location of at least a portion of an object directly.
10. The method of claim 1 where having type information associated is by including type information in the embed text format.
11. The method of claim 1 where automatically invoking does not require interactive action by the user.
12. The method of claim 1, wherein said executable application is a controllable application and further comprising the step of:
interactively controlling said controllable application on said client workstation via inter-process communications between said browser and said controllable application.
13. The method of claim 12, wherein the communications to interactively control said controllable application continue to be exchanged between the controllable application and the browser even after the controllable application program has been launched.
14. The method of claim 13, wherein additional instructions for controlling said controllable application reside on said network server, wherein said step of interactively controlling said controllable application includes the following substeps:
issuing, from the client workstation, one or more commands to the network server;
executing, on the network server, one or more instructions in response to said commands;
sending information from said network server to said client workstation in response to said executed instructions; and processing said information at the client workstation to interactively control said controllable application.
15. The method of claim 14, wherein said additional instructions for controlling said controllable application reside on said client workstation.
16. One or more computer readable media encoded with software comprising computer executable instructions, for use in a distributed hypermedia network environment, wherein the network environment comprises at least one client workstation and one network server coupled to the network environment, and when the software is executed operable to:
receive, at the client workstation from the network server over the network environment, at least one file containing information to enable a browser application to display at least a portion of a distributed hypermedia document within a browser-controlled window;
cause the client workstation to utilize the browser to:
respond to text formats to initiate processing specified by the text formats;
display at least a portion of the document within the browser-controlled window;
identify an embed text format corresponding to a first location in the document, the embed text format specifying the location of at least a portion of an object external to the file, with the object having type information associated with it;
utilize the type information to identify and locate an executable application external to the file; and
automatically invoke the executable application, in response to the identifying of the embed text format, to execute on the client workstation in order to display the object and enable an end-user to directly interact with the object while the object is being displayed within a display area created at the first location within the portion of the hypermedia document being displayed in the browser-controlled window.
17. The computer readable media of claim 16 where:
18. The computer readable media of claim 17 where:
the text formats are HTML tags.
19. The computer readable media of claim 16 where:
the information contained in the file received comprises at least one embed text format.
20. A method of serving digital information in a computer network environment having a network server coupled the network environment, and where the network environment is a distributed hypermedia environment, the method comprising:
communicating via the network server with at least one client workstation over said network in order to cause said client workstation to:
receive, over said network environment from said server, at least one file containing information to enable a browser application to display at least a portion of a distributed hypermedia document within a browser-controlled window;
execute, at said client workstation, a browser application, with the browser application:
displaying, on said client workstation, at least a portion of the document within the browser-controlled window;
21. The method of claim 20 where:
23. The method of claim 20 where:
24. A method for running an executable application in a computer network environment, wherein said network environment has at least one client workstation and one network server coupled to a network environment, the method comprising:
enabling an end-user to directly interact with an object by utilizing said executable application to interactively process said object while the object is being displayed within a display area created at a first location within a portion of a hypermedia document being displayed in a browser-controlled window, wherein said network environment is a distributed hypermedia environment, wherein said client workstation receives, over said network environment from said server, at least one file containing information to enable said browser application to display, on said client workstation, at least said portion of said distributed hypermedia document within said browser-controlled window, wherein said executable application is external to said file, wherein said client workstation executes the browser application, with the browser application responding to text formats to initiate processing specified by the text formats, wherein at least said portion of the document is displayed within the browser-controlled window, wherein an embed text format which corresponds to said first location in the document is identified by the browser, wherein the embed text format specifies the location of at least a portion of said object external to the file, wherein the object has type information associated with it, wherein the type information is utilized by the browser to identify and locate said executable application, and wherein the executable application is automatically invoked by the browser, in response to the identifying of the embed text format.
25. The method of claim 24 where:
26. The method of claim 25 where:
27. The method of claim 24 where:
28. One or more computer readable media encoded with software comprising an executable application for use in a system having at least one client workstation and one network server coupled to a network environment, operable to:
cause the client workstation to display an object and enable an end-user to directly interact with said object while the object is being displayed within a display area created at a first location within a portion of a hypermedia document being displayed in a browser-controlled window, wherein said network environment is a distributed hypermedia environment, wherein said client workstation receives, over said network environment from said server, at least one file containing information to enable said browser application to display, on said client workstation, at least said portion of said distributed hypermedia document within said browser-controlled window, wherein said executable application is external to said file, wherein said client workstation executes said browser application, with the browser application responding to text formats to initiate processing specified by the text formats, wherein at least said portion of the document is displayed within the browser-controlled window, wherein an embed text format which corresponds to said first location in the document is identified by the browser, wherein the embed text format specifies the location of at least a portion of said object external to the file, wherein the object has type information associated with it, wherein the type information is utilized by the browser to identify and locate said executable application, and wherein the executable application is automatically invoked by the browser, in response to the identifying of the embed text format.
29. The method of claim 28 where:
31. The method of claim 28 where:
32. A method for serving digital information in a computer network environment, said method comprising:
communicating via a network server with at least one client workstation over said computer network environment in order to cause said client workstation to:
receive at said client workstation, over said computer network environment from said server, at least one file containing information to enable a browser application to display, on said client workstation, at least a portion of a distributed hypermedia document within a browser-controlled window;
utilize an executable application external to said file to enable an end-user to directly interact with an object while the object is being displayed within a display area created at a first location within the portion of the distributed hypermedia document being displayed in the browser-controlled window, with said network server coupled to said computer network environment, wherein said computer network environment has at least said client workstation and said network server coupled to the computer network environment, wherein said computer network environment is a distributed hypermedia environment, wherein said client workstation executes the browser application, with the browser application responding to text formats to initiate processing specified by the text formats, wherein at least said portion of the document is displayed within the browser-controlled window, wherein an embed text format which corresponds to said first location in the document is identified by the browser, wherein the embed text format specifies the location of at least a portion of said object external to the file, wherein the object has type information associated with it, wherein the type information is utilized by the browser to identify and locate said executable application, and wherein the executable application is automatically invoked by the browser, in response to the identifying of the embed text format.
33. The method of claim 32 where:
34. The method of claim 33 where:
35. The method of claim 32 where:
36. A method for running an application program in a distributed hypermedia network environment, wherein the distributed hypermedia network environment comprises at least one client workstation and one remote network server coupled to the distributed hypermedia network environment, the method comprising:
receiving, at the client workstation from the network server over the distributed hypermedia network environment, at least one file containing information to enable a browser application to display at least a portion of a distributed hypermedia document within a browser-controlled window;
identifying an embed text format which corresponds to a first location in the document, where the embed text format specifies the location of at least a portion of an object;
identifying and locating an executable application associated with the object; and
automatically invoking the executable application, in response to the identifying of the embed text format, in order to enable an end-user to directly interact with the object, while the object is being displayed within a display area created at the first location within the portion of the hypermedia document being displayed in the browser-controlled window, wherein the executable application is part of a distributed application, and wherein at least a portion of the distributed application is for execution on a remote network server coupled to the distributed hypermedia network environment.
37. The method of claim 36 where:
39. The method of claim 36 where:
40. A method of serving digital information in a computer network environment having a network server coupled to said computer network environment, and where the network environment is a distributed hypermedia network environment, the method comprising:
communicating via the network server with at least one remote client workstation over said computer network environment in order to cause said client workstation to:
receive, over said computer network environment from the network server, at least one file containing information to enable a browser application to display at least a portion of a distributed hypermedia document within a browser-controlled window;
automatically invoking the executable application, in response to the identifying of the embed text format, in order to enable an end-user to directly interact with the object while the object is being displayed within a display area created at the first location within the portion of the hypermedia document being displayed in the browser-controlled window, wherein the executable application is part of a distributed application, and wherein at least a portion of the distributed application is for execution on the network server.
41. The method of claim 40 where:
42. The method of claim 41 where:
43. The method of claim 40 where:
44. A method for serving digital information in a computer network environment, said method comprising:
communicating via a network server with at least a remote client workstation over the computer network environment in order to receive commands from the client workstation, with the network server coupled to said computer network environment, wherein said computer network environment has at least said client workstation and said network server coupled to the computer network environment, wherein the computer network environment is a distributed hypermedia environment, wherein the client workstation receives, over the computer network environment from the server, at least one file containing information to enable a browser application to display, on the client workstation, at least a portion of a distributed hypermedia document within a browser-controlled window, wherein the client workstation executes the browser application, with the browser application responding to text formats to initiate processing specified by the text formats, wherein at least said portion of the document is displayed within the browser-controlled window, wherein an embed text format which corresponds to a first location in the document is identified by the browser, wherein the embed text format specifies the location of at least a portion of an object, wherein an executable application associated with the object is identified and located by the browser, wherein the executable application is automatically invoked by the browser, in response to the identifying of the embed text format, to enable an end-user to directly interact with the object while the object is being displayed within a display area created at the first location within the portion of the hypermedia document being displayed in the browser-controlled window, wherein the executable application is part of a distributed application, and wherein at least a portion of the distributed application is for execution on the network server;
executing one or more instructions in response to the commands;
sending information to the client workstation in response to the executed instructions, to allow processing of the information at the client workstation to enable said end-user to directly interact with said object.
45. The method of claim 44 where:
46. The method of claim 45 where:
47. The method of claim 44 where:
the information contained in the file received comprises at least one embed text format. Description
Computer networks are becoming increasingly popular as a medium for locating and accessing a wide range of data from locations all over the world. The most popular global network is the Internet with millions of computer systems connected to it. The Internet has become popular due to widely adopted standard protocols that allow a vast interconnection of computers and localized computer networks to communicate with each other. Computer systems connected to a network such as the Internet may be of varying types, e.g., mainframes, workstations, personal computers, etc. The computers are manufactured by different companies using proprietary hardware and operating systems and thus have incompatibilities in their instruction sets, busses, software, file formats and other aspects of their architecture and operating systems. Localized computer networks connected to the Internet may be incompatible with other computer systems and localized networks in terms of the physical layer of communication including the specific hardware used to implement the network. Also, different networks use differing, incompatible protocols for transferring information and are not able to communicate with each other without a translation mechanism such as a “gateway”.
The Internet provides a uniform and open standard for allowing various computers and networks to communicate with each other. For example, the Internet uses Transfer Control Protocol/Internet Protocol (“TCP/IP”) that defines a uniform packet-switched communication standard which is ultimately used in every transfer of information that takes place over the Internet.
Other Internet standards are the HyperText Transmission Protocol (“HTTP”) that allows hypertext documents to be exchanged freely among any computers connected to the Internet and HyperText Markup Language (“HTML”) that defines the way in which hypertext documents designate links to information. See, e.g., Berners-Lee, T. J., “The world-wide web,” Computer Networks and ISDN Systems 25 (1992).
A hypertext document is a document that allows a user to view a text document displayed on a display device connected to the user's computer and to access, retrieve and view other data objects that are linked to hypertext words or phrases in the hypertext document. In a hypertext document, the user may “click on,” or select, certain words or phrases in the text that specify a link to other documents, or data objects. In this way, the user is able to navigate easily among data objects. The data objects may be local to the user's computer system or remotely located over a network. An early hypertext system is Hypercard, by Apple Computer, Inc. Hypercard is a standalone system where the data objects are local to the user's system.
When a user selects a phrase in a hypertext document that has an associated link to another document, the linked document is retrieved and displayed on the user's display screen. This allows the user to obtain more information in an efficient and easy manner. This provides the user with a simple, intuitive and powerful way to “branch off” from a main document to learn more about topics of interest.
FIG. 1 shows examples of hypertext and hypermedia documents and links associating data objects in the documents to other data objects. Hypermedia document includes hypertext 20, an image icon at 22, a sound icon at 24 and more hypertext 26. FIG. 1 shows hypermedia document 10 substantially as it would appear on a user's display screen. The user is able to select, or “click” on icons and text on a display screen by using an input device, such as a mouse, in a manner well-known in the art.
When the user clicks on the phrase “hypermedia,” software running on the user's computer obtains the link associated with the phrase, symbolically shown by arrow 30, to access hypermedia document 14. Hypermedia document 14 is retrieved and displayed on the user's display screen. Thus, the user is presented with more information on the phrase “hypermedia.” The mechanism for specifying and locating a linked object such as hypermedia document 14 is an HTML “element” that includes an object address in the format of a Uniform Resource Locator (URL).
Similarly, additional hypertext 26 can be selected by the user to access hypertext document 12 via link 32 as shown in FIG. 1. If the user selects additional hypertext 26, then the text for hypertext document 12 is displayed on the user screen. Note that hypertext document 12, itself, has hypertext at 28. Thus, the user can click on the phrase “hypermedia” while viewing document 12 to access hypermedia document 14 in a manner similar to that discussed above.
Typically, the indicator is a very small image and may be a scaled down version of the full image. The indicator may be shown embedded within the text when the text is displayed on the display screen. The user may select the indicator to obtain the full image. When the user clicks on image icon 22 browser software executing on the user's computer system retrieves the corresponding full image, e.g., a bit map, and displays it by using external software called a “viewer.” This results in the full image, represented by image 16, being displayed on the screen.
An example of a browser program is the National Center for Supercomputing Application's (NCSA) Mosaic software developed by the University of Illinois at Urbana/Champaign, Ill. Another example is “Cello” available on the Internet at http://www.law.cornell.edu/. Many viewers exist that handle various file formats such as “.TIF,” “.GIF,” formats. When a browser program invokes a viewer program, the viewer is launched as a separate process. The view displays the full image in a separate “window” (in a windowing environment) or on a separate screen. This means that the browser program is no longer active while the viewer is active. By using indicators to act as place holders for full images that are retrieved and displayed only when a user selects the indicator, data traffic over the network is reduced. Also, since the retrieval and display of large images may require several seconds or more of transfer time the user does not have to wait to have images transferred that are of no interest to the user.
As discussed above, hypermedia documents allow a user to access different data objects. The objects may be text, images, sound files, video, additional documents, etc. As used in this specification, a data object is information capable of being retrieved and presented to a user of a computer system. Some data objects include executable code combined with data. An example of such a combination is a “self-extacting” data object that includes code to “unpack” or decompress data that has been compressed to make it smaller before transferring. When a browser retrieves an object such as a self-extracting data object the browser may allow the user to “launch” the self-extracting data object to automatically execute the unpacking instructions to expand the data object to its original size. Such a combination of executable code and data is limited in that the user can do no more than invoke the code to perform a singular function such as performing the self-extraction after which time the object is a standard data object.
FIG. 2 is an example of a computer network. In FIG. 2, computer systems are connected to Internet 100, although in practice Internet 100 may be replaced by any suitable computer network. In FIG. 2, a user 102 operates a small computer 104, such as a personal computer or a work station. The user's computer is equipped with various components, such as user input devices (mouse, trackball, keyboard, etc.), a display device (monitor, liquid crystal display (LCD), etc.), local storage (hard disk drive, etc.), and other components. Typically, small computer 104 is connected to a larger computer, such as server A at 106. The larger computer may have additional users and computer systems connected to it, such as computer 108 operated by user 110. Any group of computers may form a localized network. A localized network does not necessarily adopt the uniform protocols of the larger interconnecting network (i.e., Internet 100) and is more geographically constrained than the larger network. The localized network may connect to the larger network through a “gateway” or “node” implemented on, for example, a server.
Typically, a computer on the Internet is characterized as either a “client” or “server” depending on the role that the computer is playing with respect to requesting information or providing information. Client computers are computers that typically request information from a server computer which provides the information. For this reason, servers are usually larger and faster machines that have access to many data files, programs, etc., in a large storage associated with the server. However, the role of a server may also be adopted by a smaller machine depending on the transaction. That is, user 110 may request information via their computer 108 from server A. At a later time, server A may make a request for information from computer 108. In the first case, where computer 108 issues a request for information from server A, computer 108 is a “client” making a request of information from server A. Server A may have the information in a storage device that is local to Server A or server A may have to make requests of other computer systems to obtain the information. User 110 may also request information via their computer 108 from a server, such as server B located at a remote geographical location on the Internet. However, user 110 may also request information from a computer, such as small computer 124, thus placing small computer 124 in the role of a “server.” For purposes of this specification, client and server computers are categorized in terms of their predominant role as either an information requester or provider. Clients are generally information requesters, while servers are generally information providers.
For example, hypertext document 10 of FIG. 1 may be located at user 110's client computer 108. When user 110 makes a request by, for example, clicking on hypertext 20 (i.e., the phrase “hypermedia”), user 110's small client computer 108 processes links within hypertext document 10 to retrieve document 14. In this, example, we assume that document 14 is stored at a remote location on server B. Thus, in this example, computer 108 issues a command that includes the address of document 14. This command is routed through server A and Internet 100 and eventually is received by server B. Server B processes the command and locates document 14 on its local storage. Server 14 then transfers a copy of the document back to client 108 via Internet 100 and server A. After client computer 108 receives document 14, it is displayed so that user 110 may view it.
The Internet is said to provide an “open distributed hypermedia system.” It is an “open” system since Internet 100 implements a standard protocol that each of the connecting computer systems, 106, 130, 120, 132 and 134 must implement (TCP/IP). It is a “hypermedia” system because it is able to handle hypermedia documents as described above via standards such as the HTTP and HTML hypertext transmission and mark up standards, respectively. Further, it is a “distributed” system because data objects that are imbedded within a document may be located on many of the computer systems connected to the Internet. An example of an open distributed hypermedia system is the so-called “world-wide web” implemented on the Internet and discussed in papers such as the Berners-Lee reference given above.
In one application, high resolution three dimensional images are processed in a distributed manner by several computers located remotely from the user's client computer. This amounts to providing parallel distributed processing for tasks such as volume rendering or three dimensional image transformation and display. Also, the user is able to rotate, scale and otherwise reposition the viewpoint with respect to these images without exiting the hypermedia browser software. The control and interaction of viewing the image may be provided within the same window that the browser is using assuming the environment is a “windowing” environment. The viewing transformation and volume rendering calculations may be performed by remote distributed computer systems.
375 pages of Source code on 4 microfiche Appendices A and B are provided to this specification. The source code should be consulted to provide details of a specific embodiment of the invention in conjunction with the discussion of the routines in this specification. The source code in Appendix A includes NCSA Mosaic version 2.4 source code along with modifications to the source code to implement the present invention. Appendix B includes source code implementing an application program interface. The source code is written in the “C” computer language to run on an X-Window platform.
FIG. 3 is an illustration of a computer system suitable for use with the present invention. FIG. 3 depicts but one example of many possible computer types or configurations capable of being used with the present invention. FIG. 3 shows computer system 150 including display device 153, display screen 155, cabinet 157, keyboard 159 and mouse 161.
Mouse 161 and keyboard 159 are “user input devices.” Other examples of user input devices are a touch screen, light pen, track ball, data glove, etc. Mouse 161 may have one or more buttons such as buttons 163 shown in FIG. 3. Cabinet 157 houses familiar computer components such as disk drives, a processor, storage means, etc. As used in this specification “storage means” includes any storage device used in connection with a computer system such as disk drives, magnetic tape, solid state memory, bubble memory, etc. Cabinet 157 may include additional hardware such as input/output (I/O) interface cards for connecting computer system 150 to external devices such as an optical character reader, external storage devices, other computers or additional devices.
Client computer 200 includes processes, such as browser client 208 and application client 210. In a preferred embodiment, application client 210 is resident within client computer 200 prior to browser client 208's parsing of a hypermedia document as discussed below. In a preferred embodiment application client 210 resides on the hard disk or RAM of client computer 200 and is loaded (if necessary) and executed when browser client 208 detects a link to application client 210. The preferred embodiment uses the XEvent interprocess communication protocol to exchange information between browser client 208 and application client 210 as described in more detail, below. Another possibility is to install application client 210 as a “terminate and stay resident” (TSR) program in an operating system environment, such as X-Window. Thereby making access to application client 210 much faster.
Browser client 208 is a process, such as NCSA Mosaic, Cello, etc. Application client 210 is embodied in software presently under development called “VIS” and “Panel” created by the Center for Knowledge Management at the University of California, San Francisco, as part of the Doyle Group's distributed hypermedia object embedding approach described in “Integrated Control of Distributed Volume Visualization Through the World-Wide-Web,” by C. Ang, D. Martin, M. Doyle; to be published in the Proceedings of Visualization 1994, IEEE Press, Washington, D.C., October 1994.
Versions and descriptions of software embodying the present invention are generally available as hyperlinked data objects from the Visible Embryo Project's World Wide Web document at the URL address “HTTP://visembryo.ucsf.edu/”.
FIG. 6 shows yet another embodiment of the present invention. FIG. 6 is similar to FIG. 5, except that additional computers 222 and 224 are illustrated. Each additional computer includes a process labeled “Application (Distributed).” The distributed application performs a portion of the task that an application, such as application server 220 or application client 210, perform. In the present example, tasks such as volume rendering may be broken up and easily performed among two or more computers. These computers can be remote from each other on network 206. Thus, several computers, such as server computer 204 and additional computers 222 and 224 can all work together to perform the task of computing a new viewpoint and frame buffer for the embryo for the new orientation of the embryo image in the present example. The coordination of the distributed processing can be performed at client computer 200 by application client 210, at server computer 204 by application server 220, or by any of the distributed applications executing on additional computers, such as 222 and 224. In a preferred embodiment, distributed processing is coordinated by a program called “VIS” represented by application client 210 in FIG. 6.
Another type of possible application of this invention would involve embedding a program which runs only on the client machine, but which provides the user with more functionality than exists in the hypermedia browser alone. An example of this is an embedded client application which is capable of viewing and interacting with images which have been processed with Dr. Doyle's MetaMAP invention (U.S. Pat. No. 4,847,604). This MetaMAP process uses object-oriented color map processing to allow individual color index ranges within paletted images to have object identities, and is useful for the creation of, for example, interactive picture atlases. It is a more efficient means for defining irregular “hotspots” on images than the ISMAP function of the World Wide Web, which uses polygonal outlines to define objects in images. A MetaMAP-capable client-based image browser application can be embedded, together with an associated image, within a hypermedia document, allowing objects within the MetaMAP-processed image to have URL addresses associated with them. When a user clicks with a mouse upon an object within the MetaMAP-processed image, the MetaMAP client application relays the relevant URL back to the hypermedia browser application, which then retrieves the HTML file or hypermedia object which corresponds to that URL.
Message Function Message Name
Messages from server to client:
1. Server Update Done
XtNrefreshNotify
2. Server Ready XtNpanelStartNotify
3. Server Exiting XtNpanelExitNotify
Messages from client to server:
4. Area Shown XtNmapNotify
5. Area Hidden XtNunmapNotify
6. Area Destroyed XtNexitNotify
The messages in Table I are defined in the file protocol.sub.--lib.h in Appendix B. The functions of the MEAPI are provided in protocol. sub.--lib.c of Appendix B. Thus, by using MEAPI a server process communicates to a client application program to let the client application know when the server has finished updating information, such as an image frame buffer, or pixmap (Message 1); when the server is ready to start processing messages (Message 2) and when the server is exiting or stopping computation related to the server application program.
For client to server communications, MEAPI provides for the client informing the server when the image display window area is visible, when the area is hidden and when the area is destroyed. Such information allows the server to decide whether to allocate computing resources for, e.g., rendering and viewing transformation tasks where the server is running an application program to generate new views of a multi dimensional object. Source code for MEAPI fundamental functions such as handle.sub.--client.sub. --msg, register.sub.--client, register.sub.--client.sub.--msg.sub.--callback and send.sub. --client. sub.--msg may be found in protocol.sub.--lib.c as part of the source code in Appendix B. Next, a discussion of the software processes that perform parsing of a hypermedia document and launching of an application program is provided in connection with Table II and FIGS. 7A, 7B, 8A and 8B. Table II, below, shows an example of an HTML tag format used by the present invention to embed a link to an application program within a hypermedia document.
For client to server communications, MEAPI provides for the client informing the server when the image display window area is visible, when the area is hidden and when the area is destroyed. Such information allows the server to decide whether to allocate computing resources for, e.g., rendering and viewing transformation tasks where the server is running an application program to generate new views of a multi dimensional object. Source code for MEAPI fundamental functions such as handle.sub.--client.sub. --msg, register.sub.--client, register.sub.--client.sub.--msg.sub.--callback and send.sub. client. sub.--msg may be found in protocol.sub.--lib.c as part of the source code in Appendix B.
&lt EMBED
HREF = “href”
As shown in Table II, the EMBED tag includes TYPE, HREF, WIDTH and HEIGHT elements. The TYPE element is a Multipurpose Internet Mail Extensions (MIME) type. Examples of values for the TYPE element are “application/x-vis” or “video/mpeg”. The type “application/x-vis” indicates that an application named “x-vis” is to be used to handle the object at the URL specified by the HREF. Other types are possible such as “application/x-inventor”, “application/postscript” etc. In the case where TYPE is “application/x-vis” this means that the object at the URL address is a three dimensional image object since the program “x-vis” is a data visualization tool designed to operate on three dimensional image objects. However, any manner of application program may be specified by the TYPE element so that other types of applications, such as a spreadsheet program, database program, word processor, etc. may be used with the present invention. Accordingly, the object reference by the HREF element would be, respectively; a spreadsheet object, database object, word processor document object, etc.
On the other hand, TYPE values such as “video/mpeg”, “image/gif”, “video/x-sgi-movie”, etc. describe the type of data that HREF specifies. This is useful where an external application program, such as a video player, needs to know what format the data is in, or where the browser client needs to determine which application to launch based on the data format. Thus, the TYPE value can specify either an application program or a data type. Other TYPE values are possible. HREF specifies a URL address as discussed above for a data object. Where TYPE is “application/x-vis” the URL address specifies a multi-dimensional image object. Where TYPE is “video/mpeg” the URL address specifies a video object.
FIG. 7A is a flowchart describing some of the functionality within the HTMLparse.c file of routines. The routines in HTMLparse.c perform the task of parsing a hypermedia document and detecting the EMBED tag. In a preferred embodiment, the enhancements to include the EMBED tag are made to an HTML library included in public domain NCSA Mosaic version 2.4. Note that much of the source code in is pre-existing NCSA Mosaic code. Only those portions of the source code that relate to the new functionality discussed in this specification should be considered as part of the invention. The new functionality is identifiable as being set off from the main body of source code by conditional compilation macros such as “#ifdef . . . #endif” as will be readily apparent to one of skill in the art.
In general, the flowcharts in this specification illustrate one or more software routines executing in a computer system such as computer system 1 of FIG. 1. The routines may be implemented by any means as is known in the art. For example, any number of computer programming languages, such as “C”, Pascal, FORTRAN, assembly language, etc., may be used. Further, various programming approaches such as procedural, object oriented or artificial intelligence techniques may be employed.
The steps of the flowcharts may be implemented by one or more software routines, processes, subroutines, modules, etc. It will be apparent that each flowchart is illustrative of merely the broad logical flow of the method of the present invention and that steps may be added to, or taken away from, the flowcharts without departing from the scope of the invention. Further, the order of execution of steps in the flowcharts may be changed without departing from the scope of the invention. Additional considerations in implementing the method described by the flowchart in software may dictate changes in the selection and order of steps. Some considerations are event handling by interrupt driven, polled, or other schemes. A multiprocessing or multitasking environment could allow steps to be executed “concurrently.” For ease of discussion the implementation of each flowchart may be referred to as if implemented in a single “routine”.
Returning to FIG. 7, it is assumed that a hypermedia document has been obtained at a user's client computer and that a browser program executing on the client computer displays the document and calls a first routine in the HTMLparse.c file called “HTMLparse”. This first routine, HTMLparse, is entered at step 252 where a pointer to the start of the document portion is passed. Steps 254, 256 and 258 represent a loop where the document is parsed or scanned for HTML tags or other symbols. While the file HTMLparse.c includes routines to handle all possible tags and symbols that may be encountered, FIG. 7A, for simplicity, only illustrates the handling of EMBED tags.
Assuming there is more text to parse, execution proceeds to step 256 where routines in HTMLparse.c obtain the next item (e.g., word, tag or symbol) from the document. At step 258 a check is made as to whether the current tag is the EMBED tag. If not, execution returns to step 254 where the next tag in the document is obtained. If, at step 258, it is determined that the tag is the EMBED tag, execution proceeds to step 260 where an enumerated type is assigned for the tag. Each occurrence of a valid EMBED tag specifies an embedded object. HTMLParse calls a routine “get.sub.--mark” in HTMLparse.c to put sections of HTML document text into a “markup” text data structure. Routine get.sub. ---mark, in turn, calls ParseMarkType to assign an enumerated type. The enumerated type is an identifier with a unique integer associated with it that is used in later processing described below.
FIG. 7B is a flowchart of routines in file HTMLformat.c to process the enumerated type created for the EMBED tag by routines in HTMLparse.c. In the X-Window implementation of a preferred embodiment, the enumerated type is processed as if it is a regular Motif/XT widget. For details on X-Window development see, e.g., “Xlib Programming Manual,” “X Toolkit Intrinsics Programming Manual” and “Motif Programming Manual” published by O'Reilly &amp; Associates, Inc. HTMLformat is entered at step 270 where a pointer to the enumerated type to process is passed.
At step 286 a check is made as to whether the type attribute of the object, i.e., the value for the TYPE element of the EMBED tag, is an application. If so, step 290 is executed to launch a predetermined application. In a preferred embodiment an application is launched according to a user-defined list of application type/application pairs. The list is defined as a user-configurable XResource as described in “Xlib Programming Manual.” An alternative embodiment could use the MIME database as the source of the list of application type/application pairs. The routine “vis.sub.--start.sub.--external.sub. --application” in file HTMLformat.c is invoked to match the application type and to identify the application to launch.
The external application is started as a child process of the current running process (Mosaic), and informed about the window ID of the DrawingArea created in HTMLformat. The external application is also passed information about the ID of the pixmap, the data URL and dimensions. Codes for communication such as popping-up/iconifying, start notification, quit notification and refresh notification with external applications and DrawingArea refreshing are also added. Examples of such codes are (1) “setup/start” in vis.sub.--register.sub.--client and vis. sub.--get panel.sub.--window in HTMLwidgets.c; (2) “handle messages from external applications” in vis.sub.--handle panel.sub.--msg in HTMLwidgets.c; (3) “send messages to external applications” in vis.sub.—send.sub.--msg in HTMLwidgets.c; (4) “terminate external applications” in vis.sub.--exit in HTMLwidgets.c which calls vis.sub.--send.sub.--msg to send a quit message; and (5) “respond to refresh msgs” in vis.sub.--redraw in HTMLwidgets.c.
If, at step 286, the type is determined not to be an application object (e.g., a three dimensional image object in the case of application “x-vis”) a check is made at step 288 to determine if the type is a video object. If so, step 292 is executed to launch a video player application. Parameters are passed to the video player application to allow the player to display the video object within the DrawingArea within the display of the portion of hypermedia document on the client's computer. Note that many other application objects types are possible as described above.
FIG. 8B is a flowchart for routine HTML. Routine HTML takes care of “shutting down” the objects, data areas, etc. that were set up to launch the external application and display the data object. HTML is entered at step 300 and is called when the display or other processing of the EMBED tag has been completed. At step 302 the display window is removed and the memory areas for the pixmap and internal object structure is made free for other uses. Completion of processing can be by user command or by computer control.
The present invention allows a user to have interactive control over application objects such as three dimensional image objects and video objects. In a preferred embodiment, controls are provided on the external applications' user interface. In the case of a VIS/panel application, a process, “panel” creates a graphical user interface (GUI) thru which the user interacts with the data. The application program, VIS, can be executing locally with the user's computer or remotely on a server, or on one or more different computers, on the network. The application program updates pixmap data and transfers the pixmap data (frame image data) to a buffer to which the browser has access. The browser only needs to respond to the refresh request to copy the contents from the updated pixmap to the DrawingArea. The Panel process sends messages as “Msg” sending performed by routines such as vis.sub.--send.sub.--msg and vis.sub.--handle panel.sub.--msg to send events (mousemove, keypress, etc.) to the external application.
FIG. 9 is a screen display of the invention showing an interactive application object (in this case a three dimensional image object) in a window within a browser window. In FIG. 9, the browser is NCSA Mosaic version 2.4. The processes VIS, Panel and VRServer work as discussed above. FIG. 9 shows screen display 356 Mosaic window 350 containing image window 352 and a portion of a panel window 354. Note that image window 352 is within Mosaic window 350 while panel window 354 is external to Mosaic window 350. Another possibility is to have panel window 354 within Mosaic window 350. By using the controls in panel window 354 the user is able to manipulate the image within image window 352 in real time do perform such operations as scaling, rotation, translation, color map selection, etc. In FIG. 9, two Mosaic windows are being used to show two different views of an embryo image. One of the views is rotated by six degrees from the other view so that a stereoscopic effect can be achieved when viewing the images. Communication between Panel and VIS is via “Tooltalk” described in, e.g., “Tooltalk 1.1.1 Reference Manual,” from SunSoft.
FIG. 10 is an illustration of the processes VIS, Panel and VRServer discussed above. As shown in FIG. 10, the browser process, Mosaic, communicates with the Panel process via inter-client communication mechanisms such as provided in the X-Window environment. The Panel process communicates with the VIS process through a communications protocol (ToolTalk, in the preferred embodiment) to exchange visualization command messages and image data. The image data is computed by one or more copies of a process called VRServer that may be executing on remote computers on the network. VRServer processes respond to requests such as rendering requests to generate image segments. The image segments are sent to VIS and combined into a pixmap, or frame image, by VIS. The frame image is then transferred to the Mosaic screen via communications between VIS, Panel and Mosaic. A further description of the data transfer may be found in the paper “Integrated Control of Distributed Volume Visualization Through the World-Wide-Web,” referenced above.
Patent CitationsCited PatentFiling datePublication dateApplicantTitleUS4815029Sep 23, 1985Mar 21, 1989International Business Machines Corp.In-line dynamic editor for mixed object documentsUS4847604Aug 27, 1987Jul 11, 1989Doyle Michael DMethod and apparatus for identifying features of an image on a video displayUS4949248Jul 15, 1988Aug 14, 1990Caro Marshall ASystem for shared remote access of multiple application programs executing in one or more computersUS5056057May 3, 1989Oct 8, 1991Compaq Computer CorporationKeyboard interface for use in computers incorporating terminate-and-stay-resident programsUS5146553Feb 15, 1989Sep 8, 1992Hitachi, Ltd.Document editing apparatus which activates application programs in response to document data inputUS5202828May 15, 1991Apr 13, 1993Apple Computer, Inc.User interface system having programmable user interface elementsUS5204947Oct 31, 1990Apr 20, 1993International Business Machines CorporationApplication independent (open) hypermedia enablement servicesUS5206951Apr 3, 1991Apr 27, 1993Wang Laboratories, Inc.Integration of data between typed objects by mutual, direct invocation between object managers corresponding to object typesUS5274821Aug 27, 1992Dec 28, 1993International Business Machines CorporationCommunication between prolog and an external processUS5307499May 20, 1993Apr 26, 1994Singapore Computer Systems LimitedInterpretive object-oriented facility which can access pre-compiled classesUS5321806Aug 21, 1991Jun 14, 1994Digital Equipment CorporationMethod and apparatus for transmitting graphics command in a computer graphics systemUS5321807Nov 27, 1991Jun 14, 1994Mumford Christopher JAccelerated graphics display methodUS5321808Mar 20, 1992Jun 14, 1994Hewlett-Packard CompanyDual process display serverUS5339392Dec 28, 1990Aug 16, 1994Risberg Jeffrey SApparatus and method for creation of a user definable video displayed document showing changes in real time dataUS5347632Jul 28, 1989Sep 13, 1994Prodigy Services CompanyReception system for an interactive computer network and method of operationUS5367635Aug 29, 1991Nov 22, 1994Hewlett-Packard CompanyNetwork management agent with user created objects providing additional functionalityUS5390314Oct 9, 1992Feb 14, 1995American Airlines, Inc.Method and apparatus for developing scripts that access mainframe resources that can be executed on various computer systems having different interface languages without modificationUS5418908Oct 15, 1992May 23, 1995International Business Machines CorporationSystem for automatically establishing a link between an electronic mail item and a remotely stored reference through a place mark inserted into the itemUS5425141 *Mar 21, 1994Jun 13, 1995Sun Microsystems, Inc.Managing display windows of inter-related applications using hollowed windowsUS5499369Sep 30, 1994Mar 12, 1996Microsoft CorporationMethod and system for connecting objects using alert and running statesUS5537526 *Nov 12, 1993Jul 16, 1996Taugent, Inc.Method and apparatus for processing a display document utilizing a system level document frameworkUS5544320Jun 7, 1995Aug 6, 1996Konrad; Allan M.Remote information service access system based on a client-server-service modelUS5581506Feb 2, 1995Dec 3, 1996Matsushita Electric Industrial Co., Ltd.Level-shifter, semiconductor integrated circuit, and control methods thereofUS5581686Jun 6, 1995Dec 3, 1996Microsoft CorporationMethod and system for in-place interaction with contained objectsUS5606493Jun 18, 1992Feb 25, 1997International Business Machines CorporationDistributed applications processing networkUS5613058Apr 15, 1994Mar 18, 1997Microsoft CorporationMethod and system for in-place interaction with contained objectsUS5634019Jun 6, 1995May 27, 1997Microsoft CorporationMethod and system for in-place interaction with contained objectsUS5652876Sep 18, 1996Jul 29, 1997Apple Computer, Inc.Method and apparatus for launching files created by non-resident application programsUS5754175Feb 4, 1997May 19, 1998Microsoft CorporationMethod and system for in-place interaction with contained objectsUS5801701Sep 4, 1996Sep 1, 1998Microsoft CorporationMethod and system for in-place interaction with contained objectsUS5812862 *May 10, 1993Sep 22, 1998Apple Computer, Inc.Computer-human interface system for compound documentsUS5838906Oct 17, 1994Nov 17, 1998The Regents Of The University Of CaliforniaDistributed hypermedia method for automatically invoking external application providing interaction and display of embedded objects within a hypermedia documentUS6055514Jun 21, 1996Apr 25, 2000Wren; Stephen CoreySystem for marketing foods and services utilizing computerized centraland remote facilities* Cited by examinerNon-Patent CitationsReference1"A Little History of the World Wide Web," n.pag. Online.Internet: available http://www.w3.org/History.html.2"Cello WWW Browser Release 1.01a," article obtained from the Internet, ftp.law.cornell.edu/pub/L11/Cello no DDE, Mar. 16, 1994, pp. 2-9.3"NCSA Mosaic Version Information," n.pag. Online.Internet: available http://www.ncsa.uiuc.edu/SDG/Software.4"OLE 2.0: Death to Monoliths," Byte Magazine, Mar. 1994, p. 122.5"The Second Phase of the Revolution," Wired, Oct. 1994, pp. 116-152.6 *Adie, C., Network Access to Multimedia Information, 2nd ed., RARE Project OBR(93)015, RARE, pp. 1-53, Feb. 4, 1994.7Adrian Nye, Xlib Programming Manual for Version 11, O'Reilly & Associates, Inc., title page, copyright page, pp. i-xxxiii, 1-46, and index (1988).8Anonymous, "Default File Extensions in Mosaic 2.0pre4," 1 page.9Anonymous, "Default MIME Types in Mosaic 2.0pre4," 1 page.10Anonymous, "gs.interface," 2 pages (Jul. 1993). (Downloaded from "ftp://mirror.cs.wesc.edu/pub/mirrors/ghost/gnu/ghostview/" as part of the file "ghoastview-1.5.tar.gz.").11Anonymous, "xresources.h," 15 pages (1993). (Downloaded from "Index of /pub/mosaic/Unix/source/old" as part of the file "xmosaic-1.2.tar.z"; Exhibit E has 15 pages, including printouts for: (1) the web page for downloading "xmosaic-1.2.tar.z," (2) the contents of "xmosaic-1.2.tar.z," and (3) the file "xresources.h" in "xmosaic-1.2.tar.z").12Berners-Lee, "Hypertext Markup Language (HTML)," HTML Internet Draft, IIIR working group.13Berners-Lee, T.J., et al., "The World-Wide Web," Computer Networks and ISDN Systems, 25, 1992, pp. 454-459.14 *Celentano, A., et al., A Multiple Presentation Document Management System, Proc. of the 10th Annual Int'l. Conf. on Systems Documentation, ACM Press, pp. 63-71, 1992.15Coulson, G., et al., "Extensions to ANSA for Multimedia Computing," Computer Networks and ISDN Systems 25, 1992, pp. 305-323.16 *Cox, B., Object Oriented Programming: An Evolutionary Approach, Addison-Wesley, pp. 1-12, 1987.17Crowley, T., et al., "MMCONF: An Infrastructure for Building Shared Multimedia Applications," CSCW 90 Proceedings, Oct. 1990, pp. 329-342.18Cullen, J., et al., "The Use of FTAM to Access Graphical Pictures Across Wide Area Networks," Computer Networks and ISDN Systems, 1992, pp. 337-383.19Davis, H., et al., "Microcosm: An Open Hypermedia System," Interchi '93, Apr. 1993, p. 526.20Davis, H., et al., "Towards an Integrated Information Environment with Open Hypermedia System," ACM ECHT Conference, Dec. 1992, pp. 181-190.21Defendants Trial Exhibit 215: Information Regarding Emacs (CD).22Defendant's Trial Exhibit 258: Information Regarding Microsoft OLE 2.0 Toolkit Program (CD).23Defendant's Trial Exhibit 273: Information Regarding Microsoft OLE 2.01 SDK (CD).24Defendant's Trial Exhibit 326: Information Regarding Multimedia Viewer (CD).25Defendant's Trial Exhibit 34 (information regarding Viola including Viola 930512.tar.gz.zip) [Compact Disc].26Defendant's Trial Exhibit 34 File Tree Printout.27Defendant's Trial Exhibit 37 (includes "viola.TOGO.tar.Z" and other Viola information) [Compact Disc].28Defendant's Trial Exhibit 37 File Tree Printout.29 *Designing and Writing a ToolTalk Procedural Protocol-A White Paper, Sun Microsystems, Inc., pp. 1-24, 1992.30 *Dettmer, R., X-Windows-the great integrator, IEE Review, vol. 36, No. 6, pp. 219-222, Jun. 1990.31Douglas Young, The X Window System, Programming and Applications with Xt, Prentice Hall, title page, copyright page, pp. i-x, 1-13, 123-166, 280-332, 520-533 (1990).32Doyle et al., "Processing Cross-sectional Image Data for Reconstruction of Human Developmental Anatomy from Museum Specimens," Newsletter of the Association for Computing Machinery Special Interest Group on Biomedical Computing, vol. 13, No. 1, ACM Press, coverage page, table of contents, pp. 9-15 (Feb. 1993).33DTX1030-c1.pdf (MS-SUPP1205 001 part 1).34DTX1030-c2.pdf (MS-SUPP1205 001 part 2).35DTX1030-hmml.pdf (MS-SUPP1205 001 part 3).36DTX1030-html.pdf (MS-SUPP1205 001 part 4).37DTX1030-images.pdf (MS-SUPP1205 001 part 5).38DTX1030-make.pdf (MS-SUPP1205 001 part 6).39DTX1030-other.pdf (MS-SUPP1205 001 part 7).40DTX1030-ps.pdf (MS-SUPP1205 001 part 8).41DTX1030-sgml.pdf (MS-SUPP1205 001 part 9).42DTX1030-shell.pdf (MS-SUPP1205 001 part 10).43DTX1030-v1.pdf (MS-SUPP1205 001 part 11).44DTX1030-v2.pdf (MS-SUPP1205 001 part 12).45DTX1030-wais.pdf (MS-SUPP1205 001 part 13).46DTX1031-c.pdf (MS-SUPP1205 002 part 1).47DTX1031-hmml.pdf (MS-SUPP1205 002 part 2).48DTX1031-html.pdf (MS-SUPP1205 002 part 3).49DTX1031-images.pdf (MS-SUPP1205 002 part 4).50DTX1031-make.pdf (MS-SUPP1205 002 part 5).51DTX1031-other.pdf (MS-SUPP1205 002 part 6).52DTX1031-shell.pdf (MS-SUPP1205 002 part 7).53DTX1031-v.pdf (MS-SUPP1205 002 part 8).54DTX1031-wais.pdf (MS-SUPP1205 002 part 9).55E 021700: Information Regarding WebRouser (CD).56 *Engelbart, D., Knowledge-Domain Interoperability and an Open Hyperdocument System, Proc. of the 1990 ACM Conference on Computer-Supported Cooperative Work, ACM Press, pp. 143-156, 1990.57 *Feiner, S., et al., An Experimental System for Creating and Presenting Interactive Graphical Documents, ACM Transactions on Graphics, vol. 1, No. 1, pp. 59-77, Jan. 1982.58Ferrara, F., "The KIM Query System," Abstract, SIGCHI Bulletin, vol. 6, No. 3, Jul. 1994, pp. 30-39.59File Tree Printout of Defendants Trial Exhibit 215: Information Regarding Emacs.60File Tree Printout of Defendant's Trial Exhibit 258: Information Regarding Microsoft OLE 2.0 Toolkit Program.61File Tree Printout of Defendant's Trial Exhibit 273: Information Regarding Microsoft OLE 2.01 SDK.62File Tree Printout of Defendant's Trial Exhibit 326: Information Regarding Multimedia Viewer.63File Tree Printout of E 021700: Information Regarding WebRouser.64File Tree Printout of MS-ET 0166172: Information Regarding Microsoft OLE 2.01 SDK.65File Tree Printout of MS-ET 0189860: First Companion Disk for "Inside OLE 2" by Kraig Brockschmidt.66File Tree Printout of MS-ET 0189861: Second Companion Disk for "Inside OLE 2" by Kraig Brockschmidt.67Fogarty, K., et al., "Microsoft's OLE Can Be Network Trojan Horse," Network World Magazine, Jun. 27, 1994, vol. 11, No. 26, pp. 1 and 75.68 *Fresco Frequently Asked Questions, www.i.h.kyoto-u.ac.jp/-shom/doc.org/fresco/FAQ.html, pp. 1-4, Apr. 13, 1995.69 *Garg, P., et al., A Hypertext System to Manage Life Cycle Documents, Proc. of the 25th Annual Hawaii Int'l Conf. on System Sciences, 1988, IEEE, vol. 2, pp. 337-346, Jan. 1988.70 *Garg, P., et al., A Hypertext System to Manage Life Cycle Documents, Proc. of the 25th Annual Hawaii Int'l. Conf. on System Sciences, 1988, IEEE, vol. 2, pp. 337-346, Jan. 1988.71Gibbs, S., "Composite Multimedia and Active Objects," OOPSLA '91, pp. 97-112.72 *Halasz, F., Reflections on NoteCards: Seven Issues for the Next Generation of Hypermedia Systems, ACM Journal on Computer Documentation, vol. 25, No. 3, pp. 71-87, Aug. 2001, reprinting article published in 1988.73Hansen, Wilfred, "Andrew as a Multiparadigm Environment for Visual Languages," Visual Languages, 1993 IEEE Symposium, pp. 256-260.74Hansen, Wilfred, "Enhancing Documents with Embedded Programs: How Ness Extends Insets in the Andrew ToolKit," IEEE Computer Languages, 1990 International Conference.75Huynh, Duong Le, et al., "PIX: An Object-Oriented Network Graphics Environment," Comput. & Graphics, vol. 17, No. 3, 1993, pp. 295-304.76John Bradley, "xv, Interactive Image Display for the X Window System," 105 pages (Apr. 26, 1993). (Downloaded from "Index of/pub/net/infosys/NCSA/Web/Mosaic/Unix/view" as part of the file "xv-3.00.tar.z"; Exhibit J has 110 pages, including printouts for: (1) the web page for downloading "xv-3.00.tar.z," (2) the contents of "xv-3.00.tar.z." and (3) the file "xvdocs.ps.z" in "xv-3.00.tar.z.").77John Bradley, "xv, Interactive Image Display for the X Window System," 72 pages (1992). (Downloaded from "Index of /root/usr.local.src/xv-2.21" as part of the file "xv-2.21.tar.z"; Exhibit I has 77 pages, including printouts for: (1) the web page for downloading "xv-2.21.tar.z," (2) the contents of "xv-2.21.tar.z," and (3) the file "xvdocs.ps. z" in "xv-2.21.tar.z.").78Kirste, T., "Spacepicture-An Interactive Hypermedia Satellite Image Archival System," Comput. & Graphics, vol. 17, No. 3, 1993, pp. 251-260.79Labriola, D., "Remote Possibilities," PC Magazine, Jun. 14, 1994, pp. 223-228.80Lashkari, Y.Z., et al., "PLX: A Proposal to Implement a General Broadcasting Facility in a Distributed Environment Running X Windows," Comput. & Graphics, vol. 16, No. 2, 1992, pp. 143-149.81Le Hunte, Stephen, "<EEMBED>-Embedded Objects," HTML Reference Library-HTMLib V2.1, 1995: n. pag. Online.Internet.82 *Lin, J., MediaMosaic-A Multimedia Editing Environment, Proc. of the 5th Annual ACM Symposium on User Interface Software and Technology, ACM Press, pp. 135-141, 1992.83Marc Andreessen, "NCSA Mosaic for X 2.0 prerelease 4 available," www-talk email list, 4 pages (Sep. 29, 1993).84 *Meyrowitz, N., Intermedia: The Architecture and Construction of an Object-Oriented Hypermedia System and Applications Framework, Proc. of the Conf. on Object Oriented Programming Systems, Languages, and Applications, ACM Press, pp. 186- 201, 1986.85Moran, Patrick "Tele-Nicer-Slicer-Dicer: A New Tool for the Visualization of Large Volumetric Data," NCSA Technical Report (TR014), Aug. 1993.86MS Supp 1205-001 (Viola related material) [Compact Disc].87MS Supp 1205-001 File Tree Printout.88MS Supp 1205-002 (Viola related material) [Compact Disc].89MS Supp 1205-002 File Tree Printout.90MS-ET 0009786 (violaTOGO.tar.Z) [Compact Disc].91MS-ET 0009786 File Tree Printout.92MS-ET 0153301 (information regarding Viola similar to PW 1130) [Compact Disc].93MS-ET 0153301 File Tree Printout.94MS-ET 0166172: Information Regarding Microsoft OLE 2.01 SDK (CD).95MS-ET 0189860: First Companion Disk for "Inside OLE 2" by Kraig Brockschmidt (CD).96MS-ET 0189861: Second Companion Disk for "Inside OLE 2" by Kraig Brockschmidt (CD).97MS-ET 9706 (information regarding Viola similar to PW 1130) [Compact Disc].98MS-ET 9706 File Tree Printout.99 *Neuendorffer, T., ADEW: A Multimedia Interface Builder for Andrew, Proceedings Multi-Media Communications, Applications, and Technology Workshop, pp. 1-19, Jul. 1991.100 *Palay, A., Andrew Toolkit: An Overview, Tech Rept., Carnegie-Mellon University Information Technology Center, pp. 1-15, 198.101Printout of readable contents of Defendant's Trial Exhibit 34.102Printout of readable contents of Defendant's Trial Exhibit 37.103PW 1130 (Viola related material similar to Defendant's Trial Exhibit 34 but with a different file structure) [Compact Disc].104PW 1130 File Tree Printout.105 *Reichard, K., et al., X11R6: the Rumored Changes (Release 6 of the X Window System), Unix Review, vol. 11, No. 5, p. 101(5 pp. 1-4 as printed, May 1993.106Rizzo, J., "What's OpenDoc?" MacUser Magazine, Apr. 1994, pp. 119-123.107Sarna, D.E., et al., "OLE Gains Without (Much) Pain," Datamation Magazine, Jun. 15, 1994, pp. 31 and 113.108Shackelford, D.E., et al., "The Architecture and Implementation of a Distributed Hypermedia Storage System," Hypertext '93 Proceedings, Nov. 1993, pp. 1-13.109 *Solaris OpenWindows: Introduction to the ToolTalk Service-A White Paper, Sun Microsystems, Inc., pp. 1-16, 1991.110Tani, M., et al., "Object-Oriented Video: Interaction with Real-World Objects Through Live Video," May 2002, pp. 593-598.111 *The Andrew View, Carnegie Mellon Univ., www-2.cs.cmu.edu/People/AUIS/ftp/NEWSLETTERS/ASCII/93Jun.ascii, vol. 2, No. pp. 1-12 as printed, Jun. 1993.112 *The X Window System And Broadway, www.broadwayinfo.com/bwwhitesbroadwayhct.htm, Hummingbird Communications Ltd pp. 1-11, 1997.113Timothy Thiesen, "Ghostview(1) Unix Programmer's Manual," 14 pages (Jul. 1993). (Downloaded from "ftp://mirror.cs.wesc.edu/pub/mirrors/ghost/gnu/ghostview/" as part of the file "ghostview-1.5.tar.gz"; Exhibit L has 21 pages, including printouts for: (1) the FTP site for downloading "ghostview-1.5.tar.gz," (2) the contents of "ghostview-1.5.tar.gz," and (3) the file "ghostview.ps" in "ghostview-1.5.tar.gz," and (4) the file "README" in "ghostview-1.5.tar.gz.").114 *Tool Inter-Operability: A Hands On Demonstration-A Simple Demonstration of How the ToolTalk Service Works, Sun Microsystems, Inc., pp. 1-24, 1992.115 *Toye, G., et al., SHARE : A Methodology and Environment for Collaborative Product Development, Proceedings, Second Workshop on Enabling Technologies: Infrastructure for Collaborative Enterprises, 1993, IEEE, pp. 33-47, Apr. 22, 1993.116Udell, J., "Visual Basic Custom Controls Meet OLE," Byte Magazine, Mar. 1994, pp. 197-200.117University of Southern California's Mercury Project-"USC Mercury Project:Interface: Project Milestones," USC press release, obtained from Internet, http://www.usc.edu/dept/raiders.118Vaziri, A., "Scientific Visualization in High-Speed Network Environments," Computer Networks and ISDN Systems 22, 1991, pp. 111-129.119Vetter, Ronald, "Mosaic and the World-Wide Web," Computer Magazine, vol. 27, Iss. 10, pp. 49-57.120 *Wiil, U., Issues in the Design of EHTS: A Multiuser Hypertext System for Collaboration, Proc. of the 25th Hawaii Int'l. Conf. on Systems Sciences, vol. 2, pp. 629-639, Jan. 1992.121Wynne et al., "Lean Management, Group Support Systems, and Hypermedia: a Combination Whose Time Has Come," System Sciences, 1993 Annual Hawaii Int'l. Conf., pp. 112-121.* Cited by examinerReferenced byCiting PatentFiling datePublication dateApplicantTitleUS8082293 *Nov 2, 2006Dec 20, 2011The Regents Of The University Of CaliforniaDistributed hypermedia method and system for automatically invoking external application providing interaction and display of embedded objects within a hypermedia documentUS8086662 *Oct 26, 2006Dec 27, 2011The Regents Of The University Of CaliforniaDistributed hypermedia method and system for automatically invoking external application providing interaction and display of embedded objects within a hypermedia documentUS8090793 *Mar 25, 2008Jan 3, 2012Citrix Systems, Inc.Methods and apparatus for making a hypermedium interactiveUS8285782Sep 23, 2011Oct 9, 2012Citrix Systems, Inc.Methods and apparatus for making a hypermedium interactiveUS8316450 *Dec 20, 2007Nov 20, 2012Addn Click, Inc.System for inserting/overlaying markers, data packets and objects relative to viewable content and enabling live social networking, N-dimensional virtual environments and/or other value derivable from the contentUS8626951 *Apr 23, 2007Jan 7, 20144Dk Technologies, Inc.Interoperability of network applications in a communications environmentUS8732318Apr 16, 2003May 20, 2014Mymail, Ltd.Method of connecting a user to a networkUS9021070Jun 20, 2013Apr 28, 2015Mymail, Ltd.Dynamically modifying a toolbarUS9081632Apr 21, 2011Jul 14, 2015Lexmark International Technology SaCollaboration methods for non-programmatic integration systemsUS9137240Apr 26, 2013Sep 15, 2015Mymail, Ltd.System and method of accessing a network by way of different service providersUS9141263Sep 10, 2012Sep 22, 2015Thomas Drennan SelgasMethod of modifying a toolbarUS9183202 *Apr 21, 2011Nov 10, 2015Lexmark International Technology, SANotification methods for non-programmatic integration systemsUS9294335 *Apr 9, 2012Mar 22, 2016Empire Technology Development LlcProcessing load distributionUS9336377Apr 21, 2011May 10, 2016Lexmark International Technology SarlSynchronized sign-on methods for non-programmatic integration systemsUS9477665Nov 16, 2012Oct 25, 2016Virtual Creative Artists, LLCRevenue-generating electronic multi-media exchange and process of operating sameUS9501480Jun 18, 2014Nov 22, 2016Virtual Creative Artists, LLCRevenue-generating electronic multi-media exchange and process of operating sameUS9524322Sep 27, 2013Dec 20, 2016Oracle International CorporationComposite active reportsUS20070038725 *Feb 19, 2004Feb 15, 2007Jeong-Soo KwakMethod and equipment of asp service for multi-resolution image on the environment of multi-server.US20080263187 *Apr 23, 2007Oct 23, 20084Dk Technologies, Inc.Interoperability of Network Applications in a Communications EnvironmentUS20080263464 *Mar 25, 2008Oct 23, 2008Jeff MuirMethods and apparatus for making a hypermedium interactiveUS20090165140 *Dec 20, 2007Jun 25, 2009Addnclick, Inc.System for inserting/overlaying markers, data packets and objects relative to viewable content and enabling live social networking, n-dimensional virtual environments and/or other value derivable from the contentUS20090254998 *Mar 27, 2009Oct 8, 2009Plura Processing, LpWeb-browser based grid computing systemUS20120060085 *Nov 14, 2011Mar 8, 2012Vr Photo Co., Ltd.Method and equipment of asp service for multi-resolution image on the environment of multi-serverUS20120101981 *Apr 21, 2011Apr 26, 2012Randall ArmsNotification methods for non-programmatic integration systemsUS20130268573 *Apr 9, 2012Oct 10, 2013Empire Technology Development LlcProcessing load distributionEP2738673A1Dec 3, 2012Jun 4, 2014Politechnika PoznanskaMethod and system for self-configuration of a system* Cited by examinerClassifications U.S. Classification709/202, 718/106, 345/653, 709/219, 709/218, 345/419, 715/804, 719/315, 715/760, 345/638, 715/738, 715/205, 345/619, 345/654, 345/656, 345/655, 345/427, 719/310, 715/777, 345/649, 707/E17.119International ClassificationG06F9/46, G06F17/30, G06F9/50Cooperative ClassificationG06F9/5027, G06F17/30899, G06F9/5055, G06F2209/509, G06F2209/541, G06F2209/549, G06F9/547European ClassificationG06F9/54P, G06F9/50A6S, G06F17/30W9, G06F9/50A6Legal EventsDateCodeEventDescriptionApr 5, 2011CCCertificate of correctionMay 3, 2011CCCertificate of correctionApr 8, 2013FPAYFee paymentYear of fee payment: 4Dec 1, 2015ASAssignmentOwner name: EOLAS TECHNOLOGIES INC., TEXASFree format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:THE REGENTS OF THE UNIVERSITY OF CALIFORNIA;REEL/FRAME:037176/0601Effective date: 20151014RotateOriginal ImageGoogle Home - Sitemap - USPTO Bulk Downloads - Privacy Policy - Terms of Service - About Google Patents - Send FeedbackData provided by IFI CLAIMS Patent Services