Source: https://patents.google.com/patent/US6388680B1/en
Timestamp: 2019-09-15 17:04:23
Document Index: 142866919

Matched Legal Cases: ['Application No. 60', 'Application No. 60', 'art 2', 'art 2', 'art 2', 'art 2', 'art 2', 'art 2', 'art 2', 'art 2']

US6388680B1 - Multi-user extension mechanisms for client-server system - Google Patents
Multi-user extension mechanisms for client-server system Download PDF
US6388680B1
US6388680B1 US09/176,101 US17610198A US6388680B1 US 6388680 B1 US6388680 B1 US 6388680B1 US 17610198 A US17610198 A US 17610198A US 6388680 B1 US6388680 B1 US 6388680B1
US09/176,101
1998-06-29 Priority to US9112298P priority Critical
1998-10-21 Application filed by Sony Corp, Sony Electronics Inc filed Critical Sony Corp
1998-10-21 Priority to US09/176,101 priority patent/US6388680B1/en
1999-05-21 Assigned to SONY CORPORATION, SONY ELECTRONICS, INC. reassignment SONY CORPORATION ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: BAYRAKERI, SADIK
2002-05-14 Publication of US6388680B1 publication Critical patent/US6388680B1/en
2015-11-13 Assigned to SNAPTRACK, INC. reassignment SNAPTRACK, INC. ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: SONY CORPORATION
2018-03-09 Assigned to SONY CORPORATION reassignment SONY CORPORATION ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: SONY ELECTRONICS INC.
230000014509 gene expression Effects 0 Claims Description 28
Multi-user communication of multimedia content in a client-server system is provided by dividing a client's scene into non-shared and shared parts. In one embodiment, the non-shared parts of the scene are organized into one or more MPEG-4 Group nodes. The shared parts of the scene are organized into one or more multi-user group nodes, each of which is an extension of an MPEG-4 Group node. In addition to the functionality provided by an MPEG-4 Group node, a multi-user group node provides boolean exposed fields that act as switches to control initiation of interaction, update control, privacy, and object accessibility.
Provisional U.S. Patent Application No. 60/091,122 Expired, entitled “Multi-User Interaction Protocol Using Java and MPEG-4,” filed on Jun. 29, 1998, by Sadik Bayrakeri;
U.S. Patent Application Ser. No, 09/176,100, entitled “MULTI-USER INTERACTION FOR MULTIMEDIA COMMUNICATION,” filed on Oct. 16,1998 by Sadik Bayrakeri, and
U.S. patent application Ser. No. 09/176,100, entitled “METHOD AND APPARATUS FOR IMPLEMENTING MULTI-USER GROUPING NODES IN A MULTIMEDIA PLAYER,” and filed on the same date herewith by Tao Liang, Sadik Bayrakeri, and Jing Swales.
The disclosures of these applications are each incorporated herein by reference in their entirety. In addition, the benefit of priority of Provisional U.S. Patent Application No. 60/091122 is hereby claimed under 35 U.S.C. 119(e).
This invention relates generally to multi-user communication, and, more particularly, to multi-user extension mechanisms for a client-server system.
The Moving Picture Experts Group (MPEG) is a working group of the International Organization for Standardization (ISO) that generates standards for digital video and audio compression. MPEG-1 and MPEG-2 are standards for interactive video on CD-ROM and Digital Television, respectively. MPEG4 is a newly emerging standard being designed for efficient multimedia delivery and integration between a client and a server. MPEG-4 provides the flexibility to allow an end user to interact with multimedia content (e.g., audio, video, text, or any combination thereof) and to control the way that audiovisual objects are displayed. A more complete description of MPEG-4 is contained in the document ISO/IEC JTC1/SC29/WG11, N2323, published July, 1998, by the ISO.
MPEG-4 incorporates aspects of the well known Virtual Reality Modeling Language (VRML), which describe three dimensional objects and worlds using a hierarchical scene graph (or scene description). Entities contained within the scene graph are called nodes. [The terms “scene graph” and “scene description” are used interchangeably herein. VRML is described more fully in various texts, such as “The Annotated VRML 2.0 Reference Manual,” by Rikk Carey and Gavin Bell, Addison-Wesley Developers Press (1997)]. VRML 2.0 defines 54 different node types, including geometry primitives, appearance properties, sound properties, and various types of grouping nodes. Nodes store their data in fields, and VRML defines 20 different types of fields. Furthermore, nodes can contain other nodes and may be contained in more than one node. Analogously, MPEG-4 employs an MPEG-4 scene description and MPEG-4 nodes (sometimes called Binary Format for Scene (BIFS) nodes).
Two of the node types defined in MPEG-4 and VRML 2.0 are the Group and the Group2D nodes. Both the Group and Group2D nodes are mechanisms by which one associates different media objects into a scene description. One limitation of MPEG-4 and VRML is that these Group and Group2D nodes are designed for a single end user, and therefore do not provide mechanisms for multi-user interaction. Multi-user interaction, however, is important for application domains such as collaborative computing, distance learning, shared virtual worlds, virtual chat, entertainment, and e-commerce, each of which involve interaction of two or more users with each other. There exists, therefore, a need for multi-user extension mechanisms for a client-server system.
Methods and apparatus consistent with the present invention, as embodied and broadly described herein, provide multi-user communication in a client-server system.
Consistent with the invention, a method for providing multi-user communication in a client-server system includes dividing a scene description at a first client into shared parts and non-shared parts and organizing the shared parts of the scene description into at least one multi-user group node.
Consistent with the invention, an apparatus for providing multi-user communication in a client-server system includes at least one memory having program instructions, and at least one processor configured to execute the program instructions. The processor executes the program instructions to perform the operations of providing a MultiUser Group node. The MultiUser Group node includes the following: a boolean expression field to control whether changes to the state of the group node at the first client should be communicated to a second client; a boolean expression field to control whether changes to the state of the group node should be implemented in a second client; a boolean expression field to control authorization to modify the group node by a second client; and a boolean expression field to control access to the group node by a second client.
Consistent with the invention, a computer-readable medium includes instructions for controlling a computer system to perform a method for providing multi-user communication in a client-server system . The method includes dividing a scene description at a first client into shared parts and non-shared parts and organizing the shared parts of the scene description into at least one multi-user group node.
FIG. 2 is a flow diagram of a method for an initiator control mechanism consistent with the invention;
FIG. 3 is a flow diagram of a method for an update control mechanism consistent with the invention;
FIG. 4 is a flow diagram of a method for a privacy control mechanism consistent with the invention; and
FIG. 5 is a flow diagram of a method for an accessibility control mechanism consistent with the invention.
Methods and apparatus consistent with the invention provide multi-user communication of multimedia content by dividing a client's scene into non-shared and shared parts. In one embodiment, the non-shared parts of the scene are organized into one or more single-user group nodes, with which other clients cannot interact. [As used herein, the term “single-user group node” refers to a node that does not support multi-user communication, such as an MPEG-4 Group (or Group2D) node or a VRML Group (or Group2D) node. The term “multi-user group node” refers to an extended single-user group node, consistent with the invention. Although the description contained herein is based on nodes, those skilled in the art will recognize that other object based scene descriptions can be used consistent with the invention.]
FIG. 1 is a block diagram of a computer system 100 in which methods and apparatus consistent with the invention can be implemented. System 100 comprises a computer 110 connected to a server 180 via a network 170. Network 170 can be, for example, a local area network (LAN), a wide area network (WAN), or the Internet. System 100 is suitable for use with the Java™ programming language, although one skilled in the art will recognize that methods and apparatus consistent with the invention can be applied to other suitable user environments.
Computer 110 comprises several components that are all interconnected via a system bus 120. Bus 120 can be, for example, a bi-directional system bus that connects the components of computer 110, and contains thirty-two address lines for addressing a memory 125 and a thirty-two bit data bus for transferring data among the components. Alternatively, multiplex data/address lines can be used instead of separate data and address lines. Computer 110 communicates with other users' computers on network 170 via a network interface 145, examples of which include Ethernet or dial-up telephone connections.
Computer 110 contains a processor 115 connected to a memory 125. Processor 115 can be a microprocessor manufactured by Motorola, such as the 680X0 processor, a processor manufactured by Intel, such as the 80X86 or Pentium processors, or a SPARC™ microprocessor from Sun Microsystems, Inc. However, any other suitable microprocessor or micro-, mini-, or mainframe computer, can be used. Memory 125 can include a RAM, a ROM, a video memory, or mass storage. The mass storage can include both fixed and removable media (e.g., magnetic, optical, or magnetic optical storage systems or other available mass storage technology). Memory 125 can include a program, an application programming interface (API), and a virtual machine (VM) that contains instructions for handling constraints, consistent with the invention.
A user typically provides information to computer 110 via a keyboard 130 and a pointing device 135, although other input devices can be used. In return, information is conveyed to the user via display screen 140.
In one embodiment, methods and apparatus consistent with the invention employ an extension of the following two single-user group nodes (which are described in more detail in “The Annotated VRML 2.0 Reference Manual,” by Rikk Carey and Gavin Bell, Addison-Wesley Developers Press (1997)):
eventIn MFNode addChildren NULL
eventIn MFNode removeChildren NULL
exposedField MFNode children NULL
field SFVec2f bboxCenter 0,0,0
field SFVec2f bboxSize −1,−1,−1
Group2D {
field SFVec2f bboxCenter 0,0
field SFVec2f bboxSize −1,−1
In each of these single-user group nodes, the fields identified above perform the following functions:
addChildren eventln: This field specifies a list of objects that must be added to the Group node.
removeChildren eventln: This field specifies a list of objects that must be removed from the Group node.
children: This field contains the current list of objects contained in the Group node
bboxCenter: This field specifies the center of the bounding box.
bboxSize: This field specifies the dimensions (in length, width, and/or height) of the bounding box. [It is possible not to transmit the bboxCenter and bboxSize fields, but if they are transmitted they should contain all children of the group node.]
If more than one user desires to interact with a single scene, however, these single-user group nodes are insufficient because they provide no mechanism for concurrency control or access control. These deficiencies can be overcome, consistent with the invention, by creating two new types of group nodes. Any client may create a group node of either of these two types and transmit a copy (“replica”) of the new type group node to other clients. The new type group nodes can have control mechanisms for supporting multi-user communication, as shown by the following examples:
MultiUserGroup {
exposedField SFBool isInitiator TRUE
exposedField SFBool isActive TRUE
exposedField SFBool isPrivate TRUE
exposedField SFBool isAccessible TRUE
MultiUserGroup2D {
In each of these group nodes, the fields identified above perform the following functions:
addChildren eventln: This field performs the same function as in a single-user group node.
removeChildren eventln: This field performs the same function as in a single-user group node.
children: This field performs the same function as in a single-user group node.
isInitiator: This field indicates whether a client is the initiator for the group node. If this field is set to FALSE, any state changes made to the group node by the client in which it resides need not be transmitted to other clients. If this field is set to TRUE, the group node on this client is an initiator; accordingly, any state changes to its state must be transmitted to other clients. It is possible to set the isInitiator field to TRUE in more than one client. In such cases, the application program shall handle the distribution of state changes from multiple initiators to replicas.
isActive: This field indicates whether a client should implement any state changes for the node which are received from other clients, and also whether state changes for the node occurring on this client should be transmitted to other clients. If this field is set to TRUE in a client node, any node state changes received by the client from other clients will be implemented in the node residing on this client. If this field is set to FALSE, any received state changes will not be implemented. Furthermore, if this field is set to FALSE and the isInitiator field is set to TRUE, any state changes for the node occurring on this client are not transmitted to other clients.
isPrivate: This field indicates who is allowed to modify the group node. If this field is set to TRUE, only the owner of the group node may modify the group node's content. If this field is set to FALSE, other users can also modify the group node's content.
isAccessible: This field indicates who is allowed to access the group node. If this field is set to FALSE, only the owner (i.e., the creator client) of the group node can access/read the group node. If this field is set to TRUE, other clients can also access the group node's content.
bboxCenter: This field performs the same function as in a single-user group node.
bboxSize: This field performs the same function as in a single-user group node.
FIG. 2 is a flow diagram of a method for an initiator control mechanism consistent with the invention. In one embodiment, a multimedia scene is created on a user's client terminal using VRML or MPEG-4. At stage 205, the multimedia scene is divided into shared and non-shared parts. This can be accomplished, for example, by using a BIFS mechanism. The shared part of a client's scene includes MultiUseGroup nodes with which multiple user clients can interact. [Although the description for FIGS. 2-5 uses the terminology described above in reference to MultiUserGroup, those skilled in the art will appreciate that the names are explanatory only and that other multi-user group nodes or objects may be used consistent with the invention.] The non-shared part of the client's scene contains single-user group nodes that belong only to that client The client that creates the nodes may be known as the “owner” of the nodes.
At stage 210, the state of a MultiUserGroup node on a client changes. At stage 215, it is determined whether the changes should be transmitted to replicas of the MultiUserGroup node which exist on other clients. This can be accomplished by using the isInitiator boolean field, as described above. If the client is not an initiator for the MultiUserGroup node (e.g., the isInitiator field is FALSE), any state changes made at that client to the node present at that client (either original node or replica node) are not transmitted to other clients at stage 220. If the client is an initiator for the MultiUserGroup node (e.g., the isInitiator field is TRUE), then the state changes are sent to other clients at stage 225.
FIG. 3 is a flow diagram of a method for an update control mechanism consistent with the invention. In one embodiment, a scene is divided into shared and non-shared parts at stage 305 as described above in reference to FIG. 2. At stage 310, the state of a MultiUserGroup node changes on a client. At stage 315, it is determined whether the state change of the MultiUserGroup node should be transmitted to other clients and whether such other clients will implement a received state change on their node. This can be accomplished by using (in addition to the isInitiator field) the isActive boolean field, as described above. If the MultiUserGroup node is not active for certain clients (e.g., the isActive field is FALSE), those clients will not implement any received state changes for that node at stage 320. If the MultiUserGroup node is active for certain clients (e.g., the isActive field is TRUE), state changes received by those clients will be implemented at stage 325.
FIG. 4 is a flow diagram of a method for a privacy control mechanism consistent with the invention. In one embodiment, a scene is divided into shared and non-shared parts at stage 405 as described above in reference to FIG. 2. At stage 410, a client other than the owner of the MultiUserGroup node attempts to modify the MultiUserGroup node. At stage 415, it is determined whether the non-owner should be permitted to modify the MultiUserGroup node. This can be accomplished by using the isPrivate boolean field, as described above. If other clients are permitted to modify the MultiUserGroup node (e.g., the isPrivate field is FALSE), then the non-owner is permitted to modify the content of the MultiUserGroup node at stage 420. If only the owner may modify the MultiUserGroup node (e.g., the isPrivate field is TRUE), then the non-owner is not permitted to modify (i.e., is precluded from modifying) the content of the MultiUserGroup node at stage 425.
FIG. 5 is a flow diagram of a method for an accessibility control mechanism consistent with the invention. In a preferred implementation, a scene is divided into shared and non-shared parts at stage 505 as described above in reference to FIG. 2. At stage 510, a client other than the owner of the MultiUserGroup node attempts to access/read the MultiUserGroup node. At stage 515, it is determined whether the non-owner should be permitted to access/read the MultiUserGroup node. This can be accomplished by using the isAccessible boolean field, as described above. If only the owner of the MultiUserGroup can access it (e.g., the isAccessible field is FALSE), then the non-owner is not permitted to access/read the content of the MultiUserGroup node at stage 520. If other clients are permitted to access/read the MultiUserGroup node (e.g., the isAccessible field is TRUE), then the non-owner is permitted to access/read the content of the MultiUserGroup node at stage 525.
As described in detail above, methods and apparatus consistent with the invention provide multi-user communication in a client-server system. The foregoing description of an implementation of the invention has been presented for purposes of illustration and description. Modifications and variations are possible in light of the above teachings or may be acquired from practicing the invention.
For example, the foregoing description is based on extensions to MPEG-4 or VRML scene descriptions and nodes, but those skilled in the art will recognize that other programming environments (e.g., object based scene descriptions) can be used consistent with the invention. Furthermore, the foregoing description is based on a client-server architecture, but those skilled in the art will recognize that a peer-to-peer architecture may be used consistent with the invention. Moreover, although the described implementation includes software, the invention may be implemented as a combination of hardware and software or in hardware alone. Additionally, although aspects of the present invention are described as being stored in memory, one skilled in the art will appreciate that these aspects can also be stored on other types of computer-readable media, such as secondary storage devices, like hard disks, floppy disks, or CD-ROM; a carrier wave from the Internet; or other forms of RAM or ROM. The scope of the invention is therefore defined by the claims and their equivalents.
1. A method for providing multi-user communication in a client-server system having multiple clients, comprising:
dividing a scene description displayed at a first client into shared parts and non-shared parts;
organizing the shared parts of the scene description into at least one multi-user group node;
wherein the operation of organizing the shared parts of the scene description into at least one multi-user group node includes:
creating a multi-user group node at the first client by adding control mechanisms to a group node, and
adding the shared parts of the scene description to the multi-user group node; and
wherein the creating operation comprises adding to the single-user group node a field to control whether changes to the state of the node at the first client should be communicated to a second client.
2. The method of claim 1, wherein the operation of adding to the group node a field to control whether changes to the state of the group node at the first client should be communicated to a second client comprises adding to an MPEG-4 Group node a boolean expression field to control whether changes to the state of the group node at the first client should be communicated to a second client.
3. The method of claim 1, wherein the operation of adding to the group node a field to control whether changes to the state of the group node at the first client should be communicated to a second client comprises adding to an MPEG-4 Group2D node a boolean expression field to control whether changes to the state of the group node at the first client should be communicated to a second client.
4. A method for providing multi-user communication in a client-server system having multiple clients, comprising:
organizing the shared parts of the scene description into shared parts and non-shared parts; and
permitting a second client to interact with the shared parts of the scene description;
wherein the operation for organizing the shared parts of the scene description into a multi-user group node includes:
wherein the creating operation comprises adding to the group node a field to control whether changes to the state of the group node should be implemented in a second client.
5. The method of claim 4, wherein the operation of adding to the group node a field to control whether changes to the state of the group node should be implemented in a second client comprises adding to an MPEG-4 Group node a boolean expression field to control whether changes to the state of the group node should be implemented in a second client.
6. The method of claim 4, wherein the operation of adding to the group node a field to control whether changes to the state of the group node should be implemented in a second client comprises adding to an MPEG-4 Group2D node a boolean expression field to control whether changes to the state of the group node should be implemented in a second client.
7. A method for providing multi-user communication in a client-server system having multiple clients, comprising:
providing a MultiUser Group node comprising:
a boolean expression field to control whether changes to the state of the group node at the first client should be communicated to a second client;
a boolean expression field to control whether changes to the state of the group node should be implemented in a second client;
a boolean expression field to control authorization to modify the group node by a second client; and
a boolean expression field to control access to the group node by a second client.
an addChildren field;
a removeChildren field;
a children field;
a bboxCenter field;
a bboxSize field.
9. An apparatus for providing multi-user communication in a client-server system, comprising:
10. A computer-readable medium containing instructions for controlling a computer system to perform a method for providing multi-user communication in a client-server system, the method comprising:
a boolean expression field to control whether changes to the state of the group node at the first client should be communicated to a second client.
11. A computer-readable medium containing a data structure for providing multi-user communication in a client-server system, the data structure comprising:
a MultiUser Group node comprising:
a boolean expression field to control whether changes to the state of the group node at the first client should-be communicated to a second client.
12. An apparatus for providing multi-user communication in a client-server system, comprising:
means for providing a MultiUser Group node comprising:
US09/176,101 1998-06-29 1998-10-21 Multi-user extension mechanisms for client-server system Expired - Lifetime US6388680B1 (en)
US9112298P true 1998-06-29 1998-06-29
US09/176,101 US6388680B1 (en) 1998-06-29 1998-10-21 Multi-user extension mechanisms for client-server system
US6388680B1 true US6388680B1 (en) 2002-05-14
ID=26783617
US09/176,101 Expired - Lifetime US6388680B1 (en) 1998-06-29 1998-10-21 Multi-user extension mechanisms for client-server system
US (1) US6388680B1 (en)
US20020033842A1 (en) * 2000-09-15 2002-03-21 International Business Machines Corporation System and method of processing MPEG streams for storyboard and rights metadata insertion
1998-10-21 US US09/176,101 patent/US6388680B1/en not_active Expired - Lifetime
"Living Worlds," printed from Internet web-site www.livingworlds.com on Jan. 19, 1998.
Carey, R., and Bell, G., "The Annotated VRML 2.0 Reference Manual," Addison-Wesley Developers Press, 1997, title page, reference/disclosure page, and pp. 183-184.
Document: V{umlaut over (aa)}nänen, R., Verification Model of Advanced BIFS (Systems VM subpart 2), ISO/IEC JTC1/SC29/WG11 MPEG98/N2359 subpart 2, Jul. 1998.
Document: Väänänen, R., Verification Model of Advanced BIFS (Systems VM subpart 2), ISO/IEC JTC1/SC29/WG11 MPEG98/N2359 subpart 2, Jul. 1998.
Eleftheriadis, et al., Text for CD 14496-1 Systems, ISO/IEC JTC1/SC29/WG11 N1901, Nov. 21, 1997.
Koenen, R., MPEG-4 Overview-(Dublin Version), ISO/IEC JTC1/SC29/WG11 N2323, Jul. 1998, Dublin.
Koenen, R., MPEG-4 Overview—(Dublin Version), ISO/IEC JTC1/SC29/WG11 N2323, Jul. 1998, Dublin.
V{umlaut over (aa)}nänen, R., Working Draft of Advanced BIFS (Systems WD subpart 2), ISO/IECJTC1/SC29/WG11 MPEG98/N2358 subpart 2, Jul. 1998.
Väänänen, R., Working Draft of Advanced BIFS (Systems WD subpart 2), ISO/IECJTC1/SC29/WG11 MPEG98/N2358 subpart 2, Jul. 1998.
US8655980B2 (en) 2014-02-18 Networked computer system for communicating and operating in a virtual reality environment
US10305948B2 (en) 2019-05-28 Audio-video data switching and viewing system
US9501258B2 (en) 2016-11-22 Method of sharing a desktop with attendees of a real-time collaboration
RU2368940C2 (en) 2009-09-27 Synchronised graphic data and region data for systems of remote handling graphic data
DE69837833T2 (en) 2008-01-31 System and method for generating and interface formation of MPEG-encoded audio visual objects representing bitstreams
CN1109320C (en) 2003-05-21 Animation reuse in three dimensional virtual reality
DE60222890T2 (en) 2008-02-07 Methods and apparatus for Implementerung of highly interactive entertainment services using the media flow technology, which enables the provision of a distance from virtual reality services
Hagsand 1996 Interactive multiuser VEs in the DIVE system
JP4726097B2 (en) 2011-07-20 System and method for coupling a mpeg encoding audio visual objects can perform adaptive control on the interface
US6049805A (en) 2000-04-11 Dynamic event mechanism for objects with associational relationships
US7773093B2 (en) 2010-08-10 Method and apparatus for associating the color of an object with an event
EP0756420A2 (en) 1997-01-29 Information provider apparatus enabling selective playing of multimedia information by interactive input based on displayed hypertext information
US8441475B2 (en) 2013-05-14 Arrangements for enhancing multimedia features in a virtual universe
AU775795B2 (en) 2004-08-12 Application development interface for multi-user applications executable over communication networks
Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:BAYRAKERI, SADIK;REEL/FRAME:009973/0010
Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:SONY ELECTRONICS INC.;REEL/FRAME:045160/0410