Abstract:
Computer method and apparatus persists portions of group conversations in a virtual environment. A group conversation is formed of group member communications. Indications or illustrations, such as chat bubbles, are displayed to end users to represent the group member communications. The invention system enables a user to persist a portion of the group conversation by selecting respective indicators (e.g., chat bubbles) corresponding to one or more of the group member communications. The processor (persist engine) also enables any user to interact with persisted conversation portions. Interactions supported include moving, arranging, tagging with metadata and saving snapshots of the persisted conversation portions, in the virtual environment.

Description:
RELATED APPLICATIONS 
       [0001]    Subject matter of the present invention is related to that of U.S. patent application Ser. No. 10/973,124 (published as US2006/0090137A1) for “Method and Apparatus for Constructing Free-form Threaded On-line Conversations” by assignee and herein incorporated in it&#39;s entirety. 
     
    
     BACKGROUND OF THE INVENTION 
       [0002]    Virtual worlds such as Second Life, There.com, Active Worlds, World of Warcraft, etc. facilitate synchronous communication via avatar-centric chat. Specifically, users teleoperate avatars in a virtual place, and engage in a chatroom-style conversation, where messages appear as bubbles floating above the avatars&#39; heads like in comic strips. Certain instant messaging clients, such as Yahoo Instant Messenger with Yahoo Avatars, support similar interactions—where the chat participants are portrayed as avatars in a single room, and messages also graphically appear as chat bubbles. 
         [0003]    A common problem in these conversation contexts is the desire to persist interesting pieces of conversation for later reference. This could be for a document being created, or for a common activity such as collecting ideas for a brainstorm. There are a variety of solutions that can be classified into the following categories: 
         [0004]    Record Everything+Posthoc Analysis. Most of the above systems directly support, or provide add-ons, to record the entire conversation. The recording is accomplished by either logging all chat messages—or an automatically filtered set of messages—or by recording a screen movie. Afterwards someone has to manually sift through the recording to extract the “interesting” pieces, or a software application does automated analysis for “interesting” highlights, or a semi-automated process is used. As a result, this can be a labor-intensive operation depending on the complexity of the conversation. 
         [0005]    Assign a Scribe. During the conversation, someone volunteers to be a “scribe”—the note-taker. The volunteer then multitasks—he or she may want to participate in the conversation, but also has to take notes. More importantly, this volunteer has to pay enough attention to manually identify “interesting” highlights to record. As a result, this can be a labor-intensive task for the volunteer. 
         [0006]    Active Participation. During the conversation, anyone who wishes to persist an interesting piece of information makes a note. Then the notes are submitted on the spot, or collected afterwards. In a virtual world, users need a specialized authoring tool (e.g. a virtual shared post-it note) to do this. If such a tool is not supported, users need to do this manually with an external application (e.g. text editor). 
         [0007]    A related problem is the desire to create virtual artifacts with textual content, based on a conversation. Such artifacts can be useful as shared media (e.g. a reference document to pass around, a virtual poster, a set of virtual post-it notes on a whiteboard, etc). A common approach involves the user creating the desired artifact with whatever tools are available, and then copy/pasting textual content into the artifact. 
       SUMMARY OF THE INVENTION 
       [0008]    Applicants contend that the process of persisting interesting pieces of conversation can be streamlined into a first-class operation in virtual worlds as well as in avatar-based chat spaces in instant messaging environments. The present invention is akin to the “active participation” approach, but treats all chat bubbles as potential targets for anyone to persist and share as a virtual object. 
         [0009]    In one embodiment, the present invention provides computer method and apparatus for persisting a portion of group conversations in a virtual environment. The method and system comprise: 
         [0010]    providing a group conversation in a virtual environment, the group conversation being formed of a plurality of a group member (participants) communications; 
         [0011]    displaying to a user indications of the group member communications or messages; and 
         [0012]    enabling the user to persist a portion of the group conversation by the user selecting the respective displayed indications of ones of the group member communications, and the user selected ones being persisted. 
         [0013]    Embodiments of the present invention extend the existing notion of bubble-based chats in virtual environments and avatar-based IM (instant messaging) spaces in the following ways: 
         [0014]    Bubbles can be persisted by any user: Any user can select a chat bubble (including another user&#39;s bubble) and choose to persist it (preventing it from disappearing after a time limit). Other graphical illustrations or visual renditions or the like of messages by chat participants (users&#39; avatars) in addition to chat bubbles can be used. 
         [0015]    Persisted Bubbles can be moved by any user: Any user can move a persisted bubble in the virtual environment (e.g. to create groupings for a brainstorming session, to arrange bubbles into a flow diagram, assign bubbles as task objects to people, etc). 
         [0016]    Persisted Bubbles can be tagged by any user: Any user can annotate a persisted bubble with additional metadata which can be textual or graphical (e.g. assign a priority/severity level). 
         [0017]    Persisted Bubbles can be arranged on a map surface: Any user can move persisted bubbles on a graphical shared surface (e.g. the floor) where the bubble&#39;s placement on the surface has a semantic meaning (e.g. the floor has a picture of a global map, a calendar, an org chart). 
         [0018]    Arrangements of Bubbles/Surfaces can be snapshotted: Any user can save the current arrangement of bubbles and map surface for later use (e.g. saving the state of a current brainstorming session, reloading later for a presentation). 
         [0019]    Through these characteristics, the present invention enables users to easily transform casual virtual world (including gaming world environments) group conversation into persistent conversation, and furthermore, turn pieces of conversation into objects that can be manipulated into diagrams, post-it notes, and other physical artifacts found in real-world meetings. 
     
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         [0020]    The foregoing will be apparent from the following more particular description of example embodiments of the invention, as illustrated in the accompanying drawings in which like reference characters refer to the same parts throughout the different views. The drawings are not necessarily to scale, emphasis instead being placed upon illustrating embodiments of the present invention. 
           [0021]      FIG. 1  is a schematic illustration of a screen view in one embodiment of the present invention showing a user persisting a chat bubble (i.e. graphical illustration of communications). 
           [0022]      FIG. 2  is a schematic illustration of a screen view showing another user interacting with the persisted chat bubble of  FIG. 1 . 
           [0023]      FIG. 3  is a schematic illustration of a screen view showing both users of  FIG. 1  and  FIG. 2  interacting with persisted chat bubbles (graphically illustrated communications). 
           [0024]      FIG. 4  is a schematic illustration of a screen view showing a user applying metadata in the embodiment of  FIG. 1  through  FIG. 3 . 
           [0025]      FIG. 5  is a schematic view of a computer network environment in which embodiments of the present invention are implemented. 
           [0026]      FIG. 6  is a block diagram of a computer node in the network of  FIG. 5 . 
           [0027]      FIG. 7  is a flow diagram of one embodiments of the present invention. 
       
    
    
     DETAILED DESCRIPTION OF THE INVENTION 
       [0028]    A description of example embodiments of the invention follows. 
         [0029]    The present invention extends conventional bubble-based chat systems in virtual worlds and avatar-base IM spaces with the following technical mechanisms: 
         [0030]    Bubbles can be persisted by any user: a user interface action/menu option selection (e.g. double-click) allows any user to update the state of the bubble (graphical message indicator/illustration of group member communication) from non-persistent to persistent. In one embodiment this is supported by program objects of class bubble and having a state attribute. The state attribute may toggle between (be set to) “non-persistent” and “persistent”. Other attributes are as described below. 
         [0031]    Persisted Bubbles can be moved by any user: persisted bubbles are treated as conventional virtual world object models that can be moved and manipulated by a user interface action (e.g. drag and drop). 
         [0032]    Persisted Bubbles can be tagged by any user: a user interface action/menu option selection/toolbar function or operation (generally user command) allows any user to apply metadata. In response, the virtual world maintains a database  94  ( FIG. 7 ) or uses an external tagging service to associate metadata to bubble objects. 
         [0033]    Persisted Bubbles can be arranged on a map surface: persisted bubbles are treated as conventional virtual world object models that can be moved and manipulated by a user interface action, and the graphical shared surface is represented as a virtual object with target areas to apply semantics to bubble placements (e.g. trigger scripts can be assigned to target areas on the surface to associate dropped objects to dates on a calendar, people on an org chart, etc). 
         [0034]    Arrangement of Bubbles/Surfaces can be snapshotted: a user interface action instructs the virtual world to save the state of bubbles and surfaces in a given area as specified by the user. The saved information is stored in a file system or database  94  ( FIG. 7 ) that can be later recalled by another user interface action. 
         [0035]      FIGS. 1-4  are several screenshots from one embodiment illustrating the present invention. 
         [0036]    In the screen view  11  of  FIG. 1 , a yellow user  13  persists the chat bubble “Add menu item”  23  by clicking on (selecting) it. 
         [0037]    In  FIG. 2  a red user  15  drags over, moves, or otherwise relocates the persisted bubble  23  (i.e. the subject chat bubble  23  now in a persisted state) to a position in the screen view  11  closer to him  15 . 
         [0038]    In  FIG. 3 , both users  13 ,  15  arrange persisted bubbles  23 ,  25  on top of a graphical surface  19  displaying team members  17   a ,  17   b ,  17   c  to assign tasks to. The graphical surface  19  is a map surface with target areas programmed/configured to apply semantics to bubble placements as discussed above. Other map surfaces, working surfaces and the like are suitable for supporting graphical surface  19 . 
         [0039]    In the screen view  11  of  FIG. 4 , a user applies a question mark  27  (e.g. a user selected sticker in graphical interface  29 ) as metadata on “pose avatar” bubble  21 . This is supported (implemented) by common object data handling and graphical user interface techniques. The user-selected item  27  may be any one or combination of text, graphics, line art, images, multimedia and the like. 
         [0040]    A complete video showing one embodiment of the present invention can be viewed at http://slurl.com/secondlife/IBMCODESTATION/124/159/25/and is incorporated herein by reference. 
         [0041]      FIG. 5  illustrates a computer network or similar digital processing environment in which the present invention may be implemented. 
         [0042]    Client computer(s)/devices  50  and server computer(s)  60  provide processing, storage, and input/output devices executing application programs and the like. Client computer(s)/devices  50  can also be linked through communications network  70  to other computing devices, including other client devices/processes  50  and server computer(s)  60 . Communications network  70  can be part of a remote access network, a global network (e.g., the Internet), a worldwide collection of computers, Local area or Wide area networks, and gateways that currently use respective protocols (TCP/IP, Bluetooth, etc.) to communicate with one another. Other electronic device/computer network architectures are suitable. 
         [0043]    With reference to  FIGS. 5-7 , embodiments of the present invention are implemented as follows.  FIG. 7  outlines the flow of control and/or data in implementing an embodiment of the present invention. The invention system (processor or persist engine)  37  enhances a communication system  73  operating in a given virtual environment (e.g. virtual world, simulation, videogame, 3D gaming and the like). Other on-line sessions and computer network sessions accessible by two or more users for conversing or having group communications are contemplated. 
         [0044]    In particular, the communication system  73  includes a group conversation module or subsystem  75 , such as a chat mode, or instant messaging (IM) unit or other multiple user communication assembly. Preferably, the group conversation module  75  is graphical bubble-based, chat bubble-based or avatar IM based or the like. In invention system  37 , the group conversation module/means is initially generated and supported by common chat and IM communication techniques. In this respect, system  37  displays graphical chat bubbles or other indications (graphical illustrations, etc.) of individual group member (users) communications in a current session of the virtual environment. A running tally or sum list of the communications and messages across all (or the most recent) group members is shown in a window  20  ( FIGS. 1-4 ) and represents the current group conversation. 
         [0045]    Next in accordance with the present invention, chat bubbles  23  and similar indications/graphics illustrating chat participants messages in group conversations are initialized with respective programming objects  33  (step  35 ) or similar data structure. In response to user input (e.g. action or command)  31  to make a chat bubble (graphically illustrated message generally)  23  persistent as described above in  FIG. 1 , processor or persist engine procedure step  41  updates the bubble object  33  state. This effectively persists or saves a piece of a group conversation, i.e., the portion corresponding to and illustrated/represented by the user-selected chat bubble  23 . 
         [0046]    Once the bubble object  33  state is effectively set to “persist” (e.g. Persist=True), the invention system  37  (step  43 ) treats the persisted chat bubble  25  as an object model in the virtual environment screen views. Accordingly, step  43  in response to user interaction enables the persisted bubbles  25  to be interacted with, moved and arranged on a graphical surface  19 , and the like (as described above in  FIGS. 2 and 3 ). Common or known graphical user interactivity (e.g. drag and drop) is employed here. Step  44  enables a user to tag or otherwise associate metadata to the bubble object  33  corresponding to (or manifested as) the user coupling text, graphics, images or multimedia  27  with the illustrated bubble  21 ,  25  as described in  FIG. 4  above. The metadata and corresponding object association data is stored in a file  94  for example using known technology. 
         [0047]    Step  45  of invention system  37  is responsive to user command to save a snapshot of bubbles  21 ,  25  and work surfaces as arranged in the virtual environment. Step  45  accomplishes this by saving to a file or database  94  indications of (defining information regarding) the arrangements and states of bubbles  21 ,  25  and graphical surfaces  19 . Step  45  follows conventional techniques for storing such data in a manner that enables subsequent search and retrieval of the snapshot on user or application/program command (query). 
         [0048]      FIG. 5  illustrates a computer network or similar digital processing environment in which the present invention may be implemented. 
         [0049]    Client computer(s)/devices  50  and server computer(s)  60  provide processing, storage, and input/output devices executing application programs and the like. Client computer(s)/devices  50  can also be linked through communications network  70  to other computing devices, including other client devices/processes  50  and server computer(s)  60 . Communications network  70  can be part of a remote access network, a global network (e.g., the Internet), a worldwide collection of computers, Local area or Wide area networks, and gateways that currently use respective protocols (TCP/IP, Bluetooth, etc.) to communicate with one another. Other electronic device/computer network architectures are suitable. 
         [0050]      FIG. 6  is a diagram of the internal structure of a computer (e.g., client processor/device  50  or server computers  60 ) in the computer system of  FIG. 5 . Each computer  50 ,  60  contains system bus  79 , where a bus is a set of hardware lines used for data transfer among the components of a computer or processing system. Bus  79  is essentially a shared conduit that connects different elements of a computer system (e.g., processor, disk storage, memory, input/output ports, network ports, etc.) that enables the transfer of information between the elements. Attached to system bus  79  is I/O device interface  82  for connecting various input and output devices (e.g., keyboard, mouse, displays, printers, speakers, etc.) to the computer  50 ,  60 . Network interface  86  allows the computer to connect to various other devices attached to a network (e.g., network  70  of  FIG. 5 ). Memory  90  provides volatile storage for computer software instructions  92  and data  94  used to implement an embodiment of the present invention (e.g., virtual environment communication system, group conversation subsystem, invention processor code  37  for persisting conversation portions and enabling other user interaction detailed above). Disk storage  95  provides non-volatile storage for computer software instructions  92  and data  94  used to implement an embodiment of the present invention. Central processor unit  84  is also attached to system bus  79  and provides for the execution of computer instructions. 
         [0051]    In one embodiment, the processor routines  92  and data  94  are a computer program product (generally referenced  92 ), including a computer readable medium (e.g., a removable storage medium such as one or more DVD-ROM&#39;s, CD-ROM&#39;s, diskettes, tapes, etc.) that provides at least a portion of the software instructions for the invention system. Computer program product  92  can be installed by any suitable software installation procedure, as is well known in the art. In another embodiment, at least a portion of the software instructions may also be downloaded over a cable, communication and/or wireless connection. In other embodiments, the invention programs are a computer program propagated signal product  107  embodied on a propagated signal on a propagation medium (e.g., a radio wave, an infrared wave, a laser wave, a sound wave, or an electrical wave propagated over a global network such as the Internet, or other network(s)). Such carrier medium or signals provide at least a portion of the software instructions for the present invention routines/program  92 . 
         [0052]    In alternate embodiments, the propagated signal is an analog carrier wave or digital signal carried on the propagated medium. For example, the propagated signal may be a digitized signal propagated over a global network (e.g., the Internet), a telecommunications network, or other network. In one embodiment, the propagated signal is a signal that is transmitted over the propagation medium over a period of time, such as the instructions for a software application sent in packets over a network over a period of milliseconds, seconds, minutes, or longer. In another embodiment, the computer readable medium of computer program product  92  is a propagation medium that the computer system  50  may receive and read, such as by receiving the propagation medium and identifying a propagated signal embodied in the propagation medium, as described above for computer program propagated signal product. 
         [0053]    Generally speaking, the term “carrier medium” or transient carrier encompasses the foregoing transient signals, propagated signals, propagated medium, storage medium and the like. 
         [0054]    The invention can take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment containing both hardware and software elements. In a preferred embodiment, the invention is implemented in software, which includes but is not limited to firmware, resident software, microcode, etc. 
         [0055]    Furthermore, the invention can take the form of a computer program product accessible from a computer-usable or computer-readable medium providing program code for use by or in connection with a computer or any instruction execution system. For the purposes of this description, a computer-usable or computer readable medium can be any apparatus that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device. 
         [0056]    The medium can be an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system (or apparatus or device) or a propagation medium. Examples of a computer-readable medium include a semiconductor or solid state memory, magnetic tape, a removable computer diskette, a random access memory (RAM), a read-only memory (ROM), a rigid magnetic disk and an optical disk. Current examples of optical disks include compact disk-read only memory (CD-ROM), compact disk-read/write (CD-R/W) and DVD. 
         [0057]    A data processing system suitable for storing and/or executing program code will include at least one processor coupled directly or indirectly to memory elements through a system bus. The memory elements can include local memory employed during actual execution of the program code, bulk storage, and cache memories which provide temporary storage of at least some program code in order to reduce the number of times code must be retrieved from bulk storage during execution. 
         [0058]    Input/output or I/O devices (including but not limited to keyboards, displays, pointing devices, etc.) can be coupled to the system either directly or through intervening I/O controllers. 
         [0059]    Network adapters may also be coupled to the system to enable the data processing system to become coupled to other data processing systems or remote printers or storage devices through intervening private or public networks. Modems, cable modem and Ethernet cards are just a few of the currently available types of network adapters. 
         [0060]    The teachings of all patents, published applications and references cited herein are incorporated by reference in their entirety. 
         [0061]    While this invention has been particularly shown and described with references to example embodiments thereof, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the scope of the invention encompassed by the appended claims. 
         [0062]    As used herein the term “bubble” is illustrative and non-limited for indicating any graphical illustration of messages among or between chat/group conversation participants. 
         [0063]    The term “virtual environment” includes gaming environments, 3D videos, simulations, virtual worlds and other network session environments. 
         [0064]    Where programming or model objects are used in the above description of the invention, it is understood that other computer-related implementations, data structures and processor/methods and the like are suitable. The foregoing use of programming or model objects is for purposes of illustration and not limitation. 
         [0065]    Further, the computer architecture and network configuration of  FIGS. 5 and 6  are for purposes of illustration and not limitation. Other computer architectures and configurations are suitable. 
         [0066]    In the context of a computer network, the terms “on-line session” and “network session” are intended to be non-limiting examples. Other group communications sessions, conversation and/or messaging sessions and the like are suitable.