Non-real-time enhanced image snapshot in a virtual world system

A computer implemented method, a tangible computer storage medium, and a data processing system generate a non-real-time image snapshot in a virtual world. A backend rendering system receives a scene description from a virtual world server. The backend rendering system then retrieves a high fidelity graphics from a visual database. The visual database contains both the high fidelity graphics and a lightweight graphics. The lightweight graphics are utilized to render the virtual world at a virtual world client application. The backend rendering system then renders the non-real-time image snapshot, and processes the non-real-time image snapshot into the desired format. The non-real-time image snapshot can then be delivered to a client.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention is related generally to a computer implemented method, a tangible computer storage medium and a data processing system. More particularly, the present invention is related generally to a computer implemented method, a tangible computer storage medium and a data processing system for providing non-real-time enhanced image snapshots in a virtual world system.

2. Description of the Related Art

A virtual world (VW), also referred to as a metaverse, is a computer-based simulated environment. Examples of virtual worlds include Second Life®, Entropia Universe, The Sims Online®, There, and Red Light Center. Other examples of virtual worlds include multiplayer online games, such as EverQuest®, Ultima Online®, Lineage®, and World of Warcraft® (WoW).

Many virtual worlds are represented using three dimensional (3-D) graphics and landscapes. The properties and elements of the virtual world often resemble the properties of the real world, such as in terms of physics, and 3D content such as houses, and landscapes. Virtual worlds may be populated by thousands of users simultaneously. In a virtual world, users are sometimes referred to as “residents” or “agents”

The users in a virtual world can interact, inhabit, and traverse the virtual world through the use of avatars. An avatar is a graphical representation of a user that other users in the virtual world can see and interact with. The avatar's appearance is typically selected by the user and often takes the form of a cartoon-like representation of a human. However, avatars may also have non-human appearances, such as animals, elves, trolls, orcs, fairies, and other fantasy creatures.

A viewable field is the field of view for a particular user. The viewable field for a particular user may include objects, as well as avatars belonging to other users. An object is an element in a virtual world that does not represent a user. An object may be, for example, buildings, statues, billboards, signs, and advertisements in the virtual world. The viewable field of a particular user is determined by the virtual world software according to the geometries and textures that are currently loaded in a user's virtual world client. The virtual world software determines the length of time that a user views an object based on processing the data sent to each virtual world client.

Some virtual worlds allow each virtual world client to record images of the virtual world by taking a snapshot, or screenshot, of the particular user's viewable field. The snapshot is typically performed locally on the virtual world client's machine, using local 3-D rendering hardware and software. The level of detail and realism of the rendered 3-D image therefore matches the level of detail and realism rendered by the virtual world.

However, due to the interactive nature and requirements of the virtual world, the level of detail and realism rendered by the virtual world is significantly less than the level of detail and realism that is achievable with current 3-D hardware and 3-D software rendering applications. Images within the virtual world must be rendered quickly to provide the virtual world client with a smooth navigational and visual experience. The level of detail and realism rendered by the virtual world client is often compromised in order to render quickly the changing environment of the virtual world. The CPU/GPU processing requirements are simply too high to create a high quality video stream of a running virtual world on a client machine.

BRIEF SUMMARY OF THE INVENTION

According to one embodiment of the present invention, a computer implemented method, apparatus, and computer usable program product is provided for generating a non-real-time image snapshot in a virtual world. A graphics-processing unit receives a scene description from a virtual world server. The graphics-processing unit then retrieves a first set of graphics from a visual database. The visual database contains both the first set of graphics and a second set of graphics. The second set of graphics are utilized to render the virtual world at a virtual world client application. The first set of graphics are not utilized by the virtual world client application in rendering the virtual world in real time. The graphics-processing unit then renders the non-real-time image snapshot from the first set of graphics and the scene description, and processes the non-real-time image snapshot into the desired format. The non-real-time image snapshot can then be delivered to a client application.

DETAILED DESCRIPTION OF THE INVENTION

Program code located in network data processing system100may be stored on a computer recordable storage medium and downloaded to a data processing system or other device for use. For example, program code may be stored on a computer recordable storage medium on server104and downloaded to client110over network102for use on client110.

In some illustrative embodiments, program code216may be downloaded over a network to persistent storage208from another device or data processing system for use within data processing system200. For instance, program code stored in a computer readable storage medium in a server data processing system may be downloaded over a network from the server to data processing system200. The data processing system providing program code216may be a server computer, a client computer, or some other device capable of storing and transmitting program code216.

The different embodiments may be implemented using any hardware device or system capable of executing program code. As one example, the data processing system may include organic components integrated with organic components and/or may be comprised entirely of organic components excluding a human being. For example, a storage device may be comprised of an organic semiconductor.

As another example, a storage device in data processing system200is any hardware apparatus that may store data. Memory206, persistent storage208, and computer readable media218are examples of storage devices in a tangible form.

The illustrative embodiments described herein provide a computer implemented method, a tangible computer storage medium, and a data processing system for generating a non-real-time image snapshot in a virtual world. A backend rendering system receives a scene description from a virtual world server. The backend rendering system then retrieves a high fidelity graphics from a visual database. High fidelity graphics are a high fidelity version of geometry, textures, shaders that are utilized by a backend rendering system to create high fidelity snapshots and videos. Due to the bandwidth and processing requirements, high fidelity graphics are not utilized by the virtual world client application in rendering the virtual world in real time.

The visual database contains both the high fidelity graphics and a lightweight graphics. The lightweight graphics are utilized to render the virtual world at a virtual world client application. The backend rendering system then renders the non-real-time image snapshot, and processes the non-real-time image snapshot into the desired format. The non-real-time image snapshot can then be delivered to a client. A snapshot is a graphical rendering of the viewable field of a particular user's avatar, as displayed by the virtual world client application. The snapshot can be a static image, or a video of the rendering of the viewable field.

FIG. 3is a block diagram illustrating a virtual world server in accordance with an illustrative embodiment. Server300is a server associated with a virtual world. Server300may be a single, stand-alone server, or server300may be a server in a virtual world computing system or in a cluster of two or more servers. In this example, server300is a server in a computing system for rendering and managing a virtual world.

Virtual world database302is a database on the computing system for storing data used by virtual world software308to render and manage the virtual world. Virtual world database302includes object avatar rendering (OAR) table304. Object avatar rendering table304stores object universally unique identifiers and avatar universally unique identifiers.

In a virtual world, assets, avatars, the environment, and anything visual consists of universally unique identifiers (UUIDs) tied to geometric data, textures, and effects data. Geometric data is data associated with the form or shape of avatars and objects in the virtual world. Geometric data may be used to construct a rendered mesh of an avatar or object. Geometric data is distributed to a user's client computer as binary coordinates. Textures are distributed to a user's client computer as graphics files, such as JPEG files. Texture data refers to the surface detail and surface textures or color that is applied to mesh geometric data to render the object or avatar. Effects data is typically rendered by the user's client according to the user's preferences and the user's client device capabilities.

Object avatar rendering table304stores a universally unique identifier (UUID) for each selected object in the virtual world. A selected object is an object in a plurality of objects in the virtual world that is tracked, monitored, managed, or associated with object avatar rendering table304. Object avatar rendering table304also stores universally unique identifiers and other data describing avatars within a viewable field of a selected object or within a selected zone or range associated with the selected object. For example, if the selected objects include object A and object B, then object avatar rendering table304stores object A universally unique identifier, universally unique identifiers and other data for all avatars and objects within selected range of object A, object B universally unique identifier, and universally unique identifiers and other data describing all avatars and objects within selected range of object B.

Object based avatar tracking controller306stores data314in object avatar rendering table304. Data314includes the universally unique identifiers and other data describing avatars within the viewable field of one or more selected objects. When object based avatar tracking controller306needs data from object avatar rendering table304for implementing geometric and texture modifications in the virtual world, object based avatar tracking controller306sends query310to object avatar-rendering table304. In response to query310, virtual world database302sends data312to virtual world software308for utilization by object based avatar tracking controller306to track avatars and implement modifications of the selected objects to improve the location and appearance of the selected objects within the virtual world and enable improved visibility of the selected objects.

Virtual world software308is software for rendering the virtual world. Virtual world software308includes object-based avatar tracking controller306. Object based avatar tracking controller306is software for tracking avatars within the viewable field of each selected object.

Object avatar rendering table304stores a universally unique identifier (UUID) for each selected object in the virtual world. A selected object is an object in a plurality of objects in the virtual world that is tracked, monitored, managed, or associated with object avatar rendering table304. Object avatar rendering table304also stores universally unique identifiers and other data describing avatars within a viewable field of a selected object or within a selected zone or range associated with the selected object. For example, if the selected objects include object A and object B, then object avatar rendering table304stores object A universally unique identifier, universally unique identifiers and other data for all avatars within the viewable field of object A, object B universally unique identifier, and universally unique identifiers and other data describing all avatars within the viewable field of object B.

Object avatar rendering table304stores a universally unique identifier (UUID) for each selected object in the virtual world. A selected object is an object in a plurality of objects in the virtual world that is tracked, monitored, managed, or associated with object avatar rendering table304. Object avatar rendering table304also stores universally unique identifiers and other data describing avatars within a viewable field of a selected object or within a selected zone or range associated with the selected object. For example, if the selected objects include object A and object B, then object avatar rendering table304stores object A universally unique identifier, universally unique identifiers and other data for all avatars within the viewable field of object A, object B universally unique identifier, and universally unique identifiers and other data describing all avatars within the viewable field of object B.

Referring now toFIG. 4, a block diagram of the data flow between various hardware and software components of a client/server environment implementing a virtual world is shown according to an illustrative embodiment.

Virtual World client application410is a software application, executing on a client data processing system, such as one of clients110,112, and114ofFIG. 1. Virtual World client application410allows for connection to, and interaction within a virtual world. In response to an indication from a user, virtual world client application410sends snapshot request412to virtual world server414. Snapshot request412is a request for a graphical rendering of the viewable field of a particular user's avatar, as displayed by virtual world client application410. The snapshot request can be a request for a static image, or a video of the rendering of the viewable field.

Virtual world server414is a data processing system that executes software to enable a virtual world, such as virtual world server300. Responsive to receiving snapshot request412, virtual world server414forwards scene descriptions416to backend rendering system418. Scene description416is a non-graphical representation of the viewable field of a particular user's avatar. Scene description416includes virtual world location information such as an exact camera location within the virtual world, and a camera angle vector. The exact camera location within the virtual world and the camera angle vector can be expressed as coordinates in a 3-D coordinate system, such as for example in an <X, Y, Z> format. Scene description416can also include information regarding the format and delivery of the rendered snapshot, such as for example but not limited to, an image size, a file type format, delivery options, whether to include avatars within the snapshot, and desired fidelity/resolution of the snapshot. Scene description416going to backend rendering system418are a subset of the updates going to the client.

Backend rendering system418is a graphics-processing unit capable of generating images and video of activity within a virtual world. Backend rendering system418is capable of rendering images and video of higher quality than local 3-D rendering hardware and software running virtual world client application410because backend rendering system418utilizes images and textures that are themselves not used to create the virtual world as rendered by virtual world client application410.

Backend rendering system418produces high quality images by using higher quality models, shaders, and textures, or by implementing a ray tracing system. The models, shaders, textures, and ray tracing system utilized by backend rendering system418are not used in rendering the virtual world as is typically rendered by virtual world client application410. In one illustrative embodiment, backend rendering system418implements high quality ray tracing effects using a Cell Broadband Engine Architecture based system. In one illustrative embodiment, backend rendering system418implements high quality models, shaders, and textures using traditional graphical processing unit hardware.

Responsive to receiving scene description416, backend rendering system418sends request420for high fidelity objects to visual database422. Visual database422contains textures, geometry, and shaders that are needed to implement the virtual world. A scene description is downloaded onto backend rendering system418, and the needed textures, geometry, and shaders are downloaded from the Visual database. Updates are then sent to backend rendering system418as the avatar moves in the scene. Backend rendering system418adjusts and moves the rendered avatars and objects accordingly.

Visual database422stores geometric data, textures, and effects data associated with the form or shape of avatars and objects in the virtual world. Visual database422is referenced by an Object avatar-rendering table, such as Object avatar rendering table310in order to obtain geometric data, textures, and effects data to render the virtual world. Visual database422contains lightweight graphics424and high fidelity graphics426. Lightweight graphics424include the 3D geometry, textures, shaders that are utilized by both virtual world server414and virtual world client application410to render the virtual world to the client. Lightweight graphics424are delivered to the virtual world server414in response to a request428for low fidelity objects sent from virtual world server414to visual database422.

High fidelity graphics426is a high fidelity version of geometry, textures, shaders that are utilized by backend rendering system418to create high fidelity snapshots and videos. Due to the bandwidth and processing requirements, high fidelity graphics426are not utilized by virtual world client application410in rendering the virtual world in real time.

Backend rendering system418utilizes high fidelity graphics426to create high fidelity video and images430. In one illustrative embodiment, backend rendering system418implements high quality ray tracing effects using a cell based system. In one illustrative embodiment, backend rendering system418implements high quality models, shaders, textures using traditional graphical processing unit hardware.

In one illustrative embodiment, backend rendering system418implements high quality ray tracing effects using the Cell Broadband Engine Architecture based system. Backend rendering system418ray tracing determines the visibility of surfaces within an image by tracing imaginary rays of light from viewer's eye to the object. A separate ray is utilized for each resolved point, typically for each pixel of the display. For example, for a modest image resolution of 1064×768, a ray-tracer needs to trace 1064×768, or 817,152 separate rays for each displayed image. While the ray-traced image is very realistic due to the separate casting of each resolved point, the sheer number of rays that must be traced results in a longer process times that are typically unsuitable for real time processing that must occur when implementing a virtual world.

In one illustrative embodiment, backend rendering system418implements high quality ray tracing effects using a cell based system. A cell based system is a data processing system using the Cell Broadband Engine Architecture. The Cell Broadband Engine Architecture is a multi-core processor. The cell-based system differs from traditional multi-core processors in that the cell based system does not consist of multiple copies of the same core. Instead, the cell based system utilizes heterogeneous processors, consisting of one PowerPC core and a plurality of synergistic co-processor elements. The synergistic co-processor elements are typically smaller than traditional cores, and are designed for streaming workloads. The PowerPC core performs synchronization tasks and executes non-parallizeable code. The synergistic co-processor elements perform the balance of the work in a streaming manner.

In one illustrative embodiment, backend rendering system418creates create high fidelity video and images430by sending scene updates to the backend rendering system as the client moves the avatar moves in the scene and objects move or change in the scene. The backend rendering system adjusts and moves the viewing position and rendered avatars and objects accordingly.

In one illustrative embodiment, ray-traced images rendered by the Cell Broadband Engine Architecture based system of backend rendering system418are created at fast time intervals using scene updates. To create the illusion of smoother movement between the ray-traced key frames, the key frame animation software extrapolates in-between geometry and positions from scene updates to create a smoother animation.

As the client navigates the virtual world, avatar and camera movement432are communicated to virtual world server414. Additional scene descriptions434are passed to backend rendering system418, and stored in scene update database436until additional scene descriptions434are utilized by backend rendering system418. Scene update database436is a storage, such as storage108ofFIG. 1, that holds scene descriptions, such as additional scene descriptions434, until those scene descriptions are utilized by backend rendering system418to create high fidelity video and images430. Thus, scene update database436acts as a buffer system, temporarily storing additional scene descriptions434for the backend rendering system418. In this manner, backend rendering system418is able to function in a non-real time. Backend rendering system418is therefore able to create realistic images and videos without sacrificing resolution so that processing can be performed in real time.

Once backend rendering system418creates high fidelity video and images430, high fidelity video and images430are stored in video database438until high fidelity video and images430is delivered to a client system.

Referring now toFIG. 5, a visual database including both high fidelity graphics and low fidelity graphics is shown according to an illustrative embodiment. Visual database500is a visual database such as visual database422ofFIG. 4.

Visual database500includes Object identifiers510. Object identifiers510is a universally unique identifier for an object within the virtual world. Each unique object within the virtual world is referenced by a unique one of Object identifiers510. In a virtual world, assets, avatars, the environment, and anything visual consists of universally unique identifiers (UUIDs) tied to geometric data, textures, and effects data. Geometric data is data associated with the form or shape of avatars and objects in the virtual world. Geometric data may be used to construct a mesh model of an avatar or object. Geometric data is distributed to a user's client computer as binary coordinates. Textures are distributed to a user's client computer as graphics files, such as JPEG files. Texture data refers to the surface detail and surface textures or color that is applied to mesh geometric data to render the object or avatar. Effects data is typically rendered by the user's client according to the user's preferences and the user's client device capabilities.

Visual database500includes lightweight graphic512. Lightweight graphic identifiers512include the 3D geometry, textures, shaders that are utilized by both a virtual world server and a virtual world client application, such as virtual world server414and virtual world client application410respectively, to render the virtual world to the client. Lightweight graphic512are delivered to the virtual world server or the virtual world client application in response to a request for low fidelity objects sent from virtual world server to visual database.

Visual database500includes high fidelity graphics514. High fidelity graphics514is a high fidelity version of geometry, textures, shaders that are utilized by a backend rendering system, such as backend rendering system418ofFIG. 4, to create high fidelity snapshots and videos. Due to the bandwidth and processing requirements, high fidelity graphics514are not utilized by the virtual world server and the virtual world client application in rendering the virtual world in real time.

Referring now toFIG. 6, a flowchart showing the processing steps for receiving a request for, and delivering a high-fidelity non-real-time enhanced image snapshot in a virtual world is shown according to an illustrative embodiment. Process600is a software process executing on a virtual world server, such as virtual world server414ofFIG. 4.

Process600begins by receiving a snapshot request from a virtual world's client application (step610). The snapshot request is a request for a graphical rendering of the viewable field of a particular user's avatar, as displayed by virtual world client application. The snapshot request can be a request for a static image, or a video of the rendering of the viewable field. The snapshot request can be snapshot request412ofFIG. 4.

Responsive to receiving the snapshot request, process600forwards a scene description to a backend rendering system (step620). The scene description includes virtual world location information such as an exact camera location within the virtual world, and a camera angle vector. The exact camera location within the virtual world and the camera angle vector can be expressed as coordinates in a 3-D coordinate system, such as for example in an <X, Y, Z> format. The scene description can also include information regarding the format and delivery of the rendered snapshot, such as for example but not limited to, an image size, a file type format, delivery options, whether to include avatars within the snapshot, and desired fidelity/resolution of the snapshot. The scene description going to backend rendering system is a subset of the updates going to the client. That is, only the information that affects the visual or audio representation of the virtual world as rendered in the viewable field of a particular user's avatar are sent in the scene description to the backend rendering system.

At some later time when the backend rendering system has finished processing the high-fidelity non-real-time enhanced image snapshot, process600may receive the high-fidelity non-real-time enhanced image snapshot from the backend rendering system (step630). Responsive to receiving the high-fidelity non-real-time enhanced image snapshot, process600delivers the high-fidelity non-real-time enhanced image snapshot to the virtual client application (step640), with the process terminating thereafter.

Referring now toFIG. 7, a flowchart showing the processing steps for generating a high-fidelity non-real-time enhanced image snapshot in a virtual world is shown according to an illustrative embodiment. Process700is a software process executing on a backend rendering system, such as backend rendering system418ofFIG. 4.

Process700begins by receiving a scene description from a virtual world server (step710). The virtual world server can be a virtual world server such as virtual world server414ofFIG. 4. The scene description can be scene description416ofFIG. 4. The scene description includes virtual world location information such as an exact camera location within the virtual world, and a camera angle vector. The exact camera location within the virtual world and the camera angle vector can be expressed as coordinates in a 3-D coordinate system, such as for example in an <X, Y, Z> format. The scene description can also include information regarding the format and delivery of the rendered snapshot, such as for example but not limited to, an image size, a file type format, delivery options, whether to include avatars within the snapshot, and desired fidelity/resolution of the snapshot. The scene description going to backend rendering system is a subset of the updates going to the client. That is, only that information that affects the visual or audio representation of the virtual world as rendered in the viewable field of a particular user's avatar are sent in the scene description.

Responsive to receiving the scene description, process700retrieves high fidelity graphics from a visual database (step720). The visual database can be a visual database such as visual database500ofFIG. 5. The high fidelity graphics is a high fidelity version of geometry, textures, shaders that are utilized by the backend rendering system to create high fidelity snapshots and videos. Due to the bandwidth and processing requirements, these high fidelity graphics are not utilized by the virtual world server and the virtual world client application in rendering the virtual world in real time.

Responsive to retrieving the high fidelity graphics from a visual database, process700renders the high fidelity snapshot (step730). The snapshot can be either a static, or a video image. In one illustrative embodiment, backend rendering system implements high quality models, shaders, textures using traditional graphical processing unit hardware. In one illustrative embodiment, process700implements high quality ray tracing effects using a cell based system. Process700ray tracing determines the visibility of surfaces within an image by tracing imaginary rays of light from viewer's eye to the object.

In one illustrative embodiment, process700creates high fidelity video and images at fast time intervals using scene updates. To create the illusion of smoother movement between the ray-traced key frames, the key frame animation software extrapolates in-between geometry and positions from scene updates to create a smoother animation.

Responsive to retrieving the high fidelity graphics from a visual database, process700determines if any additional scene description for the current snapshot or video are present (step740). Process700can determine the presence of additional snapshots by identifying additional scene descriptions from a scene update database, such as scene update database434ofFIG. 4. The additional scene descriptions can be additional scene descriptions434ofFIG. 4. Responsive to determining that additional scene descriptions are present in the scene update database, process700iterates back to step720. The high fidelity graphics matching the additional scene descriptions are retrieved from the visual database and processed into the high fidelity snapshot.

Responsive to determining that additional scene descriptions are not present in the scene update database, process700then processes the generated high fidelity snapshot into the desired format (step750). In one illustrative embodiment, the high fidelity ray-traced images rendered by a Cell Broadband Engine Architecture based system of backend rendering system are created at fast time intervals using scene updates. These images are then formatted into a smoother movement video using a key frame animation software to extrapolate in-between geometry and positions between the ray-traced key frames from scene updates to create a smooth animation.

Process700forwards the snapshot to the virtual world server for deliver to the virtual world client application (step760), with the process terminating thereafter. By rendering the snapshot in an off-line environment such as in the backend rendering system of Process700, snapshots are created having a much higher level of detail and realism than those rendered by the virtual world. Snapshot rendered according to the described process possess a level of detail and realism that equal to that achievable with current 3-D hardware and 3-D software rendering applications without adversely influencing the performance of the virtual world server or client. Images within the virtual world continue to be rendered quickly to provide the virtual world client with a smooth navigational and visual experience, while the processing and rendering of the high fidelity video is processed separately by the backend rendering system.

Thus, the illustrative embodiments described herein provide a computer implemented method, a tangible computer storage medium, and a data processing system for generating a non-real-time image snapshot in a virtual world. A backend rendering system receives a scene description from a virtual world server. The backend rendering system then retrieves a high fidelity graphics from a visual database. The visual database contains both the high fidelity graphics and a lightweight graphics. The lightweight graphics are utilized to render the virtual world at a virtual world client application. The backend rendering system then renders the non-real-time image snapshot, and processes the non-real-time image snapshot into the desired format. The non-real-time image snapshot can then be delivered to a client.