Computerized image rendering with per-frame buffer scene segmentation

One embodiment of the present invention sets forth a technique for image rendering with per-frame buffer scene segmentation. A user specifies how geometric objects, light sources, and various types of rendering operations in a graphics scene are associated. A segmentation API translates scene data into specific instructions used by a rendering engine that cause the rendering engine to generate a set of scene segments within a set of user specified frame buffers. The scene segments may be composited together using a variable contribution value for each scene segment to generate a final image.

BACKGROUND OF THE INVENTION

Field of the Invention

Embodiments of the present invention relate generally to rendering graphics images and more specifically to computerized image rendering with per-frame buffer scene segmentation.

Description of the Related Art

High-quality graphics rendering systems are conventionally used to generate highly refined images, such as photorealistic images, from mathematical models of three-dimensional (3D) scenes. Unlike real time graphics rendering systems, which typically render sixty or more images per second, many high-quality graphics rendering systems commonly require several minutes to over a day to render a single image. Because the goal of high-quality rendering systems is to render images to the highest technically feasible standards, iterations of fine-tuning operations are frequently applied to each image before a final image is produced. Fine-tuning iterations are typically conducted by a user who may visually examine an image before accepting the image or proceeding with an additional fine-tuning iteration.

To minimize the enormous computational time and effort associated with fully rendering a high-quality image in conjunction with performing numerous fine-tuning iterations on the image, the overall rendering process is partitioned into a rendering phase and a compositing phase, wherein the fine-tuning iterations may be substantially performed in the compositing phase. In this approach, individual scene elements (objects within a virtual environment) or groups of scene elements may be independently rendered as scene segments that are stored within corresponding frame buffers. Computationally efficient compositing operations are then used to combine the scene segments to generate a final image. Contributions from individual segments may be adjusted in the compositing phase to perform fine-tuning operations without repeating a time consuming rendering phase. For example, if a specular highlight of a scene element is rendered into a separate segment, the intensity of the specular highlight may be adjusted independently via one or more compositing iterations to fine-tune the contribution of the specular highlight without re-rendering the image. Because the rendering phase comprises a significant majority of the overall computation time, and the compositing phase requires a relatively small amount of computation time, excellent overall efficiency is achieved with this approach, even when multiple compositing iterations are needed.

One key aspect to this partitioned approach involves determining how a scene should be decomposed into segments, according to individual scene elements or groups of scene elements. Typically, a user may select one or more scene elements to be rendered separately from the other scene elements, which are excluded from rendering. The scene elements of the decomposed scene are then rendered in separate rendering passes to generate each respective scene segment.

This approach has at least two significant drawbacks. The first drawback is that in a given rendering pass, excluded scene elements do not contribute to a rendered segment of selected scene elements, resulting in degradation of both image quality and realism. For example, a selected scene element that is supposed to be reflective will not actually include a reflection of a nearby scene element that is excluded from rendering. As a result of this drawback, users are faced with a complex scene element management problem, requiring additional effort to avoid obvious lapses in realism. The second drawback is that multiple rendering passes are required to render segments into multiple separate frame buffers, thereby requiring additional rendering computation and complexity, and reducing overall efficiency.

As the foregoing illustrates, what is needed in the art is a technique for enabling efficient generation and fine-tuning of high-quality graphics.

SUMMARY OF THE INVENTION

One embodiment of the present invention sets forth a method for rendering images with per-frame buffer scene segmentation. The method includes the steps of receiving geometric object data associated with a graphics scene, receiving lighting data associated with the graphics scene, receiving relationship data that associates one or more objects in the graphics scene with one or more lights in the graphics scene, generating a contribution map based on the relationship data that indicates one or more rendering pass types corresponding to one or more lighting characteristics, and generating rendering instructions for rendering the graphics scene on a segment-by-segment basis, where each segment is defined by a rendering pass type and one or more geometric objects associated with one of the one or more lighting characteristics, and each rendered segment is stored in a different frame buffer.

One advantage of the disclosed method is that it simplifies how relationships between graphics objects and rendered segments are managed and also enables multiple segments to be rendered in a single rendering phase.

DETAILED DESCRIPTION

FIG. 1illustrates a computer system100configured to implement one or more aspects of the present invention. The computer system100includes, without limitation, a central processing unit (CPU)130, a system memory110, a graphics processing unit (GPU)134, a GPU memory120, a memory bridge132, a display device136, a hard disk140, a device bridge142, a network interface144, a mouse146, and a keyboard148.

The CPU130communicates with the system memory110via the memory bridge132, which may be, e.g., a Northbridge device or subsystem. System memory110is configured to store application programs, as well as data used by or generated by the CPU130. System memory110is coupled to the memory bridge132via a system memory bus150. The memory bridge132is coupled to the GPU134via a GPU system bus152. The GPU system bus152may comprise any technically feasible data interconnect, such as the well known personal computer interconnect (PCI) express bus. The memory bridge132is also coupled to the device bridge142using an interconnect system such as PCI. The GPU134conventionally incorporates real time image rendering means for rendering both three-dimensional (3D) and two-dimensional (2D) images. The GPU134delivers pixel data to display device136, which may comprise a conventional CRT or LCD display. The GPU134is coupled to the GPU memory120using a GPU memory bus154. The GPU memory120may be configured to store data used by or generated by the GPU134. Data stored within the GPU memory120passes through the GPU134and the memory bridge132when accessed by the CPU130. In some embodiments, the integrated circuit implementing the CPU130may incorporate additional functional blocks, such as the memory bridge132and the device bridge142. In alternative embodiments, the integrated circuit implementing the GPU134may incorporate additional functional blocks, such as the memory bridge132and the device bridge142.

The device bridge142is coupled to a hard drive140, a network interface144, a mouse146, and a keyboard148. The hard drive140provides mass storage of programs and data. The network interface144provides network connectivity to other computers using a local area network (LAN) interface using any suitable technology, such as Ethernet. The mouse146and keyboard148provide user input. Other components (not explicitly shown), including USB or other port connections, CD drives, DVD drives, film recording devices, and the like, may also be connected to I/O bridge107. Communication paths interconnecting the various components inFIG. 1may be implemented using any suitable protocols, such as PCI (Peripheral Component Interconnect), PCI Express (PCI-E), AGP (Accelerated Graphics Port), HyperTransport, Quick Path Interconnect, or any other bus or point-to-point communication protocol(s), and connections between different devices may use different protocols as is known in the art.

In one embodiment, system memory110is configured to store a graphics modeling application112, a graphics rendering application114, and a compositing application116. The graphics rendering application114should include at least one shader, a rendering application programming interface (API) for interfacing to a rendering engine, and a segmentation API for interfacing to the rendering API. The shader may communicate with the rendering engine via the rendering API, via the segmentation API, or via both APIs. System memory110is also configured to store a plurality of frame buffers118, which may be configured to store scene segments rendered by the rendering engine, and an image generated by the compositing application116. The compositing application116combines segments according to a contribution value for each segment to generate a composite image. For example, the rendering application114may render segments stored in frame buffers118-0through118-2, and the compositing application116may combine the segments to generate a composite image, such as a final image, stored in frame buffer118-3.

In an alternative embodiment, a first computer system includes a modeling application, and may include a compositing application. Additionally, a set of one or more computer systems includes a least one instance of the graphics rendering application. The first computer system and the set of one or more computer systems are configured to communicate via a computer network. In this embodiment, the first computer system includes software configured to cause each computer system in the set of one or more computer systems to independently render and store scene segments.

FIG. 2illustrates a rendering application210configured to generate per frame buffer scene segmentation, according to one embodiment of the invention. The rendering application210includes a data set of scene information212, a shader222, segmentation API232, rendering API230, and rendering engine240. The rendering application210may also include shader220. The rendering application210generates a plurality of scene segments and stores the scene segments in frame buffers250.

The scene information212may include, without limitation, detailed descriptions of render objects (e.g., geometric objects), lights, cameras, render passes, and render layers that define associations between render objects, lights and render passes. Associations defined within a given render layer may be interpreted by the segmentation API232to generate a pass contribution map that is used to guide the rendering engine240in appropriately rendering each scene segment.

Scene information requiring no segmentation information may be processed using shader220, which invokes rendering API230to generate an associated image. Scene information requiring segmentation information should be processed using shader222, which invokes the segmentation API232. In response to instructions from shader222, the segmentation API232generates instructions for the rendering engine240to generate each scene segment and store each scene segment in an associated frame buffer250.

FIG. 3illustrates a three-dimensional graphics scene300. The graphics scene300includes geometric objects such as a room344comprising rectangular panels, a window340, an environment342comprising an image, object1310, object2312, and object3314. Graphics scene300also includes two lights, comprising a sun320and a bulb322.

Object1310may be rendered in isolation for the purpose of rendering scene segments into independent frame buffers250that may be used for fine-tuning the rendered image in a subsequent compositing phase. To maximize realism, object1310should be rendered with other objects and lights, in context in the scene, wherein each rendering pass or group of rendering passes may be written to a separate frame buffer.

FIG. 4illustrates associations between a rendering element and rendering passes, according to one embodiment of the invention. In this example, the rendering element is object1310ofFIG. 3. Scene objects are organized into a list of objects452. Lights are organized into a list of lights450. Rendering passes are organized into a list of passes456. Pass contributions from each rendering pass for a given object are organized into pass contribution maps454, which define relationships among objects, lights, rendering passes, and lighting characteristics. In this example, a lighting characteristic is a related set of rendering passes or types of rendering passes.

As shown, rendering passes associated with object1310include, without limitation, diffuse lighting440, ambient lighting442, specular lighting444, ambient occlusion446, matte surface mapping448, opacity450, and reflection452. Environment mapping454is only associated with the environment object342in this example. Each rendering pass imparts a specific effect on object1310. The effects of multiple rendering passes may be grouped into pass contribution maps454. For example, diffuse lighting440, ambient lighting442, specular lighting444, and ambient occlusion446produce an illumination contribution430; matte surface mapping448produces a matte pass contribution432; rendering opacity450produces an opacity contribution434; and, a reflection rendering pass452produces a ray tracing contribution436.

For each object in the scene300, the pass contribution maps454define associations between lights450, objects452, and rendering passes456. These associations may be defined using any technically feasible means. In one embodiment, an application user may define associations between objects, lights, and rendering passes using a graphical user interface (GUI) associated with the rendering application210ofFIG. 2. One exemplary layout of a GUI used to define associations between objects and rendering passes is illustrated inFIG. 4. Persons skilled in the art will recognize that a drag-and-drop scheme for defining associations may provide an intuitive and efficient entry means for users. Association data should be stored in scene information212for processing by shader222and segmentation API232. By defining, rendering, and storing results of each rendering pass in a separate frame buffer, the user is able to subsequently fine-tune the contribution of each rendering pass (e.g., in a compositing application) without performing additional time consuming rendering operations.

In one embodiment, object, light, rendering pass, and frame buffer association information needed to generate the pass contribution maps454are transmitted to segmentation API232, which performs translations that enable the rendering engine240to render each separate segment associated with each object (or set of objects) into a separate frame buffer.

FIG. 5is a flow diagram of method steps500for rendering a segmented scene, according to one embodiment of the invention. Although the method steps are described in conjunction with the systems ofFIGS. 1 and 2, persons skilled in the art will understand that any system configured to perform the method steps, in any order, is within the scope of the invention.

The method begins in step510, where the segmentation API232receives data for geometric objects and lights used in a graphics scene. This data may include, without limitation, geometric meshes, vertex attributes, lights and associated position information, and so forth. This data may be received in the form of one or more data structures, one or more API function calls that communicate data, or any other technically feasible technique. In step512, the segmentation API232receives material properties for the geometric objects. Material properties may include, without limitation, texture maps and surface colors. In step514, the segmentation API232receives configuration information and properties related to lights in the scene, including, without limitation, light emissions patterns, light maps, intensity values, and colors. In step516, the segmentation API232receives configuration information for cameras established in the scene, including location and direction details for each camera.

In step518, the segmentation API232receives requirements and properties for each requested frame buffer, which may be characterized by, without limitation, two-dimensional resolution and pixel depth. In step520, the segmentation API232receives object to frame buffer association information, thereby establishing which objects, from the list of objects452, are to be segmented and rendered into respective frame buffers. In step522, the segmentation API232receives light to frame buffer association information, thereby establishing which lights from the list of lights450are associated with specific frame buffers and related objects.

In step524, the segmentation API232generates contribution maps from association information received in steps518through522. The contribution maps relate objects, lights and rendering passes with individual frame buffers, where each rendering pass may be associated with a specific lighting characteristic or other intermediate computation performed by a shader, such as shader222.

In step526, the segmentation API232generates instructions for the rendering engine by translating scene data and contribution map data into a format suitable for the rendering engine240, while adhering to contribution map information that specifically details a mapping from scene segments to frame buffers. In other words, the contribution map information guides the rendering engine in performing rendering operations needed to satisfy requirements set forth in the contribution maps. The segmentation API232may use any technically feasible technique to translate scene data and generate rendering instructions.

In step530, the segmentation API232initiates rendering of the scene into one or more frame buffers. In one embodiment, the rendering engine renders the scene into one or more frame buffers in a single rendering phase, wherein each rendering pass for each frame buffer is computed in a single launch of the rendering engine. In an alternative embodiment, one or more rendering passes are computed per launch of the rendering engine, where each launch of the rendering engine generates one or more scene segments, as specified by the contribution map.

In sum, a technique is disclosed for rendering images with per-frame buffer scene segmentation. A user specifies associations between objects, lights, and rendering passes in a graphics scene. A segmentation API translates scene data and contribution map information to generate specific instructions to a rendering engine that cause the rendering engine to generate a set of scene segments within a set of frame buffers. A compositing application may be used to generate final images from the scene segments.