System and method for creating motion blur

An embedded, programmable motion blur system and method is described herein. Embodiments include applying displacement on a vertex level and amplification on a primitive level within a graphics pipeline. Embodiments include receiving a plurality of vertices in a first component of a graphics processing unit (GPU), displacing each of the plurality of vertices, receiving a primitive that includes at least one of the displaced vertices in a second component of the GPU, and transforming the primitive. In one embodiment, transforming comprises at least one of translation, scaling, and rotation. Further included are generating a plurality of primitive samples over a time aperture, and outputting the plurality of primitive samples to further components of the GPU for further processing for display of the scene with motion blur.

TECHNICAL FIELD

The invention is in the field of graphics processing.

BACKGROUND

Motion blur is an effect seen in photographs taken of a moving object. A camera exposes a sheet of light sensitive film to a scene for a short period of time. The light from the scene hits the film, reacts with the chemicals in the film, and results in the eventual photograph. If the scene changes during the previously described exposure, a blurred image results. Motion blur is present in most films and television programs, and may not be very noticeable, but its presence lends a realistic feeling. In computer animation, it is often desirable to create an effect that is as close to a filmed scene as possible. The absence of motion blur in computer animations is noticeable and makes the animation seem unrealistic. For example, fast movement in an animation that has no motion blur is jerky.

Various techniques have been devised to include motion blur in computer animated scenes. For example, to create motion blur in animated scenes, many additional frames are rendered than previously for display over a particular time period, so that the jerkiness of movement from frame to frame can be smoothed out. However, current techniques for including motion blur in animated scenes have significant limitations. In general, current motion blur techniques do not provide both good quality, and acceptable interaction and efficiency.

FIG. 1is a block diagram of one traditional post rendering motion blur technique100. Technique100involves a graphics processing unit (GPU)102and a software application104. The software application communicates with a host processor (not shown) and a memory (not shown) in a system. The memory may be shared with other components or functions. The memory includes an accumulation buffer that is used by the application104. Technique100performs scene processing and multi-pass rendering to generate additional samples of scene objects. A scene is rendered, or drawn, in a time t, then another is drawn in a time t+1, t+2, etc. The times define an aperture period, which is analogous to the time a camera aperture is open and exposed to a scene. The rendered scenes are superimposed one on another over the aperture period. The scene thus obtains some motion blur.

Technique100executes one rendering pass per image space sample, requiring tens of samples to achieve minimally acceptable quality, and is thus slow. Technique100is also fairly limited in its ability to improve motion blur quality. For example, technique100is especially limited in it transformation capability. Technique100is further limited to applying motion blur in a single direction conforming to a predefined vector.

Technique100also potentially imposes a computation burden. If it desirable to apply the application104to every scene, the entire process of rendering must be repeated many times, complicating the application104because the application program must manage all of the image processing and also display the images. Also, the application104takes an object like a sphere, and essentially moves it in space as a whole. Then each reproduced sample is tessellated into triangles and fed to the GPU102, impacting other graphics processing. The size of the accumulation buffer is a further constraint on the speed and quality of technique100. In addition, the accumulation buffer draws excess GPU fame buffer resources, trading off desired locality of textures in memory.

FIG. 2is a block diagram of one traditional pre-rendering motion blur technique200. Technique200involves a graphics processing unit (GPU)202and a software application204. The software application communicates with a host processor (not shown) and a memory (not shown) in a system. The memory may be shared with other components or functions. Technique200performs scene processing, and to apply motion blur, geometry amplification. Technique200preserves more sample details as compared to technique100. However, technique200imposes geometry amplification at the top of the graphics processing pipeline in the host processor, hence limiting graphics feature orthogonality, consuming additional time, and overloading the host processor to GPU bandwidth. The GPU202is required to perform unnecessary computations on a large number of vertices coming into it. As a consequence of these considerations, technique200is especially limited to applying motion blur to simple objects.

Techniques100and200are examples of prior techniques which are demanding of resources including memory and processing bandwidth. Prior techniques are forced to constrain the use of motion blur to limited transformation and/or application to simplified objects, such as spheres.

Because motion blur is traditionally controlled by a software applications which interface with fixed functionality in a GPU, known techniques suffer scalability in the presence of other graphics features, and place an additional burden on the human graphics programmer.

DETAILED DESCRIPTION

An embedded, programmable motion blur system and method is described herein. Embodiments include applying displacement on a vertex level and amplification on a primitive level within a graphics pipeline. Embodiments described herein provide more accurate motion blur functionality with improved scalability inside a graphics pipeline. The motion blur functionality is completely orthogonal to other graphics features. In addition, embodiments described herein do not overburden an existing graphics application or existing memory capacity.

One embodiment of a technique as described herein applies motion blur to any type of object, which is pre-tessellated into triangles. The transformation of the object from its initial position to its aperture extent is relatively generic, and includes, but is not limited to, translation, scaling, and rotation. Other types of transformations may also be employed in this or other embodiments of the present invention. Embodiments execute motion blur in the middle of the graphics processing pipeline, providing a significant improvement in graphics feature scalability and orthogonality. For example, both displaced and non-displaced geometries are treated similarly with regard to motion blur. Also, object (e.g. triangle) samples along the motion vector may adaptively be assigned a transparency value, based on time.

FIG. 3is a block diagram of a system300including a graphics processing unit (GPU)302according to an embodiment. System300includes a central processing unit (CPU)304, also referred to as a host processor. The CPU304communicates with a memory310, an input/output (I/O) unit308and GPU302via a memory hub306. The memory310includes memory usable by the GPU302and other components of the system300. The GPU302also includes various resources (not shown) including local memory, or graphics memory that is local to the GPU302. The memory hub306communicates with various system300components via one or more peripheral component interface (PCI)-express (PCI-E) buses. In various embodiments, one or more of the referenced components can be replicated, for example such that there are multiple GPUs302in the system300.

FIG. 4is a block diagram of the GPU302according to an embodiment. The GPU302includes various hardware and software components to provide specialized processing of graphics data. The GPU302is flexibly programmable to process data to produce a variety of results. The GPU302includes a vertex shader402, a geometry shader404, a rasterizer406, a pixel shader408, and a frame buffer410. The GPU302also includes resources412, including buffer resources and texture resources. The vertex shader402, the geometry shader404, and the pixel shader408use the resources412to store shared data.

In various embodiments, one or more of the referenced elements may be replicated, for example such that there are multiple geometry shaders404in the GPU302. The replication may be effected by programming hardware to duplicate functionality, by duplication of hardware, or any combination of the two.

The vertex shader receives a vertex as input and outputs a vertex. A vertex may have multiple attributes, including but not limited to position, normal, color, and texture coordinate (for example, an image can be applied later using the coordinate). In an embodiment that creates motion blur in a scene as described herein, the vertex shader applies displacement. For example, the vertex received represents a vertex in a scene that is being processed by the GPU302. The output vertex is the same vertex, but programmably transformed, including displacement.

The geometry shader404is a programmable unit that accepts primitives, such as, for example, a point, line, triangle or other polygon as input. Primitives are input to the geometry shader in the form of vertices, e.g. a single vertex for point, two vertices for a line, or three vertices for a triangle. Optionally, the geometry shader404may also receive the vertex data for the edge-adjacent primitives, e.g., an additional two vertices for a line, and an additional three vertices for a triangle. While the vertex shader is a one-to-one process (one vertex in and one vertex out) the geometry shader is a one-to-many process, receiving one primitive and outputting many primitives. In one embodiment, the geometry shader performs one-to-many triangle mapping, and thus provides geometry amplification within the graphics pipeline. As an example,FIG. 5Ashows a triangle502with vertices V0, V1and V2. The vertices of triangle502are input to the geometry shader404. The geometry shader runs application-specified shader code that generates vertices for output. Embodiments of the motion blur method described herein use a triangle such as triangle502as input to the geometry shader404.

FIG. 5Bis diagram of an output topology for the geometry shader404according to an embodiment. The geometry shader404is capable of outputting multiple vertices forming a single selected topology. Some geometry shader404output topologies available are triangle-strip, line-strip and point list, but the embodiment is not necessarily so limited.FIG. 5Bis a diagram of a triangle-strip topology504according to an embodiment. The vertices V0, V1, and V2are present, as well as vertices V3, V4, V5, and V6. The strip topology504provides for the reuse of vertices for increased efficiency. For example, a first triangle is formed of vertices V0, V1, and V2, another triangle is formed of V1, V3, and V2, another triangle is formed of V1, V4, and V3, and so on. An N-vertices strip form surface yields N−2 triangles.

The number of primitives output by the geometry shader404can vary freely within any invocation of the geometry shader404, though the maximum number of vertices that can be emitted may be declared statically in the shader code before hand. Strip topology lengths output from an invocation of the geometry shader404can be arbitrary. The geometry shader404code utilizes two primary topology statements: emit and cut. Each emit statement produces one vertex at the output. A cut statement indicates a break in a strip topology, and a new start for a primitive. An embodiment of motion blur shader code as further described below runs on the geometry shader404and generates a known number of triangles on the output. The output triangles are disjoint, and hence a cut statement takes place for every three vertices emitted.

The output of the geometry shader404may be fed to the rasterizer406and/or out to a resource buffer in memory412. Output fed to memory412is expanded to individual point/line/triangle lists. Output fed to the rasterizer406is also expanded to individual point/line/triangle lists.

In one embodiment the geometry shader404includes various levels of software to perform processing including, geometry processing, and vertex processing. The levels of software include high-level shader language (HLSL), and Open GL Shading Language (GLSL). HLSL and GLSL are high-level shading languages that can be analogized to high-level languages such as C++. In an embodiment, the high-level shading languages are compiled into the hardware of the GPU302to provide more efficiency, speed, flexibility and capability, including embedded or integrated motion blur capability.

Referring again toFIG. 4, the rasterizer406receives the amplified primitives that are output from the geometry shader404. The rasterizer406prepares an image for display or printing according to known methods.

The pixel shader408receives the output of the rasterizer408. A pixel shader is a graphics function that calculates effects on a per-pixel basis as known in the art.

The frame buffer410receives the output of the pixel shader408.

An algorithm for creating motion blur with the geometry shader404according to an embodiment will now be described with reference toFIGS. 6A and 6B.FIG. 6Ais a diagram of a triangle as it is linearly transformed from time0to time aperture according to an embodiment. Time0is the time at which a hypothetical camera aperture opens on a scene, and time aperture is the time at which the hypothetical camera aperture closes. A triangle with vertices V0, V1and V2is input to the geometry shader404at time0. The dotted lines form a volume that is called the convex hull. There can also be two convex hulls.FIG. 6Bis a diagram of two convex hulls defined by a triangle as it is linearly transformed from time0to time aperture.

Referring again toFIG. 6A, the triangles interposed between time0and time aperture are samples that show the intermediate triangles between time0and time aperture. The compiled shading code to execute on the geometry shader includes first triangle transformation information regarding transformations the triangle undergoes over the time period from time0to time aperture. The geometry shader uses the information to perform operations on the input triangle, including translation, scaling, and rotation.

The geometry shader404linearly transforms the input triangle to obtain its position and orientation at time aperture. Then, the convex hull is obtained from the triangle at time0and the triangle at time aperture. The geometry shader404constructs the triangle samples in the delimited convex hull at programmable intervals. The input triangle, samples and the triangle at aperture time are output to the rasterizer406.

Current software applications are not able to perform translation, scaling, and rotation as described because of prohibitive efficiency costs. Current software applications also avoid applying motion blur on a primitive (e.g., triangle) basis for the reasons previously mentioned. Current software application solutions to motion blur are limited to avoid prohibitively or unacceptably slow interaction speed. Embodiments of the invention can apply motion blur to different objects in a same field moving in different direction, for example by assigning processing of one object to one geometry shader, and assigning processing of the other object to another geometry shader.

Motion blur pseudo code for execution by the geometry shader404according to an embodiment is shown below.

The above description of illustrated embodiments of the invention is not intended to be exhaustive or to limit the invention to the precise form disclosed. While specific embodiments of, and examples for, the invention are described herein for illustrative purposes, various equivalent modifications are possible within the scope of the invention, as those skilled in the relevant art will recognize. The teachings of the invention provided herein can be applied to other systems, not only for the system including graphics processing as described above.

For example, a blurred image produced as described herein may be output to a variety of display devices, including computer displays that display moving pictures and printers that print static images.

The various operations described may be performed in a very wide variety of architectures and distributed differently than described. As an example, in a distributed system a server may perform some or all of the rendering process.

In other embodiments, some or all of the hardware and software capability described herein may exist in a printer, a camera or some other device. The motion blur techniques described herein may be applied as part of a process of constructing complete polygonal structures or geometries (e.g., including light parameters, etc.) from a video sequence. The geometries are then processed as described herein to include motion blur.

The elements and acts of the various embodiments described above can be combined to provide further embodiments. These and other changes can be made to the invention in light of the above detailed description.

In general, in the following claims, the terms used should not be construed to limit the motion blur method and system to the specific embodiments disclosed in the specification and the claims, but should be construed to include any processing systems that operate under the claims to provide motion blur processing. Accordingly, the motion blur method and system is not limited by the disclosure, but instead the scope of the motion blur method and system is to be determined entirely by the claims.

While certain aspects of the method and apparatus for motion blur processing are presented below in certain claim forms, the inventors contemplate the various aspects of the method and apparatus for motion blur processing in any number of claim forms. For example, while only one aspect of the method and apparatus for motion blur processing may be recited as embodied in computer-readable medium, other aspects may likewise be embodied in computer-readable medium. Accordingly, the inventors reserve the right to add additional claims after filing the application to pursue such additional claim forms for other aspects of the method and apparatus for motion blur processing.