Command buffers for web-based graphics rendering

The described embodiments provide a system that renders graphics for a computer system. During operation, the system loads a software client and a software service in the computing system. Next, the system receives a set of rendering commands from the software client in a command buffer, wherein the rendering commands include at least one of a state change command, a resource allocation command, a direct memory access (DMA) command, buffer data, and a synchronization command. Finally, the system uses the software service to render an image corresponding to the rendering commands by reading the rendering commands from the command buffer and executing the rendering commands.

RELATED APPLICATION

The subject matter of this application is related to the subject matter in a co-pending non-provisional application by inventors Robin Green, Evangelos Kokkevis, Matthew Papakipos and Gregg Tavares and filed 16 Jul. 2008 entitled “Web-Based Graphics Rendering System,” having Ser. No. 12/174,586.

The subject matter of this application is also related to the subject matter in a co-pending non-provisional application by J. Bradley Chen, Matthew T. Harren, Matthew Papakipos, David C. Sehr, and Bennet S. Yee, entitled “Method for Validating an Untrusted Native Code Module,” having Ser. No. 12/117,634, and filing date 8 May 2008.

The subject matter of this application is also related to the subject matter in a co-pending non-provisional application by J. Bradley Chen, Matthew T. Harren, Matthew Papakipos, David C. Sehr, Bennet S. Yee, and Gregory Dardyk, entitled “Method for Safely Executing an Untrusted Native Code Module on a Computing Device,” having Ser. No. 12/117,650, and filing date 8 May 2008.

BACKGROUND

The present embodiments relate to graphics rendering techniques. More specifically, the present embodiments relate to techniques for using command buffers in web-based rendering of graphics.

2. Related Art

Computer systems often include a number of native applications that require complex three-dimensional (3D) scenes to be rendered, such as computer games and computer-aided design (CAD) systems. To render 3D scenes, these native applications may use graphics application programming interfaces (APIs) that direct calculations related to graphics rendering to dedicated graphics processing units (GPUs). The additional computational power provided by these GPUs can greatly improve graphics quality and throughput.

Web applications, which have become more prevalent in recent years, are typically written in scripting languages that are unable to utilize low-level graphics APIs that provide graphics hardware acceleration. Instead, graphics rendering for web applications is typically performed by CPUs instead of GPUs. The software-based nature of web-based graphics rendering may thus limit the graphics capabilities of web applications. However, unlike native applications, web applications provide a number of advantages. For example, web applications are capable of executing on multiple platforms, do not require installation, and can be more secure than native applications.

The tradeoff between web application security and native graphics performance may be addressed using a browser plugin that renders graphics for web applications by interfacing with a local graphics hardware device (e.g., a GPU). Such a plugin may correspond to a complex software system that includes various mechanisms for obtaining scene information from the web applications; storing the scene information; processing the scene information using transforms, effects, and shaders; and sending commands to the graphics hardware for rendering the scene. Consequently, the plugin itself may include a number of potential security vulnerabilities that may be exploited by other applications and/or bugs that may lead to system failures.

Hence, what is needed is a mechanism for safely executing the plugin while maintaining communication between the plugin and graphics hardware.

SUMMARY

Some embodiments provide a system that renders graphics for a computer system. During operation, the system loads a software client and a software service into the computing system. Next, the system receives a set of rendering commands from the software client in a command buffer, wherein the rendering commands include at least one of a state change command, a resource allocation command, a direct memory access (DMA) command, buffer data, and a synchronization command. Finally, the system uses the software service to render an image corresponding to the rendering commands by reading the rendering commands from the command buffer and executing the rendering commands.

In some embodiments, the system also writes the buffer data to a shared memory buffer using the software client and further executes the rendering commands using the software service by transferring the buffer data to a resource from the shared memory buffer.

In some embodiments, the resource corresponds to at least one of a frame buffer, a vertex buffer, a texture buffer, and an index buffer.

In some embodiments, executing the rendering commands involves asynchronously creating the resource from the resource allocation command in the command buffer.

In some embodiments, executing the rendering commands involves formatting the rendering commands and sending the formatted rendering commands to a graphics-processing unit (GPU) of the computing system, which renders the image.

In some embodiments, the system also synchronizes interaction between the software client and the software service.

In some embodiments, synchronizing interaction between the software client and the software service involves at least one of:(i) setting or reading a pointer associated with the command buffer;(ii) setting or reading a token associated with the command buffer; and(iii) synchronizing interaction between the software client and a GPU of the computing system.

In some embodiments, the software client corresponds to at least one of a native code module, a web application, and a graphics library.

In some embodiments, the software client includes a scene graph renderer.

In some embodiments, the command buffer is a ring buffer.

In some embodiments, the ring buffer includes a put pointer corresponding to a last rendering command from the rendering commands written by the software client and a get pointer corresponding to a next rendering command from the rendering commands to be read by the software service.

In some embodiments, the get pointer is stored in shared memory between the software client and the software service.

DETAILED DESCRIPTION

Embodiments provide a method and system for rendering graphics in a computing system. The computing system may be, for example, a personal computer (PC), a mobile phone, a personal digital assistant (PDA), a graphing calculator, a portable media player, a global positioning system (GPS) receiver, and/or another electronic computing device. More specifically, embodiments provide a method and system for rendering graphics using command buffers. The command buffers allow asynchronous communication to occur between a software client and a software service. Furthermore, a command buffer interface may facilitate synchronous communication between the software client and the software service.

In one or more embodiments, the software client is executed as a native code module in a secure runtime environment. The secure runtime environment may enable safe execution of the software client on the computing system by validating the native code module and restricting access to system resources. Communication with graphics hardware may be provided by the software service, which may be installed locally on the computing system as a trusted code module. Graphics rendering may thus occur by writing rendering commands to the command buffer using the software client and reading rendering commands from the command buffer using the software service. The software service may then format the rendering commands and send the formatted rendering commands to a GPU of the computing system for rendering on the GPU.

In one or more embodiments, the command buffer is used to facilitate interaction between a web application implemented using a native code module and the GPU. In other words, the native code module may write rendering commands directly into a hardware command buffer that is used by the GPU to render images for the web application. As a result, the web application may directly request execution of rendering commands by the GPU using the native code module and command buffer.

FIG. 1Ashows a schematic of an embodiment of a system. More specifically,FIG. 1Ashows a system for command buffer-based rendering of graphics on a computing system102. The system includes a software client106and a software service108interacting through a command buffer interface104, a command buffer110, and a shared memory buffer112. The system also includes a rendering engine118and graphics-processing unit (GPU)120for rendering an image associated with software client106. Each component of the system is described in further detail below.

Computing system102may correspond to an electronic device that provides one or more services or functions to a user. For example, computing system102may operate as a mobile phone, personal computer (PC), global positioning system (GPS) receiver, portable media player, personal digital assistant (PDA), and/or graphing calculator. In addition, computing system102may include an operating system (not shown) that coordinates the use of hardware and software resources on computing system102, as well as one or more applications that perform specialized tasks for the user. For example, computing system102may include applications such as an email client, address book, document editor, web browser, and/or media player. To perform tasks for the user, applications may obtain the use of hardware resources (e.g., processor, memory, I/O components, wireless transmitter, GPU120, etc.) on computing system102from the operating system, as well as interact with the user through a hardware and/or software framework provided by the operating system.

To enable interaction with the user, computing system102may include one or more hardware input/output (I/O) components, such as a pointing device and a display screen. Each hardware I/O component may additionally be associated with a software driver (not shown) that allows the operating system and/or applications on computing system102to access and use the hardware I/O components.

Those skilled in the art will appreciate that the functionality of certain applications on computing system102may depend on the applications' ability to utilize graphics hardware acceleration on computing system102. For example, the graphics quality of a computer game may be based on the ability of the computer game to communicate with GPU120.

Those skilled in the art will also appreciate that certain applications on computing system102may be unable to utilize GPU120in graphics rendering due to limitations in resource access and/or performance. In particular, graphics rendering for web applications may be performed using software that executes on a CPU of computing system102rather than GPU120. As a result, graphics in web applications may be slow and/or suboptimal compared to graphics in native applications that employ graphics hardware acceleration.

To enable graphics support and graphics hardware acceleration for web applications, operations related to graphics processing may be offloaded to a web browser plugin in computing system102. The plugin may expose the capabilities of GPU120to the web applications, thus allowing the web applications to utilize graphics hardware acceleration, including the application of vertex and pixel shaders. Plugin-based graphics hardware acceleration for web applications is described in a co-pending non-provisional application by inventors Robin Green, Evangelos Kokkevis, Matthew Papakipos and Gregg Tavares and filed 16 Jul. 2008 entitled “Web-Based Graphics Rendering System,” having Ser. No. 12/174,586, which is incorporated herein by reference.

However, the plugin may correspond to a complex software system that includes a number of security vulnerabilities. Furthermore, the plugin may include a scene graph renderer114that includes limited ability to accept low-level graphics rendering commands. To increase the security and versatility of the plugin, the plugin may be implemented using a command buffer interface104between two software components: software client106and software service108. Software client106and software service108may execute on separate processes within computing system102. Furthermore, software client106and software service108may communicate and implement graphics rendering using a combination of asynchronous and synchronous mechanisms, as explained below.

As shown inFIG. 1A, scene graph renderer114is implemented using software client106. In other words, software client106may include functionality to obtain and/or store a graphics model to be rendered for an application, such as a web application. The graphics model may include, for example, a set of shapes composed of triangles or polygons, one or more light sources, a camera, and/or one or more rendering effects (e.g., shaders, culling, blending, etc.). As described in the above-referenced application, the graphics model may additionally be stored in one or more data structures, such as scene graphs, buffers, and/or effects.

To render an image corresponding to the graphics model, software client106may transmit a set of rendering commands to software service108. A command parser116in software service108may then format and/or process the rendering commands. The formatted and/or processed rendering commands may be passed from software service108to rendering engine118, which executes the rendering commands by communicating with GPU120. For example, the rendering commands may correspond to low-level commands that are parsed by command parser116and translated into function calls into a rendering engine such as a Direct3D (Direct3D™ is a registered trademark of Microsoft Corp.) or OpenGL (OpenGL™ is a registered trademark of Silicon Graphics, Inc.) renderer. The function calls may then be translated into graphics processing instructions for GPU120by the renderer. Alternatively, rendering engine118may interface directly with GPU120by issuing hardware instructions to GPU120, or command buffer interface104may correspond to a direct interface with GPU120, as described in further detail below with respect toFIG. 1B.

Those skilled in the art will appreciate that interaction between software client106and software service108may require synchronous inter-process communication (IPC) mechanisms such as message passing, remote procedure calls (RPC), and/or sockets. Such IPC mechanisms may be provided by command buffer interface104in the form of function calls between software client106and software service108. On the other hand, IPC techniques alone may not pass individual rendering commands from software client106to software service108at a rate that is high enough to enable real-time computer graphics. For example, an RPC mechanism may be unable to support the number of function calls and/or rendering commands (e.g., one million) required to render a three-dimensional (3D) scene for a computer game at 60 frames per second.

To enable real-time rendering of 3D graphics, command buffer interface104may also include mechanisms by which software client106and software service108may operate asynchronously. More specifically, asynchronous communication between software client106and software service108may be enabled using command buffer110and shared memory buffer112.

Command buffer110may be created upon loading and/or initialization of software client106and/or software service108. For example, command buffer110may be created by software service108and passed to software client106through command buffer interface104. Moreover, both software client106and software service108may map command buffer110into their respective address spaces for direct access to command buffer110.

In one or more embodiments, command buffer110includes functionality to store rendering commands from software client106to software service108. As described above, such rendering commands may correspond to low-level graphics rendering commands that are generated on the order of tens of thousands of rendering commands per frame. Rendering commands stored in command buffer110may include, for example, state change commands, resource allocation commands, direct memory access (DMA) commands, buffer data, and/or synchronization commands. Most rendering commands may not require a return value, thus allowing issuance of the rendering commands by software client106and execution of the rendering commands by software service108to be independent and/or asynchronous.

Command buffer interface104may allow software client106to write sets of rendering commands to command buffer110. As the rendering commands are written to command buffer110, the rendering commands may be read by command parser116and executed by software service108and/or rendering engine118. One or more images corresponding to the rendering commands may thus be produced by GPU120through the transmission of rendering commands from software client106to software service108using command buffer110.

As described above, rendering engine118may correspond to an OpenGL, Direct3D, and/or other renderer that interfaces with GPU120to render graphics. Furthermore, the functionality of rendering engine118may be accessed through an interface (e.g., application programming interface (API)) with other software components, such as software service108. Consequently, software service108may execute rendering commands from command buffer110by reading and/or interpreting the rendering commands using command parser116and making corresponding function calls to rendering engine118through the interface with rendering engine118.

In other words, software service108may format the rendering commands from command buffer110such that the rendering commands are understood by rendering engine118and conveyed to GPU120by rendering engine118. The formatting may be accomplished by mapping each rendering command to a command table that stores API calls to rendering engine118for each rendering command. The rendering command may then be executed by making the API calls to rendering engine118using parameters and/or arguments accompanying the rendering command in command buffer110. Encoding schemes for rendering commands are discussed in further detail below with respect toFIGS. 2A-2B.

Synchronization of actions between software client106and software service108may also be provided by command buffer interface104. In particular, command buffer interface104may include mechanisms for synchronizing the writing of rendering commands to command buffer110by software client106with the reading and execution of rendering commands from command buffer110by software service108. In other words, command buffer interface104may provide concurrency control mechanisms between software client106and software service108to enforce the correct execution of rendering commands by software service108, rendering engine118, and/or GPU120.

In one or more embodiments, command buffer110is implemented as a ring buffer (i.e., circular buffer). Furthermore, command buffer110may include a set of pointers (e.g., offsets into command buffer110) that denote the effective start and end of the ring buffer. More specifically, command buffer110may include a “put” pointer that represents the last rendering command written to command buffer110by software client106. Command buffer110may also include a “get” pointer that represents the next rendering command to be read from command buffer110by software service108. Consequently, command buffer interface104may provide mechanisms for controlling the put pointer by software client106, controlling the get pointer by software service108, and/or reading the values of either pointer by software client106and/or software service108.

For example, the put and get pointers may both be stored in the address space of software service108. Software client106may write to the put pointer and read the get pointer through a message passing and/or RPC mechanism provided by command buffer interface104. However, because the RPC mechanism may be associated with significant overhead, software client106may only update the put pointer after writing a batch of (e.g., several) rendering commands to command buffer110. Similarly, software client106may cache the value of the get pointer and only update the cached value when the value of the put pointer nears the value of the get pointer.

Alternatively, each pointer may be stored in the address space of the respective controlling entity. In other words, the put pointer may be stored in the address space of software client106, and the get pointer may be stored in the address space of software service108. Command buffer interface104may then provide mechanisms for sending the value of the put pointer to software service108and/or the value of the get pointer to software client106. Finally, the get pointer may be stored in shared memory (e.g., shared memory buffer112) that may be directly accessed by both software client106and software service108to reduce the need for IPC between software client106and software service108. However, to store the get pointer in shared memory, command buffer interface104may also implement access control restrictions that prevent software client106from writing to the memory address corresponding to the get pointer.

Those skilled in the art will appreciate that basic ring buffer synchronization may be implemented using the get and put pointers of command buffer110. In other words, rendering commands from the get pointer to the put pointer may correspond to rendering commands that have not been read or executed by software service108and should not be modified. Similarly, rendering commands from the put pointer to the get pointer may correspond to rendering commands that have been read and/or executed and can be modified. Software client106may continue writing rendering commands into command buffer110until the value of the put pointer reaches the value of the get pointer, indicating that command buffer110is full. Along the same lines, software service108may continue reading and/or executing rendering commands from command buffer110until the value of the get pointer reaches the value of the put pointer, indicating that command buffer110is empty.

Command buffer interface104may also enable synchronization of software client106and rendering engine118and/or GPU120. Because graphics rendering through renderers such as OpenGL and Direct3D renderers and/or GPU120may also be asynchronous, notifications (e.g., rendering commands) regarding state changes in rendering engine118and/or GPU120may be useful to software client106. For example, software client106may use notifications from GPU120, such as OpenGL fences, to perform tasks such as occlusion queries and/or management of dynamic data buffers. GPU120execution synchronization is described in further detail below with respect toFIG. 3.

In one or more embodiments, resource allocation is managed using rendering commands that are specified by software client106in command buffer110and executed by software service108. In particular, software client106may write a rendering command that specifies the creation and allocation of a particular type of resource (e.g., vertex buffer, index buffer, frame buffer, texture buffer, etc.) using an index (e.g., identifier) for the resource and a size of the resource. Upon reaching the rendering command, software service108may allocate space for the resource (e.g., on GPU120) and store a handle for the resource in a resource table under the index specified by software client106. Subsequent rendering commands from software client106to software service108involving the resource may thus be referenced by the resource's index in the resource table. Furthermore, because resources may be asynchronously allocated and managed through rendering commands in command buffer110, delays associated with IPC mechanisms for resource creation may be avoided.

For example, software client106may write a rendering command into command buffer110for the creation of a vertex buffer. Software service108may then execute the rendering command by allocating the vertex buffer in GPU120through one or more API calls and store the handle for the vertex buffer in the resource table under the index specified in the rendering command. Software client106may also write rendering commands into command buffer110for clearing, reading from, and/or writing to the vertex buffer by software service108using the vertex buffer's index.

Those skilled in the art will appreciate that graphics rendering (e.g., 3D graphics rendering) may involve the storage and transmission of large amounts of buffer data. For example, rendering of a single frame in a 3D computer game may require the storage and transmission of several to tens of megabytes of vertex, texture, index, and/or frame buffer data between software client106and software service108. Such buffer data may be written to command buffer110by software client106along with commands for processing and/or rendering the data. However, the buffer data may exceed the allocated size of command buffer110and/or preclude rendering commands from being stored in command buffer110until the buffer data is read from command buffer110by software service108.

To address the storage limitations of command buffer110, command buffer interface104may include functionality to allocate a shared memory buffer112for storing buffer data associated with graphics rendering. Shared memory buffer112may include frame, vertex, texture, and/or index buffer data to be transmitted from software client106to software service108. In one or more embodiments, shared memory buffer112is created by software client106during initialization of software client106and shared with software service108using an IPC mechanism (e.g., RPC, message passing, socket call, etc.). In addition, software client106may create multiple shared memory buffers112to organize the allocation of resources and/or store rendering data for multiple software clients. Alternatively, shared memory buffer112may be created by software service108during initialization of software service108and shared with software client106using IPC. The shared nature of shared memory buffer112may additionally allow both software client106and software service108to directly access (e.g., read from, write to, etc.) shared memory buffer112from their respective address spaces.

To transmit data to software service108using shared memory buffer112, software client106may write the data directly to shared memory buffer112. Software client106may then write a rendering command to command buffer110that prompts software service108to transfer the data in shared memory buffer112to a resource managed by software service108. After software service108executes the transfer command, the data in shared memory buffer112may be modified or freed by software client106.

For example, software client106may write several megabytes of vertex buffer data into shared memory buffer112. Software client106may then write a rendering command into command buffer110for transferring the vertex buffer data in shared memory buffer112into a vertex buffer managed by software service108(e.g., stored in a resource table managed by software service108). The transfer command may include the offset of the vertex buffer data in shared memory buffer112and the index of the vertex buffer in the resource table. To execute the transfer command, software service108may use the offset of the vertex buffer data to locate the vertex buffer data and the index of the vertex buffer to copy the vertex buffer data to the vertex buffer. After the vertex buffer data is copied, software client106may modify or free the vertex buffer data from shared memory buffer112. For example, software client106may write more vertex buffer data for copying into the vertex buffer into shared memory buffer112, or software client106may write index buffer data to be copied into an index buffer into shared memory buffer112.

In one or more embodiments, state changes produced by software service108are tracked using a token. Such state changes may involve, for example, transfer of data in shared memory buffer112, rendering engine118, and/or GPU120. The token may be stored in the address space of software service108and updated by software service108with a particular identifier (e.g., an integer) to reflect the completed execution of a rendering command. The identifier may further be provided by software client106through a “SetToken” rendering command written to command buffer110. Software client106may then obtain the value of the token using a synchronous (e.g., socket) call with software service108. In other words, software client106may instruct software service108to track certain changes in state by executing the “SetToken” rendering command after executing one or more rendering commands that cause each of the changes in state.

For example, software client106may write a “SetToken” rendering command that increments the token after each important rendering command (e.g., transfer commands, GPU120state change commands, etc.) in command buffer110. Upon executing an important rendering command in command buffer110, software service108increments the token to indicate that the important rendering command has been completed. Consequently, the token may allow software client106to obtain information about multiple state changes (e.g., executed rendering commands) in a single IPC query to software service108rather than repeatedly requesting the value of the get pointer in more frequent IPC queries. More specifically, the token may provide another synchronization mechanism between various independently executing components of computing system102, such as software client106, software service108, rendering engine118, and/or GPU120.

As mentioned previously, software service108may operate in conjunction with multiple software clients. Such software clients may include, for example, web applications, graphics libraries, and/or native code modules. As a result, software service108may include multiple command buffer interfaces to facilitate secure operation with different types of software clients. For example, command buffer interface104may include bindings for general-purpose languages such as C++ as well as higher-level languages for web applications such as Javascript (Javascript™ is a registered trademark of Sun Microsystems, Inc.). Command buffer interface104may also allow graphics libraries such as OpenGL and Direct3D to be re-implemented as software client106. For example, command buffer interface104may connect an OpenGL software client106that communicates with software service108to render using a Direct3D rendering engine118and vice versa.

Furthermore, the independent execution of software service108and software client106may facilitate safer execution of applications that request GPU120resources. In particular, the parsing and processing of rendering commands by software service108prior to communicating with rendering engine118and/or GPU120may prevent applications that run as software clients from directly accessing GPU120, portions of memory, and/or other resources on computing system102. As a result, the system ofFIG. 1Amay reduce the incidence of system crashes, graphics driver bugs, file system corruption, and/or other security-related issues. The security and safety of graphics rendering may additionally be increased by executing software client106in a native code module, as discussed below with respect toFIGS. 1B-1C.

FIG. 1Bshows a schematic of an embodiment of a system. More specifically,FIG. 1Bshows a system for providing hardware-accelerated graphics rendering to a web application124. As shown inFIG. 1B, web application124executes within a web browser122on computing system102. Web application124may be obtained from a server using a network connection with the server and loaded within web browser122. For example, web application124may be downloaded from an application server over the Internet by web browser122. Furthermore, web application124may execute on computing system102regardless of the type of platform (e.g., operating system, drivers, etc.) associated with computing system102.

Once loaded, web application124may provide features and user interactivity comparable to that of native applications on computing system102. For example, web application124may function as an email client, document editor, media player, computer-aided design (CAD) system, and/or computer game. Web application124may also include dynamic user interface elements such as menus, buttons, windows, sub-windows, icons, animations, and/or other graphical objects that emulate analogous user interface elements in native applications. In other words, web application124may correspond to a rich Internet application (RIA).

More specifically, web application124may include graphics rendering capabilities that are typically associated with native applications, such as graphics hardware acceleration using GPU120. To render graphics using GPU120, a native code module132associated with web application124may be used as software client106. Like web application124, native code module132may be obtained from one or more servers by web browser122. Furthermore, native code module132may be executed natively within a plugin126associated with web browser122to provide hardware-accelerated graphics rendering capabilities to web application124. Alternatively, some or all of web application124may execute within native code module132.

In one or more embodiments, plugin126includes a variety of mechanisms to ensure the safe execution of native code module132. In particular, native code module132may be validated by a validator128provided by plugin126prior to execution. Native code module validation is described in a co-pending non-provisional application by inventors J. Bradley Chen, Matthew T. Harren, Matthew Papakipos, David C. Sehr, and Bennet S. Yee, entitled “Method for Validating an Untrusted Native Code Module,” having Ser. No. 12/117,634, and filing date 8 May 2008, which is incorporated herein by reference.

Once native code module132is validated, native code module132may be loaded into a secure runtime environment130provided by plugin126. Native code execution in a secure runtime environment is described in a co-pending non-provisional application by inventors J. Bradley Chen, Matthew T. Harren, Matthew Papakipos, David C. Sehr, Bennet S. Yee, and Gregory Dardyk, entitled “Method for Safely Executing an Untrusted Native Code Module on a Computing Device,” having Ser. No. 12/117,650, and filing date 8 May 2008, which is incorporated herein by reference.

Furthermore, because native code module132may include binary code that runs directly on hardware, native code module132may be platform independent with respect to the operating system of computing system102, web browser122, and/or other software components on computing system102. As described in the above-referenced applications, plugin126and/or native code module132may also include mechanisms for executing on a variety of instruction set architectures, including the use of “fat binaries” and binary translators.

As shown inFIG. 1B, native code module132communicates directly with GPU120through command buffer interface104. In other words, native code module132may write GPU commands and buffer data directly into a hardware command buffer110and/or shared memory112located on GPU120. Alternatively, native code module132may write GPU commands and/or buffer data into memory locations that are directly accessible by GPU120using function calls provided by command buffer interface104(e.g., a GPU interface). Consequently, the system ofFIG. 1Bprovides a mechanism for communication between GPU120and web application124while ensuring the safe execution of rendering commands using native code module132, secure runtime environment130, and/or validator128.

FIG. 1Cshows a schematic of an embodiment of a system in accordance with an embodiment. In particular,FIG. 1Cshows a system for implementing a secure plugin138that provides graphics hardware acceleration for web applications on computing system102. The functionality of plugin138is described in a co-pending non-provisional application by inventors Robin Green, Evangelos Kokkevis, Matthew Papakipos and Gregg Tavares and filed 16 Jul. 2008 entitled, “WEB-BASED GRAPHICS RENDERING SYSTEM,” having Ser. No. 12/174,586, which is incorporated herein by reference.

As shown inFIG. 1C, plugin138includes native code module134executing in secure runtime environment130. As described above, native code module134may function as a safely executing software client106. As a result, native code module134may provide a scene graph renderer (e.g., scene graph renderer114ofFIG. 1A) to web applications that use plugin138. Native code module134may then communicate with a trusted code module136using command buffer interface104, command buffer110, and/or shared memory112. As a result, trusted code module136may implement the functionality of software service108.

In other words, native code module134may correspond to untrusted code that executes safely within secure runtime environment130. However, because secure runtime environment130may not allow communication with hardware devices such as GPU120, native code module134may use command buffer interface104to interact with trusted code module136, which executes outside secure runtime environment130and includes the capability to communicate with GPU120. Trusted code module136may also use command buffer interface104to process rendering commands from native code module134and to send the rendering commands to rendering engine118and/or GPU120. Because the majority of plugin138may be implemented and executed using native code module134, plugin138may operate in a manner that minimizes the risk of crashes, bugs, and/or security vulnerabilities in computing system102.

FIG. 2Ashows a command200(e.g., rendering command). In particular,FIG. 2Ashows an encoding scheme for a rendering command200that may be written by a software client (e.g., software client106ofFIG. 1A) to a command buffer. Rendering command200may then be read and decoded from the command buffer by a software service (e.g., software service108ofFIG. 1A) for execution on a GPU.

As shown inFIG. 2A, command200includes a size202, an identifier204, and a set of arguments206-208. Size202and identifier204may be stored on the same element of the command buffer, while arguments206-208may each be stored on an element of the command buffer. For example, if elements in the command buffer correspond to 32-bit words, size202may occupy the upper eight bits of an element and identifier204may occupy the lower 24 bits of the element. Arguments206-208may also be padded to 32 bits for storage in the command buffer. Alternatively, multiple arguments may also be stored on the same element, as discussed below with respect toFIG. 2B.

Size202may represent the total size of command200in the number of elements in the command buffer taken up by command200. In other words, size202may correspond to the number of arguments206-208in command200incremented by one. As a result, if size202is stored in the upper eight bits of a 32-bit element, command200may not exceed 255 elements. Identifier204may allow command200to be identified and executed by the software service. For example, identifier204may correspond to an index in a global command table that is managed by the software service. The index may map to a set of function calls that may be made by the software service to a rendering engine or GPU to carry out the functionality of command200. Alternatively, command200may correspond to a resource allocation, DMA, and/or synchronization command that may be completed solely by the software service. Arguments206-208may correspond to the parameters of command200.

Command220includes a size222of “4” (e.g., four elements), an identifier224corresponding to “SetVertexSource,” and five arguments226-234(e.g., “Index,” “Format,” “Stride,” “Resource Index,” and “Offset”). Unlike command210, command220includes three arguments226-230in the same element of the command buffer. For example, in a 32-bit word element, arguments226-228may each occupy eight bits, while argument230may occupy 16 bits. Consequently, shorter arguments in a command (e.g., command220) may be stored in the same command buffer element of the command buffer. Decoding of commands and arguments may be accomplished by obtaining the encoding scheme of each command by looking up the command's index and/or identifier in the global command table.

FIG. 3shows an exemplary command buffer300. As described above, command buffer300may correspond to a ring buffer that includes a get302pointer and a put304pointer, as well as a series of rendering commands. Get302may correspond to the next rendering command to be read by a software service executing the rendering commands, and put304may correspond to the last rendering command written to command buffer300by a software client.

Rendering commands between get302and put304(e.g., “SetTextureBufferData,” “SetToken3,” “Draw,” “SetFence,” “FinishFence,” “SetToken4”) may correspond to commands that are to be executed by the software service, while rendering commands not between get302and put304may correspond to commands that have already been executed by the software service. As a result, rendering commands outside of the commands between get302and put304may be rewritten with new rendering commands by the software client. For example, the software client may add two new rendering commands to command buffer300by writing the new rendering commands to the elements containing “CreateEffect” and “SetEffect” and updating put304to point to the location of the “CreateEffect” command.

Rendering commands in command buffer300may also reference a shared memory buffer312that is also accessible to the software client and software service. In particular, vertex data306, index data308, and texture data310may be written in bytes to shared memory buffer312by the software client. The software client may then write transfer commands to command buffer300that reference the data. In particular, “SetVertexBufferData” references vertex data306, “SetIndexBufferData” references index data308, and “SetTextureBufferData” references texture data310. When the software service reaches a transfer command, the software service may use the reference (e.g., offset) and/or other arguments (e.g., length, resource index, etc.) in the transfer command to locate the relevant data and copy the data to a resource managed by the software service. For example, “SetVertexBufferData” may copy vertex data306to a vertex buffer, “SetIndexBufferData” may copy index data308to an index buffer, and “SetTextureBufferData” may copy texture data310to a texture buffer.

After executing a transfer command in command buffer300, the software service may set a token indicating completion of the data transfer. For example, after executing “SetVertexBufferData,” the software service may execute “SetToken1,” which may set the token to the value “1.” After executing “SetIndexBufferData,” the software service may set the token to the value “2” using the command “SetToken2.” After executing “SetTextureBufferData,” the software service may set the token to the value “3” using the command “SetToken3.” The software client may determine the progress of the software service by obtaining the value of the token through an IPC mechanism rather than repeatedly obtaining the value of get302.

Command buffer300may also include commands for synchronizing the software client and the GPU rendering the image corresponding to the rendering commands. In particular, “SetFence” may correspond to a synchronization command that can be queried for completion of a rendering command by the GPU. For example, “SetFence” may prompt the software service to set an OpenGL fence with the GPU. Similarly, “FinishFence” may not return until the GPU has finished executing the condition specified in “SetFence.” The software client may also write “SetToken4” to command buffer300after “SetFence” as a state change notification indicating that the fence has finished executing. The software client may subsequently receive the state change notification as an updated token value (e.g., “4”) after the software service executes “SetToken4” in command buffer300.

FIG. 4shows a flowchart illustrating the process of rendering graphics in a computing system. In one or more embodiments, one or more of the steps may be omitted, repeated, and/or performed in a different order. Accordingly, the specific arrangement of steps shown inFIG. 4should not be construed as limiting the scope of the technique.

Initially, a software client and a software service are loaded (operation402) in the computing system. The software client and software service may correspond to independently executing software components of the computing system. In addition, loading of the software client and software service may involve creating and sharing a command buffer and/or one or more shared memory buffers between the software client and software service. In particular, each buffer (e.g., command buffer, shared memory buffer) may be created by either the software client or software service. The buffer may then be provided to the other software component using an IPC mechanism and mapped to both components' address spaces. As a result, the command buffer and shared memory buffer(s) may be directly accessed by both the software client and the software service.

Next, one or more rendering commands are received from the software client in the command buffer (operation404). The rendering commands may correspond to state change commands, resource allocation commands, DMA commands, buffer data, and/or synchronization commands. The rendering commands may be written to the command buffer by the software client to enable asynchronous interaction between the software client and the software service. After the rendering commands are written to the command buffer by the software client, a put pointer in the command buffer is updated by the software client. The new value of the put pointer may signal the software service to begin reading rendering commands from the command buffer (operation406) in the order in which the rendering commands were written to the command buffer by the software client.

Each rendering command may then be formatted (operation408) and executed (operation410) by the software service and/or a GPU on the computing system. For example, the rendering command may be encoded with an identifier that maps to a global command table managed by the software service. The software service may use the global command table to decode the command in the command buffer and obtain a set of function calls to the GPU that implement the functionality of the rendering command. In addition, the software service may make the function calls using arguments stored with the rendering command in the command buffer. On the other hand, certain rendering commands, such as resource allocation commands, DMA commands, and/or state change commands, may not require GPU execution and may be carried out by the software service. For example, the software service may execute a transfer command by copying data in the shared memory buffer to a resource such as a vertex buffer.

The software client and/or software service may continue executing if additional rendering commands are available (operation412). In particular, the software client may continue writing commands to the command buffer (operation404) and updating the put pointer. The software service may concurrently read each rendering command (operation406), format the rendering command (operation408), and execute the rendering command (operation410). As described above, synchronization (e.g., IPC) mechanisms using a get pointer, put pointer, and/or token may be used to synchronize the execution of the software client and software service. One or more images related to the rendering commands may thus be produced by the combined synchronous and asynchronous interaction of the software client and the software service.