Abstract:
A method and system for automating unit performance testing in integrated circuit design is disclosed. One embodiment of the present invention sets forth a method, which includes the steps of generating a first performance data for the unit to operate on a workload, embedding the first performance data in the workload for a register transfer level (RTL) implementation of the unit to operate on, and determining whether the expected performance of the unit is achieved based on the comparison between the first performance data and a second performance data, wherein the second performance data is generated after the RTL implementation of the unit operates on the workload.

Description:
BACKGROUND OF THE INVENTION 
     1. Field of the Invention 
     Embodiments of the present invention relate generally to integrated circuit design and more specifically to a method and system for automating unit performance testing in integrated circuit design. 
     2. Description of the Related Art 
     Unless otherwise indicated herein, the approaches described in this section are not prior art to the claims in this application and are not admitted to be prior art by inclusion in this section. 
     Integrated circuit (IC) design involves the creation of electronic components, such as transistors, resistors, capacitors, and the metallic interconnect of these components onto a piece of semiconductor. Broadly speaking, digital IC design can be divided into three phases: 1) electronic system-level (ESL) design phase, 2) register transfer level (RTL) design phase, and 3) physical design phase. In the ESL phase, the user functional specification is defined and verified. In the RTL design phase, this user functional specification is converted into RTL description, which specifies in detail how each bit of the IC should behave on every clock cycle. Lastly, in the physical design phase, a chip design is generated based on the RTL description and a library of available logic gates. Here, issues such as which logic gates to use, where to place the gates, and how to wire them together are addressed. After resolving these issues, the chip is taped out, and the design data is converted into photomasks. 
     As ICs become increasingly complex and contain a large number of units, designing and verifying such ICs becomes more and more difficult.  FIG. 1A  is a conventional approach of verifying both the functional correctness and the performance correctness of a chip  100  prior to entering the physical design phase. Suppose the chip  100  is a graphics processing unit. In step  102 , system level stimuli are generated to be fed into the functional model verification environment (FModel), which is designed to generate a set of expected test results according to the user functional specification for the entire chip  100  in step  104 . The system level stimuli here generally refer to graphics related commands. In step  106 , the full-chip RTL implementation generates a set of actual test results also based on the same system level stimuli. Then, the expected test results are compared to the actual test results in step  108  to determine whether the chip  100  as a whole performs the specified functions as expected. Similarly, the same system level stimuli are fed into the performance simulator in step  110  and also the full-chip RTL implementation of the chip  100  in step  112 . The performance simulator estimates a number of clock cycles for the entire chip  100  to respond to or operate on the system level stimuli. This estimated clock cycle count is then compared with the actual clock cycle count that the full-chip RTL implementation takes to determine whether the performance of the full-chip RTL implementation is considered to be correct. 
     The aforementioned approach has several shortcomings. One, given the complexity of the chip and also the traffic streams that it may receive, developing and deploying the programs and environment to effectively and thoroughly verify the entire chip  100  is difficult and time consuming, because such efforts require the consideration of all possible operating paths in the chip  100 . Two, even if the full-chip tests are developed and deployed and even if they successfully detect deviations from the expected results, they still lack any flexibility to efficiently identify the failing units in the chip that cause such deviations. Three, to test on a full-chip level also means that such testing cannot begin until the RTL implementation of the chip  100  is completed. This serial dependency between the RTL implementation of the chip and the testing of the chip often leads to either insufficient amount of testing on the chip or intolerable delays in releasing the chip. 
       FIG. 1B  is yet another conventional approach in verifying the functional correctness of the chip  100  prior to entering the physical design phase but at the unit level of the chip  100 . Here, the system level stimuli generated in step  122  still go into the FModel in step  124 . However, unlike step  104  in  FIG. 1A , the FModel generates interface transactions associated with the stimuli and also the expected functional output for each unit in the chip  100 . Each “interface transaction” refers to work for a particular unit in the chip  100  to operate on. Then, in steps  126  and  128 , the interface transactions are applied to the RTL implementations of the designated units, such as unit  1  and unit N. The actual functional outputs of the RTL implementations of unit  1  and unit N are compared with the expected functional outputs of the same units from the FModel in step  130  to determine whether functional correctness of each RTL implementation of the unit is achieved. One significant drawback of this conventional approach is the inability to spot performance issues at the unit level. Worse yet, some performance bugs at one unit may hinder efforts to identify other performance bugs associated with other units. 
     As the foregoing illustrates, what is needed is an improved way of automating unit-level performance testing in IC design and address at least the problems set forth above. 
     SUMMARY OF THE INVENTION 
     A method and system for automating unit performance testing in integrated circuit design is disclosed. One embodiment of the present invention sets forth a method, which includes the steps of generating a first performance data for the unit to operate on a workload, embedding the first performance data in the workload for a register transfer level (RTL) implementation of the unit to operate on, and determining whether the expected performance of the unit is achieved based on the comparison between the first performance data and a second performance data, wherein the second performance data is generated after the RTL implementation of the unit operates on the workload. 
     One advantage of the disclosed method and system is that performance verification for each unit in an IC can occur in parallel and with different test patterns. This allows for testing on the front end and efficiently identifying the failing units. 
    
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
       So that the manner in which the above recited features of the present invention can be understood in detail, a more particular description of the invention, briefly summarized above, may be had by reference to embodiments, some of which are illustrated in the appended drawings. It is to be noted, however, that the appended drawings illustrate only typical embodiments of this invention and are therefore not to be considered limiting of its scope, for the invention may admit to other equally effective embodiments. 
         FIG. 1A  is a conventional approach of verifying both the functional correctness and the performance correctness of a chip prior to entering the physical design phase; 
         FIG. 1B  is yet another conventional approach in verifying the functional correctness of a chip prior to entering the physical design phase but at the unit level of the chip; 
         FIG. 2  is a flow chart illustrating method steps of conducting unit-level performance testing for a chip, according to one embodiment of the present invention; 
         FIG. 3  is a flow chart illustrating the method steps of generating and embedding expected performance data on a unit-level basis for automated testing, according to one embodiment of the present invention; 
         FIG. 4  is a conceptual diagram of a unit test-bench, according to one embodiment of the present invention; and 
         FIG. 5  is a simplified block diagram of a computing device configured to implement one or more aspects of the present invention. 
     
    
    
     DETAILED DESCRIPTION 
     Throughout this disclosure, a “state bundle” generally refers to a data structure, which a unit in a GPU uses to transmit state information to the various units of the GPU. The term “bundle decoder” is used interchangeably with “decoder” in this disclosure. Each bundle decoder is associated with a unit in the GPU and is configured to retrieve various performance markers (also referred to as triggers). 
       FIG. 2  is a flow chart illustrating method steps of conducting unit-level performance testing for a chip  200 , according to one embodiment of the present invention. Suppose there are N units in the RTL implementation of the chip  200 . In step  202 , system level stimuli along with the expected performance data for each of the N units are generated to be fed into the FModel. Subsequent paragraphs will further detail how such expected performance data is generated. Based on the received information, the FModel in step  204  generates interface transactions, or workload to be performed, for each of the N units and also embeds the expected performance data in such interface transactions. These interface transactions are then applied to each of the N units in the RTL implementation such as shown in steps  206  and  208 . The actual performance of each of the N units is measured as the unit operates on the interface transactions and is then compared with the embedded expected performance data in step  210 . In one implementation, the embedded expected performance data is not operated on or tampered with by any of the N units. In addition, in the same manner as described in conjunction with  FIG. 1B  above, the functional correctness for each of the N units is also verified in step  210 . 
       FIG. 3  is a flow chart illustrating the method steps of generating and embedding expected performance data on a unit-level basis for automated testing, according to one embodiment of the present invention. In step  300 , system level stimuli for the chip  200  are generated and are fed into a performance measuring tool in step  302 . The performance measuring tool is capable of estimating how each unit in the chip  200  should perform if given such system level stimuli. Some examples of the performance measuring tool include, without limitation, a performance simulator program and a performance modeling program. The estimated clock cycles for each unit are generated in step  304 . The estimated clock cycle information is then bundled with the same system level stimuli in step  306  before the bundled information is sent to a unit test-bench for processing in step  308 . Subsequent paragraphs will detail one implementation of the unit test-bench. 
     Similar to the discussions above, in step  310 , the system level stimuli and the estimated clock cycle information are fed into the FModel, which generates interface transactions for the N units in the chip  200  and also the functional outputs for each of the N units of the chip  200  accordingly. The generated interface transactions are then processed by the RTL implementations of the N units in step  312 , and the actual functional outputs and the actual performance data measured from such processing are compared with the expected functional outputs and the estimated clock cycle information to determine whether functional and performance correctness is achieved in step  314 . 
       FIG. 4  is a conceptual diagram of a unit test-bench  400 , according to one embodiment of the present invention. The unit test-bench  400  includes a unit under test  402  and a decoder  404  and also a performance checker  406  associated with such a unit. Suppose the unit under test  402  corresponds to unit  1  of the N units in the chip  200 . Suppose further that the estimated clock cycle count for unit  1  is 100 clock cycles. As mentioned above, the unit test-bench  400  receives both the system level stimuli and also the estimated clock cycle count. In one implementation, the estimated 100 clock cycles are stored in a state bundle  408  associated with the unit ID corresponding to unit  1 . In addition to the estimated 100 clock cycles (i.e., theory cycle count shown in  FIG. 4 ), the state bundle  408  should also include a performance begin trigger (e.g., perf_begin) and a performance end trigger (e.g., perf_end). As the unit under test  402  operates on the input command stream derived from the system level stimuli, it is also receiving decoded information, such as the perf_begin or the perf_end triggers from the decoder  404 . Its associated performance checker  406  computes the actual number of clock cycles starting from the time the unit test-bench  400  receives the state bundle containing the perf_begin trigger and the time the unit test-bench  400  receives the state bundle containing the perf_end trigger. The performance checker  406  also compares this actual number of clock cycles with the theory cycle count of 100 to see whether the RTL implementation of unit  1  achieves the expected performance requirements. It should be noted that the unit test-bench  400  does not modify the state bundle  408 , and the performance checker  406  subtracts the clock cycles associated with state bundles from the calculation of actual number of clock cycles. 
     It should be noted that the estimated clock cycle count for each unit can be generated from sources other than the performance measuring tool discussed above. Alternatively, the estimated clock cycle count can be generated by a designer of the unit. In addition, in one implementation, each unit in the chip  200  is associated with its own unit test-bench, and each unit test-bench can operate independently from the others. In other words, unit-level testing can occur in parallel, and different traffic patterns can be applied to the unit test-benches. 
     Furthermore, the unit test-bench  400  can be reconfigured to verify the performance of a group of units (also referred to as a super-unit). The theory cycle count is computed as a user specified Boolean equation of the various units that make up the super-unit. The flexibility of this alternative setup also allows certain latency to be subtracted out through this super-unit test-bench. In particular, the theory cycle count typically specifies the number of cycles required to process a given work load but fails to consider the latency to go through a single unit. For a super-unit, this latency may be significant and needs to be considered. 
       FIG. 5  is a simplified block diagram of a computing device  500  configured to implement one or more aspects of the present invention. The computing device  500  includes a processor  502 , which accesses a memory module  506  via a high speed I/O bridge  504 . The high speed I/O bridge  504  also manages the connections from the processor  502  to on-chip memory modules, such as caches, and a dedicated graphics processing channel, such as the Accelerated Graphics Port. The memory module  506  stores information and instructions to be executed by the processor  502  and may store temporary variables or other intermediate information during the execution of the instructions. 
     The high speed I/O bridge  504  manages the data-intensive pathways and supports high speed peripherals, such as, without limitation, display, gigabit Ethernet, fiber channel, and Redundant Array of Independent Disks (“RAID”). The high speed I/O bridge  504  is also coupled with secondary a I/O bridge  510 , which supports secondary peripherals  512 , such as, without limitation, disk controllers, Universal Serial Bus (“USB”), audio, serial, system Basic Input/Output System (“BIOS”), the Industry Standard Architecture (“ISA”) bus, the interrupt controller, and the Intelligent Driver Electronics (“IDE”) channels. 
     In one implementation, various programs containing sequences of instructions are developed. Specifically, a performance test program is designed to generate the system level stimuli as discussed above. A performance simulator program or a performance modeling program is developed to generate the estimated clock cycle count. An FModel program is developed to generate interface transactions and state bundles. A unit test-bench program is developed to measure unit-level performance data and determine whether performance correctness is achieved. The instructions of these various programs generally are read into the main memory module  506  before they are executed by the processor  502 . Execution of these instructions causes the processor  502  to perform the process steps described herein. In alternative embodiments, hard-wired circuitry may be used in place of or in combination with software instructions to implement one or more aspects of the present invention. 
     While the forgoing is directed to embodiments of the present invention, other and further embodiments of the invention may be devised without departing from the basic scope thereof. For example, aspects of the present invention may be implemented in hardware or software or in a combination of hardware and software. One embodiment of the invention may be implemented as a program product for use with a computer system. The program(s) of the program product define functions of the embodiments (including the methods described herein) and can be contained on a variety of computer-readable storage media. Illustrative computer-readable storage media include, but are not limited to: (i) non-writable storage media (e.g., read-only memory devices within a computer such as CD-ROM disks readable by a CD-ROM drive, flash memory, ROM chips or any type of solid-state non-volatile semiconductor memory) on which information is permanently stored; and (ii) writable storage media (e.g., floppy disks within a diskette drive or hard-disk drive or any type of solid-state random-access semiconductor memory) on which alterable information is stored. Such computer-readable storage media, when carrying computer-readable instructions that direct the functions of the present invention, are embodiments of the present invention. Therefore, the above examples, embodiments, and drawings should not be deemed to be the only embodiments, and are presented to illustrate the flexibility and advantages of the present invention as defined by the following claims.