Abstract:
A computer-implemented method to debug testbench and the associated circuit design by recording a trace of call frames along with the activities of the circuit design. By correlating and displaying the recorded call frames, the method enables users to easily trace the execution history of the subroutines and debug the testbench code. In addition, users can trace the source code of the testbench by using the trace of call frames. Furthermore, users can debug with a virtual simulation, which is done by post-processing the simulation records stored in a database.

Description:
CROSS REFERENCE TO RELATED APPLICATIONS 
       [0001]    This application claims the benefit of U.S. Provisional Application No. 61/333,262, filed May 11, 2010, and entitled “Method and system for function trace debugging in System Verilog”, which is hereby incorporated herein by reference. 
     
    
     BACKGROUND OF THE INVENTION 
       [0002]    1. Field of the Invention 
         [0003]    The present invention relates to a computer-implemented method for debugging a circuit design with a testbench in the field of integrated circuit (IC) design, and in particular to a method for debugging the testbench using post-processing approach. 
         [0004]    2. Description of the Prior Art 
         [0005]    Post-processing approach is often used for hardware debugging since saved simulation results are sufficient to provide hardware engineers with the ability to debug the hardware design. During hardware simulation, signal values at circuit nodes will be recorded for debugging throughout the entire simulation. Moreover, signal values only change at discrete simulation times. Therefore, during hardware simulation, signal value changes are often saved in files (also called dump files) in industry standard formats, such as Value Change Dump (VCD) or Fast Signal Database (FSDB). During post-processing debugging, waveform viewers are often used to read VCD or FSDB files to display signal value changes with respect to simulation times for helping users debug the behavior of the hardware design conveniently. 
         [0006]    The testbench written in high level language, such as System Verilog or C++, however, is more like traditional software in that objects can be created and deleted dynamically; variable values can change again and again while the simulation time stays unchanged; and functions and tasks (which will be collectively referred to as “subroutines” hereafter) can be called recursively if so desired. Using the conventional way of hardware debugging, such as signal value dumps and waveform viewing, is inadequate for debugging the testbench. Therefore, it is better to use a software debugging approach to debug the test bench, much like using an interactive debugger such as “GNU Debugger (GDB)” to debug a C++ program. While it&#39;s possible to do interactive debugging for the testbench, users often suffer from poor performance due to the simulator spending a long time evaluating the hardware part. 
         [0007]    Therefore, in conventional hardware simulation and debugging, it is very difficult to integrate both hardware debugging and testbench debugging together due to their intrinsic differences in operations. 
         [0008]    System Verilog provides an advantage in addressing the verification complexity challenge. However, there is a gap for IC designers when it comes to the debug and analysis of System Verilog testbench (SVTB). The accepted “dumpvars-based” techniques are not practical for the object-oriented testbench. Nevertheless, engineers do need to know what the testbench is doing at any given point in time. Thus far, engineers have been forced to revert to low-level, text-based message logging and subsequent manual analysis of the resulting text log files. Logging—the process of recording the history—has been widely used in systems and software environments. 
         [0009]    Most System Verilog libraries used today provide some built-in utilities to log messages generated from the testbench into a low-level text files that can be analyzed after simulation, engineers then manually correlate the testbench data to the design activity in order to debug the testbench. Therefore, this is a painful and ineffective approach to debug the testbench itself by using the logging messages alone. 
         [0010]    U.S. Pat. No. 6,934,935 entitled “Method and Apparatus for Accurate Profiling of Computer Programs” discloses a method and apparatus for profiling the execution of a computer program, including the actual CPU cycles spent in each function and the caller-callee (i.e., who-calls-who) relationships. To collect the runtime data, it has to insert software codes into the program. The collected data can be used to analyze the performance of the program and provide hints as to which parts of the program can be optimized to speed up the execution of the program. However, in testbench code executions, the focus is not on the CPU cycles spent in each subroutine. Consequently, the disclosure of U.S. Pat. No. 6,934,935 is aimed at evaluating software performance, but not debugging a testbench. 
         [0011]    Therefore, what is needed is a technique to record the behavior of SVTB functions and tasks at the same time with the activities of the DUT so that the history of the testbench execution can be correlated to the DUT in a simulation by using the same simulation time stamps. Then, the recorded information can be used to provide post-processing debugging capabilities to users so that the DUT and SVTB can be debugged together effectively and efficiently. 
       SUMMARY OF THE INVENTION 
       [0012]    One object of the present invention is to provide a solution to display both the DUT simulation results and testbench execution history on graphic windows correlatively at the same simulation time. Thus users can debug DUT and testbench simultaneously in an efficient way. 
         [0013]    One embodiment in the present invention is to provide a computer-implemented method to record necessary debugging information, comprising testbench call history, into a database by the following steps. First, for each subroutine (that is, a System Verilog task or function) of the plurality of the subroutines in the testbench, providing a first call-back routine which will be called before the code section of the subroutine is executed. Next for each subroutine of the plurality of the subroutines in the testbench, providing a second call-back routine which will be called after the code section of the subroutine is executed. Then, the simulation controlled by a simulator for testing a hardware or IC design can be started, wherein the simulator timing control will decide when to advance the simulation time one step at a time from zero until the end of the simulation. To those skilled in the art, it is a straight forward manner to register such call-back routines for a subroutine using System Verilog Programming Language Interface (PLI) functions. 
         [0014]    After the simulation started, for each subroutine of the plurality of the subroutines in the testbench, recording the first simulation time at which the corresponding first call-back is executed, a tag indicating the beginning of the subroutine, and the identification of the subroutine in the call frame when the corresponding first call-back routine is called. Next, for each subroutine of the plurality of the subroutines in the testbench, recording the second simulation time at which the corresponding second call-back is executed, a tag indicating the ending of the subroutine, and the identification of the subroutine in the call frame when the corresponding second call-back routine is called. As a result, the trace of call frames is formed according to the order of the call-backs which are called one by one at their respective simulation time which can be saved into a database for analyzing latter on. 
         [0015]    With testbench call history and other information, such as log messages and value change data, recorded in a database, we can display waveforms and log messages in graphic windows for users to debug the testbench along with the DUT. Furthermore, the testbench call history can also be shown to users in graphic windows in the format of call frames according to a specified simulation time at which the call frames are recorded. Users can easily obtain the information of the call stacks at specific simulation time by simply clicking on the waveform window to display them. In addition, by clicking the corresponding fields in call frames, users can quickly find the corresponding source code segments running at the specified simulation time. 
         [0016]    Moreover, users can run the simulation virtually according to the records in the database; in other words, users can virtually run the simulation again and again to debug the testbench and DUT without actually running the simulation with the simulator. For example, user can set a breakpoint and the virtual simulation will stop at the breakpoint quickly without re-running the simulation. 
         [0017]    Accordingly, with the features mentioned above, users can debug DUT and testbench in an interactive way through a user friendly graphic interface efficiently and effectively. 
         [0018]    Other objects, technical contents, features and advantages of the present invention will become apparent from the following description taken in conjunction with the accompanying drawings wherein are set forth, by way of illustration and example, certain embodiments of the present invention. 
     
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         [0019]    The foregoing aspects and many of the accompanying advantages of this invention will become more readily appreciated as the same becomes better understood by reference to the following detailed description, when taken in conjunction with the accompanying drawings, wherein: 
           [0020]      FIG. 1  is a schematic block diagram of a testbench environment; 
           [0021]      FIG. 2  is a schematic block diagram of the present invention; 
           [0022]      FIG. 3A  illustrates the definition for data structure of beginning call frame; 
           [0023]      FIG. 3B  illustrates the definition for data structure of ending call frame; 
           [0024]      FIG. 4A  is a simplified testbench program block to explain the call frame recording mechanism; 
           [0025]      FIG. 4B  illustrates the details of the first call-back routine; 
           [0026]      FIG. 4C  illustrates the details of the second call-back routine; 
           [0027]      FIG. 5A  and  FIG. 5B  illustrate a schematic flow chart to explain some use cases about call frame handling; 
           [0028]      FIG. 6  illustrates an example of displaying call frames and waveforms in graphic windows; and 
           [0029]      FIG. 7  is illustrates an example of displaying source code of subroutines in graphic windows. 
       
    
    
     DETAILED DESCRIPTION OF THE INVENTION 
       [0030]    The detailed explanation of the present invention is described as following. The described preferred embodiments are presented for purposes of illustrations and description, and they are not intended to limit the scope of the present invention. 
       Environment Introduction: 
       [0031]    Firstly, please refer to  FIG. 1 , which is a schematic block diagram of a typical testbench environment. In order to test a DUT  12 , testbench  10  generates test patterns which comprise transactions to Bus Functional Model (BFM)  11  module. The BFM  11  is responsible for translating the transactions into bus operations to the DUT  12 . The BFM  11  also receives the bus operations from DUT  12  and eventually gets back to testbench  10  in order to prepare the next transaction for testing the DUT  12 . Please note that only the BFM  11  and the DUT  12  will consume simulation times to emulate the real hardware behavior. 
         [0032]    Simulator timing control  13  will decide when it is necessary to advance the simulation time, simulator evaluates all the blocks or statements in the testbench  10  and the DUT  12  at any given step of simulation time, the simulator timing control  13  will advance the step of the simulation time when the blocks or statements containing time consuming operators are the only ones left to be evaluated after all other blocks or statements have been evaluated already. The time consuming statements include many different types, such as time delay statements or wait statements in Verilog. 
         [0033]    Thus BFM  11  and DUT  12  containing time consuming operators will consume simulation times. The testbench  10  comprises two types of subroutines: first type of subroutine, which do not call BFM  11  either directly or indirectly, will have the same simulation time recoded at the beginning and the ending of the subroutine execution; and the second type of subroutine, which calls BFM  11  either directly or indirectly to send transaction data to DUT  12 , will have different simulation times recorded at the beginning and the ending of the subroutine execution due to the simulation time delays in BFM  11  which contains time consuming operators. 
       Call Frame Recording: 
       [0034]    Next, please refer to  FIG. 2 , which is a flow chart to illustrate one embodiment of the present invention. The testbench, as illustrated in step  20 , includes a program block, a plurality of classes containing subroutines and variables. 
         [0035]    In step  21 , a recording control module, called PLI module hereafter, is introduced; the PLI module serves as a control program to register the call-backs and obtain information pertaining to the testbench through the System Verilog Programming Language Interface (PLI). For instance, through the PLI module, a first call-back routine and a second call-back routine can be registered and information such as status of variables and arguments of a subroutine can be obtained so that the call-back routines can record them respectively. 
         [0036]    In another embodiment for step  21 , the invocation of first call-back routine can be provided by inserting a call statement to the first call-back routine before the code section of each of the subroutine; and the invocation of second call-back routine can be provided by inserting a call statement to the second call-back routine after the code section of each of the subroutine. 
         [0037]    After the first and the second call-backs are provided, the simulation starts as in step  22 . In step  23 , once a subroutine is called in the testbench, the corresponding first call-back routine will be executed first to record the call frame as shown in step  24  and stores the data into a database in step  26 . And then the code section of the subroutine will be executed to perform the original task of the subroutine. After the code section of the subroutine is executed, the corresponding second call-back routine will be executed to record the call frame in step  25  and stores the information into a database in step  26 . Since the subroutine is executed one after another, and the information and activities of the subroutine will be stored into the database in the order each subroutine is executed at its corresponding simulation time, thereby forming a trace of call frames in the order of each call-back being executed one after another, in which the call frame contains the corresponding simulation time at which the call-back is called. With the testbench executed and all call frames recorded, the simulation stops in step  27 . 
         [0038]    To further detail the call frame mentioned above, next please refer to  FIG. 3A  which is the data structure of beginning call frame  3   a  to define a set of important debugging information including: the index of the call frame  31   a  which is an accumulated count showing the ordinal of the call frame; the identification of the subroutine  32   a  to identify the subroutine by using a subroutine ID or a unique name to represent the subroutine; the tag  33   a  to indicate the beginning of the subroutine which is about to be executed; the first simulation time  34   a  to record the simulation timestamp when the subroutine execution begins; the identification of the caller subroutine  35   a  which calls the subroutine; the code position  36   a  including a line-number of the source file at which the subroutine call is executed and a count to indicate number of times that the subroutine calls are executed at the line-number; the variable status  37   a  which is the initial variable status which records the values of arguments and variables accessible to the subroutine at the beginning of the execution of the subroutine; the ending call frame index  38   a  which records the index of the corresponding ending call frame paired with the current beginning call frame. 
         [0039]    Next, please refer to  FIG. 3B  which is the data structure of ending call frame  3   b  to define a set of important debugging information including: the index of the call frame  31   b  which is an accumulated count showing the ordinal of the call frame; the identification of the subroutine  32   b  to identify the subroutine by using a subroutine ID or a unique name to represent the subroutine; the tag  33   b  to indicate the end of the subroutine which has been executed; the second simulation time  34   b  to record the simulation timestamp when the subroutine execution ends; the identification of the caller subroutine  35   b  which calls the subroutine; the code position  36   b  including a line-number of the source file at which the subroutine call is executed and a count to indicate number of times that the subroutine calls are executed at the line-number; the variable status  37   b  which is the latest variable status which records the values of arguments and variables accessible to the subroutine at the end of the execution of the subroutine; the beginning call frame index  38   b  which records the index of the corresponding beginning call frame paired with the current end call frame. 
         [0040]    Please note that, as mentioned earlier, if the subroutine calls or wait for BFM to finish a bus transaction, the first simulation time in the beginning call frame will be different from the second simulation time in the ending call frame. Otherwise, the subroutine will be executed in zero simulation time and the first simulation time in the beginning call frame is the same as the second simulation time in the ending call frame. 
         [0041]    Based on the definition of the call frame described above, an embodiment for performing this invention is provided to illustrate more details about recording the trace of call frames as shown in  FIG. 4A . For instance, the testbench includes a program block  4 , and a subroutine_Y  40  which is called by subroutine_X  41 . When the subroutine_Y  40  is about to be executed, the first call-back routine  42  will be executed and a beginning call frame will be created by the first call-back routine  43 , which is shown in  FIG. 4B , to record the beginning information of the current subroutine contained in the data structure of beginning call frame. The information in the data structure includes: the index of the call frame; the identification of (callee) subroutine, subroutine_Y  40 ; the tag set as BEGINNING to indicate that this is the beginning of subroutine_Y  40 ; the first simulation time at which the first call back routine is executed; the identification of caller subroutine_X  41  which can be derived from call frame history; the initial variable status before entering the code section  42  of the subroutine_Y  40 ; and the ending call frame index which is temporarily set to zero and should be updated later when ending call frame index is available; the code position to indicate the line-number where the subroutine_Y  40  is called by subroutine_X  41  and a count to indicate the number of times subroutine_Y  40  is called at the line-number. 
         [0042]    In one embodiment, the code position information can be obtained by using System Verilog Programming Interface (PLI) functions to query the simulator from inside the first call-back routine. 
         [0043]    Consequently, a complete beginning call frame is constructed and can be recorded into the database. 
         [0044]    After the code section  42  of subroutine_Y  40  is executed, the second call-back routine  44  will be executed and an ending call frame will be created by the second call-back routine  44 , which is shown in  FIG. 4C , to record the ending information of the current subroutine contained in the data structure of ending call frame. First, part of the data is the same as in the corresponding beginning call frame, which includes: identification of the subroutine; identification of the caller subroutine; code position information. Thus they can be filled into the ending call frame by the second call-back routine. Next, other information which will be filled in the ending call frame by the second call-back routine  44  includes: the index of the call frame; the tag set as END to indicate that this is the end of subroutine_Y  40  in the tag field of the ending call frame; the latest variable status should also be written in the corresponding field of the ending call frame; the second simulation time at which the second call-back routine  44  is executed; and the beginning call frame index which is copied from the index of call frame field of the corresponding beginning call frame. Consequently, a complete ending call frame is constructed and can be recorded into the database. In addition, the ending call frame index field of the corresponding beginning call frame should be updated by copying the index field from the corresponding ending call frame. With the ending and beginning call frame indices available, the pairing relation between beginning and ending call frames can be easily identified. Another embodiment for deciding the pairing relation between beginning and ending call frames is to trace backwards to find the first beginning call frame which contains the same identification of the subroutine, identification of caller subroutine and code position information. 
         [0045]    By repeatedly performing the steps, as shown in  FIG. 4 , for all the subroutines, we can record an indexed series of call frames into the database for the entire simulation period. 
       Frame Data Visualization: 
       [0046]      FIG. 5A  and  FIG. 5B  illustrate the details of how to use the recorded call frames and debugging information to help users to debug the testbench. For example, when users are interested in the call frames recorded at a particular simulation time, users can specify the time information by entering it in an input window or clicking at an icon, which represents the simulation time, located at the time axis of waveforms. In step  50   a,  once the time information is entered, a target call frame can be identified from all of the call frames recorded in the database, as illustrated in step  51   a,  by searching the first matched call frame which has the same simulation time as users specified. In step  52   a,  once the target call frame is found, some call frames located before or after the target call frame can be displayed in the graphic window as well. 
         [0047]      FIG. 6  illustrates an example of displaying multiple graphic windows with one window  60  showing waveforms and another window  61  showing call frames. In the waveform window  60 , when a mouse cursor  601  moved into the window  60 , a vertical dotted line  602  will be invoked to align with and select a simulation time by moving the mouse cursor  601  onto one of the time stamps  603  on the waveform, which allow users to perform some time related operations to interact with other windows. In the call frame window  61 , certain number of call frames  611  can be displayed in the order they were executed with the target call frame located around the center of the call frames and pointed by an arrow  612 . Moreover, users can move scrollbar  613  up and down to trace the call frames for debugging the testbench. 
         [0048]    Please refer back to  FIG. 5A . Once the target call frame is identified, users can invoke and view the source code of the corresponding subroutine of the target call frame. In step  53   a,  when a call frame is selected, the corresponding callee subroutine ID can be extracted from the call frame to locate the source code of the callee subroutine. Consequently, as illustrated in step  54   a,  the source code of the callee subroutine can be displayed in a graphic window for users to view.  FIG. 7  illustrates an example for displaying source code in graphic windows. The call frame  71  can be zoomed in to view all the fields in the call frame, such as callee subroutine ID  711  and caller subroutine ID  712 ; and the callee subroutine ID  711  can be clicked by users to generate a graphic window  72  for source code viewing. 
         [0049]    Please refer back to  FIG. 5B , which illustrates how users can trace back to the caller subroutine from the call frame of a callee subroutine which is called by the caller subroutine. In one embodiment, users can be provided an option menu with one of the options, say “trace back”. In step  55   a,  after users choose the “trace back” option, the caller subroutine ID and the code position information are extracted from the call frame of the callee subroutine. An alternative way to obtain caller subroutine ID is to search the trace of call frames backward, starting from the call frame of the callee subroutine, to find the first call frame with a tag indicating that the subroutine begins but not ends yet. Once the call frame of the caller subroutine is found, the caller subroutine ID can be extracted and used to find the source code of the caller subroutine. In step  56   a  and  57   a,  the source code of the caller subroutine can be displayed in a graphic window with an arrow pointing to the line number where the callee subroutine is called. 
         [0050]      FIG. 7  illustrates an example of displaying graphic windows, wherein a caller subroutine is shown in graphic window  73  and pointed by an arrow  731  to a line number where a callee subroutine is called. Furthermore, with the source code displayed in the graphic windows, users can choose to perform a “single step” command to execute the source code line by line to debug the testbench—since all the initial values of variables and arguments accessible to the subroutine are recorded in the call frame already—thereby allowing the subroutine to be re-executed to get the intermediate results line by line in the subroutine. 
         [0051]    Another useful feature for users is to run the simulation virtually by using post-processing debugging. Users can set some breakpoint conditions in order to stop the virtual simulation at certain simulation time or event; and the simulation will be executed virtually by replaying the records saved in the database without running the real simulation of the circuit design with the testbench again. Once one of the breakpoint conditions is met, the virtual simulation will be suspended and all the related information, such as call frames, waveforms and corresponding source code, can be displayed in graphic windows with the breakpoint condition indicated. 
         [0052]    In summary, all the abovementioned graphic windows, such as waveform windows, call frame windows, and source code windows, can be activated simultaneously within one screen for users to view and debug the testbench easily and effectively. 
         [0053]    The foregoing descriptions of specific embodiments of the present invention have been presented for purposes of illustrations and description. They are not intended to be exclusive or to limit the invention to the precise forms disclosed, and obviously many modifications and variations are possible in light of the above teaching. The embodiments were chosen and described in order to best explain the principles of the invention and its practical application, to thereby enable others skilled in the art to best utilize the invention and various embodiments with various modifications as are suited to particular use contemplated. It is intended that the scope of the invention be defined by the Claims appended hereto and their equivalents.