Detection of unchecked signals in circuit design verification

An unchecked signal detection mechanism runs simulation tests of circuit designs that normally pass. Before a simulation test is run, noise is injected into one randomly chosen signal. A random constant value is assigned to the randomly chosen signal. The constant random value is forced on the selected signal for the duration of the simulation test. Signals for which simulation tests always pass, even when their value is forced, are likely not checked and declared as suspect. The subset of suspect signals is then checked to determine whether their checkers are indeed missing or defective. Any verification flaws (holes) found are then fixed.

FIELD OF THE INVENTION

The present invention relates to the field of verification, and more particularly relates to a method of detecting signals in a circuit design that are not checked in a verification environment.

SUMMARY OF THE INVENTION

There is thus provided in accordance with the present invention, a method for use on a computer of detecting unchecked signals in simulation tests of a circuit design, said method comprising injecting noise into one randomly selected signal in each simulation test, determining a set of signals whose corresponding simulation tests always pass, and declaring said set of signals as suspect for a missing or defective checker.

There is also provided in accordance with the invention, a method for use on a computer of detecting unchecked signals in simulation tests of a circuit design, said method comprising injecting noise into one randomly selected signal in each simulation test, determining a set of signals whose corresponding simulation tests always pass, declaring said set of signals as suspect for a missing or defective checker, and checking whether checkers associated with said suspect signals are missing or defective.

There is further provided in accordance with the invention, a computer program product for detecting unchecked signals in simulation tests of a circuit design, the computer program product comprising a computer usable medium having computer usable code embodied therewith, the computer usable program code comprising computer usable code configured for injecting noise into one randomly selected signal in each simulation test, computer usable code configured for determining a set of signals whose corresponding simulation tests always pass, and computer usable code configured for declaring said set of signals as suspect for a missing or defective checker.

DETAILED DESCRIPTION OF THE INVENTION

Conventional verification environments for testing the correctness of a circuit design by simulation typically include, among other verification components, checkers to verify that signals of the design behave as expected, relative to themselves and relative to other signals, when random tests are run on the simulated design.

These checkers report an error each time they detect that a signal deviates from its expected behavior. Numerous tests are run and each test passes unless one of the checkers reports an error. If checkers for some signals are missing or defective, however, it may happen that no errors will be reported for those signals. In this case, one of three scenarios is possible: (1) no checker function exists to check for circuit errors; (2) a checker function does exist to check for circuit errors but it is never called; and (3) a function does exist to check for circuit errors and is called, but flaws exist in the checker function.

Further, the failure or lack of checking of those signals may not be noticed, since it might be assumed that those signals always behave correctly in the circuit design. Such a resulting flaw or hole in the verification environment is significant since it can cause circuit design bugs to go unreported.

The present invention provides a mechanism to detect signals which are not checked in the verification environment. Use of the unchecked signal detection mechanism in a verification/simulation environment helps to close such verification flaws or holes.

A block diagram illustrating an example computer processing system adapted to implement the system and methods of the present invention is shown inFIG. 1. The computer system, generally referenced10, comprises a processor12which may comprise a digital signal processor (DSP),7central processing unit (CPU), microcontroller, microprocessor, microcomputer, ASIC or FPGA core. The system also comprises static read only memory18and dynamic main memory20all in communication with the processor. The processor is also in communication, via bus14, with a number of peripheral devices that are also included in the computer system. Peripheral devices coupled to the bus include a display device24(e.g., monitor), alpha-numeric input device25(e.g., keyboard) and pointing device26(e.g., mouse, tablet, etc.)

The computer system is connected to one or more external networks such as a LAN or WAN23via communication lines connected to the system via data I/O communications interface22(e.g., network interface card or NIC). The network adapters22coupled to the system enable the data processing system to become coupled to other data processing systems or remote printers or storage devices through intervening private or public networks. Modems, cable modem and Ethernet cards are just a few of the currently available types of network adapters. The system also comprises magnetic or semiconductor based storage device21and/or28for storing application programs and data. The system comprises computer readable storage medium that may include any suitable memory means, including but not limited to, magnetic storage, optical storage, semiconductor volatile or non-volatile memory or any other memory storage device.

Software adapted to implement the system and methods of the present invention is adapted to reside on a computer readable medium, such as a magnetic disk within a disk drive unit. Alternatively, the computer readable medium may comprise a floppy disk, removable hard disk, Flash memory16, EEROM based memory, bubble memory storage, ROM storage, distribution media, intermediate storage media, execution memory of a computer, and any other medium or device capable of storing for later reading by a computer a computer program implementing the method of this invention. The software adapted to implement the system and methods of the present invention may also reside, in whole or in part, in the static or dynamic main memories or in firmware within the processor of the computer system (i.e. within microcontroller, microprocessor or microcomputer internal memory).

Other digital computer system configurations can also be employed to implement the system and methods of the present invention, and to the extent that a particular system configuration is capable of implementing the system and methods of this invention, it is equivalent to the representative digital computer system ofFIG. 1and within the spirit and scope of this invention.

Once they are programmed to perform particular functions pursuant to instructions from program software that implements the system and methods of this invention, such digital computer systems in effect become special purpose computers particular to the method of this invention. The techniques necessary for this are well-known to those skilled in the art of computer systems.

It is noted that computer programs implementing the system and methods of this invention will commonly be distributed to users on a distribution medium such as floppy disk or CD-ROM or may be downloaded over a network such as the Internet using FTP, HTTP, or other suitable protocols. From there, they will often be copied to a hard disk or a similar intermediate storage medium. When the programs are to be run, they will be loaded either from their distribution medium or their intermediate storage medium into the execution memory of the computer, configuring the computer to act in accordance with the method of this invention. All these operations are well-known to those skilled in the art of computer systems.

Unchecked Signal Detection

A general block diagram illustrating the unchecked signal detection mechanism of the present invention is shown inFIG. 2. The mechanism, generally referenced70, comprises a circuit design72, noise injection block74, circuit simulation test block76, signal verification block78and check block84.

The unchecked signal detection mechanism runs simulation tests76of circuit designs72that normally pass. Before a simulation test is actually run, however, noise is injected (block74) into one randomly chosen signal. The constant value assigned to the randomly chosen signal is also randomly determined. A constant random value is forced on the selected signal for the duration of the simulation test. Signals for which simulation tests always pass (block78), even when their value is forced, are likely not checked and labeled as suspect82. The subset of suspect signals (typically a small set) is then checked (automatically or manually) (block84) to see if whether their checkers are indeed missing or defective, the results86of which are then reported.

A flow diagram illustrating an example method of detecting unchecked signals is shown inFIG. 3. First, all signals of the circuit design to be checked are listed in a file or other data entity (step30). In the simulation and checking verification environment of the circuit design, an option to select one of these signals randomly, once per simulation test, is provided (step32). Once a signal is randomly selected, a random value is chosen for it (step34). The randomly chosen value is forced onto the selected signal (which is also randomly chosen) for the duration of the simulation test (step36). It is noted that the ability to force a value onto a signal is a feature of conventional circuit simulators.

The collection of tests in simulation which are already in use in the verification environment of this particular circuit design are then run (step38). Note that it is necessary to use only simulation tests which are known (from previous testing of this circuit design) to pass when the random forcing option of the unchecked signal detection mechanism of the present invention is not used.

For each simulation test run, a record of which signal had its value randomly forced and whether the test simulation passed or failed is recorded (step40). After a large number of random tests have been run with the random forcing option enabled, the test results recorded in step40are analyzed (step42). In one example embodiment, a script written for that purpose may be used to analyze the recorded results.

Any signal which has been forced to a random value at least a threshold number of times without any test failing is declared suspect for a missing or defective check (step44). Note that the threshold value may be predetermined and/or configurable. In one example, the threshold is greater than or equal to five.

For each signal declared as suspect, the verification program is analyzed and studied (either automatically or manually) in order to find any missing or defective checkers (step46). The above process (i.e. steps32through46) is repeated until no additional suspected unchecked signals are detected (step48).

A flow diagram illustrating an example method of analyzing the verification program is shown inFIG. 4. The following method is performed automatically (via programmatical means) or manually (via human examination of code) for each signal declared as suspect to determine whether signal checking is really missing or defective.

First, it is examined whether the signal is looked at somewhere in the verification program (step50). This can be achieved, for example, by searching the files of the program for the name of the signal. The file search can be performed using the well known Unix “grep” search command. Since a signal is commonly assigned a symbolic name within the program, one must first search the files for the signal name to find its symbolic name and then search again to see if and where the symbolic name is used.

If the program contains a code segment which looks at the signal (step52), check if that code is ever called (step54). This can be determined, for example, using well known code coverage techniques, or by inserting print statements in the code segment, running simulation tests and seeing if the inserted messages print. If code which looks at the signal is called (step54), the code is inspected to see whether it contains checks of the signal and if these checks, when they fail, cause an error message to be reported (step56). Note that as part of this debugging process, a deliberate error in the check in step56can temporarily be inserted (step58). A simulation test is then run and then it is determined whether the error message of the check prints (step60).

Any verification flaws (or holes) found in steps50through60described supra are then fixed (step62). Note that the fix applied to the code may be performed programmatically or manually depending on the implementation of the mechanism.

The corresponding structures, materials, acts, and equivalents of all means or step plus function elements in the claims below are intended to include any structure, material, or act for performing the function in combination with other claimed elements as specifically claimed. The description of the present invention has been presented for purposes of illustration and description, but is not intended to be exhaustive or limited to the invention in the form disclosed. As numerous modifications and changes will readily occur to those skilled in the art, it is intended that the invention not be limited to the limited number of embodiments described herein. Accordingly, it will be appreciated that all suitable variations, modifications and equivalents may be resorted to, falling within the spirit and scope of the present invention. The embodiments were chosen and described in order to best explain the principles of the invention and the practical application, and to enable others of ordinary skill in the art to understand the invention for various embodiments with various modifications as are suited to the particular use contemplated.

It is intended that the appended claims cover all such features and advantages of the invention that fall within the spirit and scope of the present invention. As numerous modifications and changes will readily occur to those skilled in the art, it is intended that the invention not be limited to the limited number of embodiments described herein. Accordingly, it will be appreciated that all suitable variations, modifications and equivalents may be resorted to, falling within the spirit and scope of the present invention.