Patent Publication Number: US-8990746-B1

Title: Method for mutation coverage during formal verification

Description:
FIELD OF THE INVENTION 
     The embodiments of the present disclosure relate to a method of formal verification, and more particularly, to a mechanism for mutation coverage associated with formal verification. 
     BACKGROUND 
     In general, it may be desirable to identify errors in semiconductor design while the semiconductor is in the design phase. As production moves from design to tooling and fabrication, the cost to fix design errors increases greatly. This great increase in cost may be attributable to specialized fabrication hardware that may have to be modified to repair a design error. A very small change in design can have cascading effects to other areas of the semiconductor. If these changes are made in while the semiconductor is in the design phase, software design tools can simply modify the architecture appropriately with little ramifications. If, however, a design error is found after production has transitioned to tooling and fabrication, a design change with cascading effects could necessitate the retooling of the entire line at considerable cost. 
     As design errors are much less costly to remedy while the semiconductor is in the design phase, much effort has been expended into methods of ensuring proper semiconductor designs in this stage. The process of ensuring proper design is referred to as verification. Design verification is generally focused into two areas: simulation and formal verification. Simulation involves testing the design against a sample set of inputs and checking the outputs against their expected values. Formal verification involves testing the universe of possible inputs against their expected outputs. 
     SUMMARY 
     Accordingly, an embodiment of the present disclosure is directed to a computer-implemented method for formal verification. The method may include providing an electronic design associated with the integrated circuit. The method may further include generating one or more faults in a cone of influence of an assertion and placing a constraint configured to model an original design for the one or more faults. The method may also include initiating formal verification on the electronic design while ignoring all electronic design constraints. The method may further include determining if the assertion is passing, wherein determining includes activating an original design for a subset of faults. If the assertion is passing, the method may include activating a single fault from the subset, determining if the assertion is passing and if the assertion does pass, deleting the single fault from the subset. 
     One or more of the following features may be included. The method may further include, if the assertion does not pass, temporarily adding all constraints to the electronic design. The method may also include simulating the electronic design with the temporarily added constraints and determining if there is a deadend. If there is a deadend, the method may also include obtaining the fault for the deadend and activating the original design for the obtained fault. The method may further include generating a fault list, including the obtained fault and activating the remaining faults. 
     In another embodiment of the present disclosure a computer-readable storage medium having stored thereon instructions, which when executed by a processor result in a number of operations is provided. Operations may include receiving an electronic design associated with the integrated circuit. Operations may further include generating one or more faults in a cone of influence of an assertion and placing a constraint configured to model an original design for the one or more faults. Operations may also include initiating formal verification on the electronic design while ignoring all electronic design constraints. Operations may further include determining if the assertion is passing, wherein determining includes activating an original design for a subset of faults. If the assertion is passing, operations may include activating a single fault from the subset, determining if the assertion is passing and if the assertion does pass, deleting the single fault from the subset. 
     One or more of the following features may be included. Operations may further include, if the assertion does not pass, temporarily adding all constraints to the electronic design. Operations may also include simulating the electronic design with the temporarily added constraints and determining if there is a deadend. If there is a deadend, operations may also include obtaining the fault for the deadend and activating the original design for the obtained fault. Operations may further include generating a fault list, including the obtained fault and activating the remaining faults. 
     In yet another embodiment of the present disclosure a computing system for formal verification is provided. The system may include at least one processor and at least one memory architecture coupled with the at least one processor, wherein the at least one processor is configured to receive, via a computing device, an electronic design associated with the integrated circuit and to generate one or more faults in a cone of influence of an assertion. The at least one processor further configured to place a constraint configured to model an original design for the one or more faults and to initiate formal verification on the electronic design while ignoring all electronic design constraints. The at least one processor further configured to determine if the assertion is passing, wherein determining includes activating an original design for a subset of faults. If the assertion is passing, the at least one processor may activate a single fault from the subset, determine if the assertion is passing and if the assertion does pass, delete the single fault from the subset. 
     One or more of the following features may be included. If the assertion does not pass, the at least one processor may temporarily add all constraints to the electronic design. The at least one processor may simulate the electronic design with the temporarily added constraints and determine if there is a deadend. 
     It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory and are intended to provide further explanation of embodiments of the present disclosure as claimed. 
    
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
       The accompanying drawings, which are included to provide a further understanding of embodiments of the present disclosure and are incorporated in and constitute a part of this specification, illustrate embodiments of the present disclosure and together with the description serve to explain the principles of embodiments of the present disclosure. 
         FIG. 1  diagrammatically depicts an mutation coverage process coupled to a distributed computing network; 
         FIG. 2  is an illustration of a simulation method in accordance with the present disclosure; 
         FIG. 3  is an illustration of a formal verification method according to the present disclosure; 
         FIG. 4  is another illustration of a formal verification method according to the present disclosure; 
         FIG. 5  is a logical block diagram of a sixteen-bit comparator; 
         FIG. 6  is an exemplary flowchart of a method of performing formal verification according to an embodiment of the present disclosure; 
         FIG. 7  is an exemplary flowchart corresponding to block  220  of  FIG. 6 ; 
         FIG. 8  is an exemplary flowchart corresponding to block  240  of  FIG. 7 ; 
         FIG. 9  is an exemplary flowchart corresponding to the combination of blocks  220  and  240  of  FIG. 6  and  FIG. 7  respectively; 
         FIG. 10  is an exemplary flowchart corresponding to the combination of  FIG. 6  and  FIG. 9 ; 
         FIG. 11  is a flowchart showing an exemplary first iteration through the method shown in  FIG. 10 ; 
         FIG. 12  is a flowchart showing an exemplary second iteration through the method shown in  FIG. 10 ; 
         FIG. 13  is a flowchart showing an exemplary third iteration through the method shown in  FIG. 10 ; 
         FIG. 14  is a flowchart depicting an embodiment of a debugging process consistent with the present disclosure; 
         FIG. 15  is a flowchart depicting an embodiment of a debugging process consistent with the present disclosure; 
         FIG. 16  is a flowchart depicting an embodiment of a mutation coverage process consistent with the present disclosure; 
         FIG. 17  is a flowchart depicting an embodiment of a mutation coverage process consistent with the present disclosure; and 
         FIG. 18  is a flowchart depicting an embodiment of a mutation coverage process consistent with the present disclosure. 
     
    
    
     DETAILED DESCRIPTION 
     Reference will now be made in detail to the embodiments of the present disclosure, examples of which are illustrated in the accompanying drawings. The present disclosure may, however, be embodied in many different forms and should not be construed as being limited to the embodiments set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of the present disclosure to those skilled in the art. In the drawings, the thicknesses of layers and regions may be exaggerated for clarity. Like reference numerals in the drawings denote like elements. 
     System Overview 
     Referring to  FIG. 1 , there is shown constraint minimization process  10 , debugging process  11 , and mutation coverage process  13  that may reside on and may be executed by server computer  12 , which may be connected to network  14  (e.g., the internet or a local area network). Examples of server computer  12  may include, but are not limited to: a personal computer, a server computer, a series of server computers, a mini computer, and a mainframe computer. Server computer  12  may be a web server (or a series of servers) running a network operating system, examples of which may include but are not limited to: Microsoft Windows XP Server™; Novell Netware™; or Redhat Linux™, for example. Additionally and/or alternatively, the constraint minimization process  10 , debugging process  11 , and mutation coverage process  13  may reside on a client electronic device, such as a personal computer, notebook computer, personal digital assistant, or the like. 
     As will be discussed below in greater detail, constraint minimization process  10  may be used to minimize the number of constraints required for formal verification of an integrated circuit design. In this way, the present disclosure may be used to detect a reduced list of constraints that may be functionally required to verify a particular assertion. The methods described herein may utilize a variety of techniques, including, but not limited to, constraint solving and dead-end detection. 
     As is also discussed below, debugging process  11  may be configured to debug an unreachable design target detected by formal verification. These targets may be unreachable because of a particular design bug. In order to understand and fix the design bug, embodiments disclosed herein may provide information to the user that may assist in the debugging process. 
     In some embodiments, the mutation coverage process  13  described herein may be configured to provide a guided mechanism for mutation coverage using formal verification. Mutation coverage is a fault based testing technique that measures the effectiveness of test suites. Faults may be introduced into the system by creating a set of faulty versions called Mutants. Test cases may be used to execute these mutants with the goal of causing mutants to produce incorrect output. The mutants causing incorrect output are considered covered for a given set of test suites. 
     The instruction sets and subroutines of constraint minimization process  10 , debugging process  11 , and mutation coverage process  13 , which may be stored on storage device  16  coupled to server computer  12 , may be executed by one or more processors (not shown) and one or more memory architectures (not shown) incorporated into server computer  12 . Storage device  16  may include but is not limited to: a hard disk drive; a tape drive; an optical drive; a RAID array; a random access memory (RAM); and a read-only memory (ROM). 
     Server computer  12  may execute a web server application, examples of which may include but are not limited to: Microsoft IIS™, Novell Webserver™, or Apache Webserver™, that allows for HTTP (i.e., HyperText Transfer Protocol) access to server computer  12  via network  14 . Network  14  may be connected to one or more secondary networks (e.g., network  18 ), examples of which may include but are not limited to: a local area network; a wide area network; or an intranet, for example. 
     Server computer  12  may execute one or more server applications (e.g., server application  20 ), examples of which may include but are not limited to, e.g., Lotus Domino™ Server and Microsoft Exchange™ Server. Server application  20  may interact with one or more client applications (e.g., client applications  22 ,  24 ,  26 ,  28 ) in order to execute constraint minimization process  10 , debugging process  11 , and/or mutation coverage process  13 . Examples of client applications  22 ,  24 ,  26 ,  28  may include, but are not limited to, design verification tools such as those available from the assignee of the present disclosure. These applications may also be executed by server computer  12 . In some embodiments, constraint minimization process  10 , debugging process  11 , and/or mutation coverage process  13  may be a stand-alone application that interfaces with server application  20  or may be an applet/application that is executed within server application  20 . 
     The instruction sets and subroutines of server application  20 , which may be stored on storage device  16  coupled to server computer  12 , may be executed by one or more processors (not shown) and one or more memory architectures (not shown) incorporated into server computer  12 . 
     As mentioned above, in addition/as an alternative to being a server-based application residing on server computer  12 , the constraint minimization process  10 , debugging process  11 , and/or mutation coverage process  13  may be a client-side application (not shown) residing on one or more client electronic devices  38 ,  40 ,  42 ,  44  (e.g., stored on storage devices  30 ,  32 ,  34 ,  36 , respectively). As such, constraint minimization process  10 , debugging process  11 , and/or mutation coverage process  13  may be a stand-alone application that interfaces with a client application (e.g., client applications  22 ,  24 ,  26 ,  28 ), or may be an applet/application that is executed within a client application. As such, the constraint minimization process  10 , debugging process  11 , and/or mutation coverage process  13  may be a client-side process, a server-side process, or a hybrid client-side/server-side process, which may be executed, in whole or in part, by server computer  12 , or one or more of client electronic devices  38 ,  40 ,  42 ,  44 . 
     The instruction sets and subroutines of client applications  22 ,  24 ,  26 ,  28 , which may be stored on storage devices  30 ,  32 ,  34 ,  36  (respectively) coupled to client electronic devices  38 ,  40 ,  42 ,  44  (respectively), may be executed by one or more processors (not shown) and one or more memory architectures (not shown) incorporated into client electronic devices  38 ,  40 ,  42 ,  44  (respectively). Storage devices  30 ,  32 ,  34 ,  36  may include but are not limited to: hard disk drives; tape drives; optical drives; RAID arrays; random access memories (RAM); read-only memories (ROM), compact flash (CF) storage devices, secure digital (SD) storage devices, and memory stick storage devices. Examples of client electronic devices  38 ,  40 ,  42 ,  44  may include, but are not limited to, personal computer  38 , laptop computer  40 , personal digital assistant  42 , notebook computer  44 , a data-enabled, cellular telephone (not shown), and a dedicated network device (not shown), for example. Using client applications  22 ,  24 ,  26 ,  28 , users  46 ,  48 ,  50 ,  52  may utilize formal analysis, testbench simulation, and/or hybrid technology features verify a particular integrated circuit design. 
     Users  46 ,  48 ,  50 ,  52  may access server application  20  directly through the device on which the client application (e.g., client applications  22 ,  24 ,  26 ,  28 ) is executed, namely client electronic devices  38 ,  40 ,  42 ,  44 , for example. Users  46 ,  48 ,  50 ,  52  may access server application  20  directly through network  14  or through secondary network  18 . Further, server computer  12  (e.g., the computer that executes server application  20 ) may be connected to network  14  through secondary network  18 , as illustrated with phantom link line  54 . 
     The various client electronic devices may be directly or indirectly coupled to network  14  (or network  18 ). For example, personal computer  38  is shown directly coupled to network  14  via a hardwired network connection. Further, notebook computer  44  is shown directly coupled to network  18  via a hardwired network connection. Laptop computer  40  is shown wirelessly coupled to network  14  via wireless communication channel  56  established between laptop computer  40  and wireless access point (i.e., WAP)  58 , which is shown directly coupled to network  14 . WAP 58 may be, for example, an IEEE 802.11a, 802.11b, 802.11g, Wi-Fi, and/or Bluetooth device that is capable of establishing wireless communication channel  56  between laptop computer  40  and WAP 58. Personal digital assistant  42  is shown wirelessly coupled to network  14  via wireless communication channel  60  established between personal digital assistant  42  and cellular network/bridge  62 , which is shown directly coupled to network  14 . 
     As is known in the art, all of the IEEE 802.11x specifications may use Ethernet protocol and carrier sense multiple access with collision avoidance (CSMA/CA) for path sharing. The various 802.11x specifications may use phase-shift keying (PSK) modulation or complementary code keying (CCK) modulation, for example. As is known in the art, Bluetooth is a telecommunications industry specification that allows e.g., mobile phones, computers, and personal digital assistants to be interconnected using a short-range wireless connection. 
     Client electronic devices  38 ,  40 ,  42 ,  44  may each execute an operating system, examples of which may include but are not limited to Microsoft Windows™, Microsoft Windows CE™, Redhat Linux™, or a custom operating system. 
     Constraint Minimization, Debugging, and Mutation Coverage Processes 
     The term “constraint” as used herein, may refer to a condition that may limit the set of behaviors to be considered. A constraint may represent real requirements (e.g., clocking requirements) on the environment in which the design is used, or it may represent artificial limitations (e.g. mode settings) imposed in order to partition the verification task. 
     The term “assertion” as used herein, may refer to a statement that a given property is required to hold and/or a directive to verification tools to verify that it does hold. 
     The term “design” as used herein, may refer to a model of a piece of hardware, which may be described in a hardware description language. A design may typically involve a collection of inputs, outputs, state elements, and combinational functions that compute next state and outputs from current state and outputs. 
     For illustrative purposes client application  22  will be discussed. However, this should not be construed as a limitation of the present disclosure, as other client applications (e.g., client applications  24 ,  26 ,  28 ) may be equally utilized. 
       FIG. 2  is an illustration of a simulation method in accordance with the present disclosure. Referring to  FIG. 2 , a simulation verification method  100  may need to be tested over a universe  101  of possible test points. Testing of the simulation verification method  100  may be performed on a particular area of the universe  101 , for example the cross-hatched area  102 . A portion of the universe  101  may be left untested, for example the untested area  103 . 
     In simulation  100 , only small portions  102  of the universe  101  may be tested. These small tested portions  102  may represent exemplary inputs to test particular functionality of a device under verification. Simulation may often be a desirable method of verification as it may be comparatively less time intensive than formal verification. However, simulation leaves certain portions of the universe  101  untested  103 . 
       FIG. 3  is an illustration of a formal verification method according to the present disclosure. Referring to  FIG. 3 , a formal verification method  110  may be applied to a universe  111  of possible test points. As shown in  FIG. 3 , the tested portion  112  depicted by the crossed hatched area represents the entire universe  111  of possible test points. 
     In formal verification  110 , all portions  112  of the universe  111  may be tested. If a device passes formal verification it is proven to conform to the specification. Generally, no areas are left untested. Formal verification is often a desirable method of verification as the result is mathematical certainty that a design conforms to the specification. However, formal verification may often take much longer than simulation and can result in testing of invalid input combinations that can cause erroneous behavior. For example, the formal verification of an adder having two eight-bit inputs and an eight-bit output would produce erroneous results in the case of an overflow. However, if this adder were implemented in a device constrained such that the inputs never exceeded seven bits, this problem would not occur. 
     Formal verification may include both design style and capacity limitations. In terms of design styles, formal verification generally may only handle the synthesizable subset of hardware description languages (HDL). Simulation on the other hand can handle both synthesizable and non-synthesizable constructs. Formal verification may also have severe capacity limitations. As formal verification is an exhaustive procedure, it is typically limited to block-level design verification. Simulation may be better suited for arbitrary large chip and system on chip designs. 
       FIG. 4  is another illustration of a formal verification method according to the present disclosure. As shown in  FIG. 4 , a design may be subjected to formal verification over a broad range of inputs  122  within the universe  121 . Small areas  123  are left untested. The untested areas  123  could represent impossible combinations of inputs for which there is no need to test or invalid inputs which cause erroneous behavior in the design under test. 
     For example, on a ten-button device where each button is sequentially encoded with a four-bit number, an impossible input would be a four-bit encoding for which no button exists. Given that it is likely the designers did not consider this impossible contingency, such an input could cause non-deterministic behavior and possibly lead to an error state or a dead end. 
     The term “dead end” as used herein, may refer to a state for which there is no solution for the inputs. In this particular case, while formal verification would be possible for a theoretical button eleven, it would be unnecessary to do so. Such a restriction upon the range of inputs during formal verification is called a constraint. Constraints can also be placed on the outputs as well as internal nets. 
     Referring again to  FIG. 4 , the area  123  can represent an area defined by a set of constraints. Formal verification may then be performed over area  122 . However, the area  123  may represent limitations defined by many constraints, some of which may be unnecessary for the successful formal verification of the design. It is possible that some constraints that define the area  123  could be removed, thereby expanding the area defined by  122 . If formal verification of the design passes over an expanded area  122 , then formal verification could be expected to pass over the original area  122  and the removed constraint was unnecessary. Removing the unnecessary constraints may reduce the processing time required to ensure the remaining constraints are not violated and may reduce the time to complete formal verification. 
     One method of formal verification is assertion based verification (ABV). In ABV, a design can be verified by proving that a certain set of assertions is always true. In the example of the eight-bit adder described above, an assertion that verifies the proper function of the adder might specify that the most significant bit of either input is never ONE. If this assertion holds true through formal verification, it can be assured that an overflow condition at output of the adder will not occur. At this point it should be possible to see that it is unlikely that the adder will pass formal verification as the adder has two eight-bit inputs and nothing apparent to limit the inputs to seven-bit numbers. Accordingly, some set of constraints which limits the input to the adder is necessary so that the assertion is not activated. 
     Referring now to  FIG. 5  a logical block diagram of a sixteen-bit comparator is provided. As shown in  FIG. 5  a sixteen-bit comparator  140  may include a first sixteen-bit input  150 , a second sixteen-bit input  160 , and three outputs  170 ,  180 , and  190  indicating equal to, less than, or greater than, respectively. 
     When the first input  150  is equal to the second input  160 , the output  170  may be HIGH and outputs  180  and  190  may be LOW. When the first input  150  is less than the second input  160 , the output  180  may be HIGH and the remaining outputs  170  and  190  may be LOW. When the first input  150  is greater than the second input  160 , the output  190  may be HIGH and the remaining outputs  170  and  180  may be LOW. 
     To fully verify the sixteen-bit comparator shown in  FIG. 5  through exhaustive simulation, every possible combination of sixteen-bit input vectors would need to be tested. If each combination could be tested in one microsecond, and there are 2 16 *2 16  combinations, exhaustive simulation for this simple component could take over an hour. In modern 32-bit and 64-bit systems the time complexity to exhaustively verify a component increases exponentially to thousands of years. Using an assertion, the same verification could be completed in less than a minute. 
     Constraints for formal verification may be chosen using structural cone of influence analysis of the assertion and the constraints. The phrase “cone of influence” as used herein, may refer to all of the nets that are part of the transitive fanin traversal of the net. In other words, by performing the transitive fanin transversal the cone of influence may be obtained. For example, if a net “out=a &amp; b” then “a” and “b” may be in the cone of influence of “out”. In order to identify the cone of influence of an assertion or constraint, obtain the nets on which the assertion or the constraint is written and perform transitive fanin traversal of those nets. All of the nets that are part of this traversal are considered as in the cone of influence of the nets. A constraint is considered important to an assertion if the cone of influence of the constraint intersects with the cones of influence of the assertion. A list of necessary constraints can be obtained by iterating over the list of constraints and finding overlap between their respective cones of influence. 
     The problem with existing structural cone of influence analysis is that even a small intersection of nets results in picking up the constraint. For example, clock or reset nets are common in almost all the constraints and assertions. In these cases all the constraints may be picked up for verification of the assertion. This limitation can be avoided by marking the clock and reset nets as “ignore” nets. An ignore net means that a constraint may not be identified if the only net intersecting the cones of influence of the assertion is the ignore net. The ignore technique can work effectively if extra constraints are picked up because of a global net such as clock or reset. 
     In practical examples, there may be situations where extra constraints are picked up because of a local internal net in the design. Such an example is the case of a power shutoff where one clock domain deliberately shuts off another clock domain. In this case, an ignore technique does not work. Additionally, the ignore approach sometimes results in false positives where necessary constraints are not added because nets were marked as ignore. Further, this technique has not been shown to significantly reduce the number of constraints. 
     ABV has capacity and performance limitations because of the set of constraints. Designs that use ABV may have constraints on inputs, outputs and internal nets. A large set of constraints can add significant complexity to the task of formal verification. The modeling of constraints can require auxiliary code to be written which can include a large portion of the device under test in the cone of influence of the assertion being verified. This in turn may increase the complexity of the verification. 
     Referring now to  FIG. 6 , an exemplary flowchart of a method of performing formal verification according to an embodiment of the present disclosure is provided. As shown in  FIG. 6 , a method of performing formal verification may include attempting  210  to prove an assertion  200  without a list of constraints  205 , determining  215  if the assertion passed or is valid, updating  220  a list of isolated constraints  206 , and reproving  260  the assertion with the isolated constraint list  206 . It should be appreciated that the list of constraints  205  are unisolated and, the unisolated list of constraints may initially comprise all of the known constraints for the integrated circuit design. The isolated list of constraints may be updated during execution of the methods described herein. 
     As discussed above, an exemplary method of the present disclosure may begin by attempting  210  to prove an assertion  200  without any constraints  205 . If the assertion  205  passes at  215 , then no constraints  205  may be necessary and the assertion  200  being verified may be reported  265  as a “pass” without constraints. If, however, the assertion  200  fails at  215 , a constraint from the constraint list  205  may be added  220  to the isolated constraint list  206 . An attempt to reprove  260  the assertion  200  may then be performed, in this particular example, with the list of isolated constraints  206 . The sequence of reproving  260 , and updating  220  the list of isolated constraints  206  may continue until a combination of constraints is obtained, which allows the assertion to pass. If the assertion passes, a “pass” may be reported  265  along with the list of isolated constraints  206  required for the assertion to pass. 
     Referring now to  FIG. 7  an exemplary flowchart corresponding to block  220  of  FIG. 6  is provided.  FIG. 6  and  FIG. 7 , depict an exemplary method of updating  220  the list of isolated constraints  206  in further detail. Updating  220  may include obtaining  225  the counterexample of the cone of influence of the assertion, replaying  230  the counterexample with at least one constraint associated with at least one of the isolated list and the unisolated list (e.g., with remaining constraints  205  enabled), isolating  240  constraints from the list of constraints  205 , and adding  250  the isolated constraints to the list of isolated constraints  206 . The term “counterexample” as used herein, may refer to an example used to disprove a particular statement. In some embodiments, constraints isolated on previous iterations of the method may not enabled when replaying  230  the counterexample. However, it should be noted that when replaying the counterexample, one or more of the isolated or unisolated list may be utilized. 
     After updating  220  the list of isolated constraints  206 , the assertion  200  may be proved again  260 , this time with the list of isolated constraints  206 . On each pass through logical block  220 , one or more constraints may be added  250  to the list of isolated constraints  206 . If the assertion passes, a “pass” may be reported along with the list of isolated constraints  206 . 
     In some embodiments, the constraints may be used to define the design environment. These may be the same Portable Specification Language (PSL), SystemVerilog Assertions (SVA), and Open Verification Libarary (OVL) constraints that may be used to describe the environment in a particular application. Embodiments of the present disclosure may utilize a constraint solver to perform various operations. In this way, the constraint solver may use the combinational or sequential PSL, SVA, and OVL constraints as inputs and may generate simulation traces that meet those constraints. If there is a conflict between constraints during trace generation, then the simulation may reach a dead-end state and may not continue. Generally, constraints are functions over state and input variables of the design. In certain hybrid technologies, constraints may be used to define the design environment and may be solved by the constraint solver. For example, consider a case in which a constraint solver could not generate a solution for the simulation to take place. This may happen due to a conflict between constraints or between constraints and the design. In this case, the simulation may not continue from the existing state, which implies that the simulator has reached a dead-end state. There may be no possible next states in the design due to a conflict between the current design state, pin, or property constraints. In this way, deadend isolation techniques may be used to detect the constraints that are causing the conflict. 
     Referring now to  FIG. 8 , an exemplary flowchart corresponding to block  240  of  FIG. 7  is provided. Isolating  240  constraints may include expanding  241  the cone of influence of the counterexample using a constraint solver and determining  242  if there are any deadends. If no deadends are found the verification is stopped and “fail” may be reported  244  along with any isolated constraints. If a deadend is found, the constraints responsible for the deadend may be isolated  243  and added  250  to the list of isolated constraints (as shown in  FIGS. 7-9 ). 
     If no deadends are found at  242 , then the current set of remaining constraints could not be applied to cause the assertion to pass. In this case, a “fail” may be reported  244  along with any constraints that have been isolated on previous iterations of the method. This failure may be reported to the user so that he/she may debug the design and find the underlying cause of the assertion failure. 
     Referring now to  FIG. 9 , an exemplary flowchart for the combination of blocks  220  and  240  of  FIG. 7  and  FIG. 8  is provided. Updating  220  a list of isolated constraints according to an embodiment of the present disclosure may include obtaining  225  the counterexample of the cone of influence of the assertion, replaying  230  the counterexample with at least one constraint associated with at least one of the isolated list and the unisolated list (e.g., replaying the counterexample with all remaining constraints enabled), isolating constraint(s)  240 , and adding  250  the isolated constraint(s) to the list of isolated constraints. Isolating the constraint(s)  240  may further include expanding  241  the cone of influence of the counterexample using a constraint solver, determining  242  if there are deadends, and isolating  243  the constraint responsible for the deadend. If no deadends are found, the verification may be stopped and a “fail” may be reported  244  along with the list of isolated constraints. 
     Referring now to  FIG. 10  an exemplary flowchart depicting the combination of  FIG. 6  and  FIG. 9  is provided. As shown in  FIG. 10 , an attempt to prove an assertion  200  may be made initially with a list of constraints  205 . If the assertion passes at  215 , a “pass” may be reported  265 . If the assertion fails at  215 , the counterexample of the cone of influence of the assertion may be obtained  225 . The counterexample may then be replayed  230  with at least one constraint associated with at least one of the isolated list and the unisolated list (e.g., with all constraints  205  enabled). In some embodiments, the cone of influence of the counterexample may be expanded  241  using a constraint solver. If no deadends are found at  242 , the verification may be stopped and a “fail” may be reported  244  along with any constraints isolated on previous iterations of the method. If a deadend is found at  242 , the constraint responsible for causing the deadend may be isolated  243  and added  250  to the list of isolated constraints  206 . The assertion  200 , may then be proven again  260 , this time with the list of isolated constraints  206 . The method may then begin a subsequent iteration by determining  215  if the assertion is passing with the list of isolated constraints. 
     Referring now to  FIG. 11 , a flowchart showing an exemplary first iteration through the method of  FIG. 10  is provided. As shown in  FIG. 11 , a first iteration through the method of  FIG. 10  may include attempting to prove an assertion without constraints  310   a , an assertion failure  315   a , replaying the counterexample with all unisolated constraints  330   a , identifying a constraint as responsible for a deadend  340   a , and adding that constraint to a list of isolated constraints  350   a . Lists  206   a ,  205 ,  206   b , and  207   a  of isolated and remaining constraints are maintained on the right for reference. 
     The assertion may be initially proven  310   a  without constraints. Attempting to prove assertions without constraints may increase the efficiency of formal verification as significant processor time may be required to effectively enforce constraints throughout formal verification. However, in this example, the assertion has failed  315   a . The circumstances leading to the assertion failure (the counterexample) may be replayed  330   a  with all constraints  205  enabled. The cone of influence of the counterexample may be expanded using a constraint solver until a constraint which causes a deadend is found. In this example, constraint C3 may be identified as being responsible for the deadend  340   a . If no deadend causing constraint can be identified, then there may be no set of constraints which may cause the assertion to pass and the formal verification fails. Constraint C3 may then be added  350   a  to the list of isolated constraints  206   b  and removed from the list of remaining constraints  207   a  in preparation for the next iteration of the method. 
     Referring now to  FIG. 12  a flowchart showing an exemplary second iteration through the method of  FIG. 11  is provided. As shown in  FIG. 12 , a second iteration through the method of  FIG. 10  may include attempting to reprove the assertion with the isolated constraint list  360   b , an assertion failure  315   b , replaying the counterexample with all unisolated constraints  330   b , identifying a second constraint as responsible for a deadend  340   b , and adding that constraint to a list of isolated constraints  350   b . Lists  206   c ,  207   b ,  206   d , and  207   c  of isolated and remaining constraints are maintained on the right for reference. 
     At the beginning of the second iteration, the assertion may be proven  360   b  with the list of isolated constraints  206   c . At this point, the list of isolated constraints may  206   c  contain only a single constraint, C3. However, even with constraint C3, the assertion fails  315   b . The counterexample may be replayed  330   b  with all remaining constraints C1, C2, and C4  207   b  enabled. The cone of influence of the counterexample may be expanded using a constraint solver until a constraint which causes a deadend is found. In this second iteration, constraint C4 may be identified as being responsible for the deadend  340   b . Constraint C4 may then be added  350   b  to the list of isolated constraints  206   d  and removed from the list of remaining constraints  207   c  in preparation for the next iteration of the method. 
     Referring now to  FIG. 13  a flowchart showing an exemplary third iteration through the method shown in  FIG. 10  is provided. As shown in  FIG. 13 , a third iteration through the method of  FIG. 10  may include attempting to reprove  360   c  the assertion with the isolated constraints  206   e , an assertion pass  315   c , and reporting the pass  365   c  and isolated constraints  206   e . Lists  206   e  and  207   d  of isolated and remaining constraints are maintained on the right for reference. 
     At the beginning of the third iteration, the assertion may be proven  360   c  with the list of isolated constraints  206   e . At this point, the list of isolated constraints  206   e  may contain both C3 and C4. In this example, the assertion may pass  315   c  when both isolated constraints  206   e  C3 and C4 are enabled. The result of the formal verification may be reported  365   c  along with the isolated constraints  206   e . It should be noted that while this example shows three iterations, various numbers of iterations may be necessary in order to perform the methods of the present disclosure. 
     Referring also to  FIGS. 14-15 , embodiments depicting debugging process  11  consistent with the present disclosure are provided. In some embodiments, debugging process  11  may provide a mechanism to debug the unreachable design targets detected by formal verification. Embodiments of debugging process  11  may be used in conjunction with the teachings of constraint process  10 . For example, in some cases, some or all of the aspects of debugging process  11  may be employed prior to utilization of constraint minimization process  10 . However, the present disclosure is not intended to be limited to this example. 
     Formal verification may detect whether a verification check holds true with respect to the given design and environment. Formal verification techniques may be used any suitable design, including but not limited to, verifying RTL designs for the verification checks written using PSL/SVA languages. This may involve multiple flows where the assertions are internally generated. One such flow may include automatically detecting unreachable targets in the design. Formal verification provides a unique counter-example in case a target is hit. The counter-example (e.g. trace) may include the signal transitions leading to the scenario when the target is hit. However, when any target is reported as unreachable by formal tools, the tool does not provide any debug information. In many cases, the targets are unreachable because of some design bug. To understand and fix the design bug, any information provided to the user to debug the cause would bring significant value add to the user. Accordingly, embodiments of debugging process  10  may assist the users in determining the cause associated with the target being unreachable. 
     Embodiments of debugging process  10  may be used to determine a smaller local region of an electronic design (e.g. a “halo”) in which the target is unreachable. In some embodiments, debugging process  10  may utilize iterative formal verification runs. In this way, debugging process may start with the smallest local halo. If the target is unreachable, the halo may be reported back to the user. On the other hand, if the target is hit, the halo may be refined. In some embodiments, refining the halo may include an iterative constraint minimization technique such as those described in U.S. Pat. No. 8,316,332, which is incorporated herein by reference in its entirety. In some embodiments, debugging process  10  may continue refining the halo until the target is unreachable. Once the target becomes unreachable, the halo may be reported back to the user. 
     One possible example consistent with embodiments of debugging process  11  is provided below: 
     /******************************* 
     module top(out, in1, in2, in3, clk); 
     input [3:0]in1, in2, in3; 
     input clk; 
     output [3:0] out; 
     wire [3:0] wire1, wire2, wire3, wire4, wire5, wire6, wire7, wire8; 
     assign wire1=in1; 
     assign wire2=wire1 &amp; in2; 
     assign wire3=wire2; 
     assign wire4=wire3 &amp; in3; 
     assign wire5=0; 
     assign wire6=wire5 &amp; wire4; 
     assign wire8=(wire2 &amp; wire3) A  (wire1 &amp; wire4); 
     assign wire7=wire6 &amp; wire2; 
     assign out=wire7 &amp; wire8; 
     //ps1 COV: cover {out}; 
     endmodule 
     ****************/ 
     Accordingly, 
     the iterative approach described herein may identify nets as: 
     Iteration 1: out 
     Iteration 2: wire7 
     Iteration 3: wire6 
     Iteration 4: wire5 
     Referring again to  FIG. 14 , embodiments of debugging process  11  may include splitting  1402  all of the nets in the cone of influence of the target. For example, in one particular embodiment, one option may include cutting all of the flip-flops. This would provide the local halo at the flop boundary. Additionally and/or alternatively, embodiments may include cutting combinational assignments in case the user is interested in a halo having finer level of granularity. 
     Debugging process  11  may further include, for all split nets, placing  1404  a constraint that re-joins the net. Formal verification may begin by ignoring  1406  all of the constraints and employing an iterative constraint minimization approach such as those discussed above. Ignoring all of the constraints may include may correspond to the initial halo where all the nets are cut, and may correspond to the initial halo of the fan-in cone. If the target is hit in the local halo, iterative constraint minimization may iteratively add constraints. The addition of a constraint may be equivalent to rejoining the split nets. In this way, the halo may be refined efficiently with the addition of required nets. At any iteration, if the target the unreachable, the verification may stop. 
     Debugging process  11  may then determine  1408  if the target is unreachable. If so, the halo may be reported  1410  to the user. This may constitute the minimal halo with which the target is unreachable. Since, this halo is far smaller than the full cone of influence, this would help in the debug process. 
     If the target is not unreachable, debugging process  11  may temporarily join  1412  all of the split nets by adding all of the constraints. Debugging process  11  may replay  1414  the trace using all of the joined nets. There may be a deadend with the replay for some constraint. Debugging process  11  may permanently join  1416  the net corresponding to the reported deadend and split back the temporarily joined nets. 
     In some embodiments, the final result of the iterative constraint minimization may provide a set of constraints with which the target is unreachable. It should be noted that in the penultimate iteration, the target was reachable and only in the last iteration the target became unreachable. Accordingly, the net joined in the last iteration is of interest and may be the probable cause of the unreachable target. 
     In some embodiments, the debugging process shown in  FIG. 14  may be executed for an unreachable cover target. Since, the target is unreachable it may be mandatory that after a few iterations, the target becomes unreachable and the halo is reported to the user. The constraint minimization process discussed above may be used to automatically add required constraints for the verification using the replay technique. 
     Referring now to  FIG. 15 , an embodiment consistent with debugging process  11  is provided. Embodiments may include providing  1502 , via a computing device, an electronic design associated with the integrated circuit. Embodiments may further include splitting  1504  one or more nets in a cone of influence of a target associated with the electronic design. In some embodiments, splitting one or more nets may include splitting all of the nets associated with the cone of influence. For each split net, embodiments may include placing  1506  a constraint that re-joins the net. Embodiments may also include identifying  1508  a local region of the electronic design for which the target is unreachable. During formal verification, embodiments may include ignoring  1510  all constraints associated with the local region of the electronic design. 
     As discussed above, mutation coverage is a fault based testing technique that measures the effectiveness of test suites. Faults may be introduced into the system by creating a set of faulty versions called Mutants. Test cases may be used to execute these mutants with the goal of causing mutants to produce incorrect output. The mutants causing incorrect output may be considered covered for a given set of test suites. 
     Referring now to  FIG. 16 , an embodiment depicting a mutation associated with a multiplexer is shown. In this example, a mutation may be applied in such a way that there are more behaviors of the design. That is, the original behavior and also the mutated behavior. To activate the mutation, the multiplexer controlling the value “Input” remains a free net. Since, formal verification tools may perform verification for both the values of the “input” net {0,1}, this ensures that the mutated version of the design has mutant behavior and also the original behavior. It should be noted that the mutant may be of any type, including, but not limited to net inversion, net cut, etc. When “input” is tied to value “1” using a constraint, the design behaves just like the original design. 
     There are many categories of mutations which are commonly used to get the coverage. One category of mutation is flop-based mutation. There are a few other combinational mutations also used for the mutation coverage. 
     For assertion based verification, the quality of assertions written by the user can be measured based upon the mutation coverage. The idea is to apply several mutations in the design to see if the assertion starts failing after applying the mutation. Mutation coverage can also help to identify the over constraint scenario under which the assertions have passed. 
     As discussed above, one approach to mutation is to cut a flop. Some existing techniques may perform a random flop cut and check the assertion status. This step involves one formal verification run. This needs to be repeated for every flop and for every assertion and for every mutation category. However, this tends to be an expensive approach as every flop in the cone of influence may need to be mutated and checked one by one. Moreover, this needs to be performed for every assertion and for every mutation category. 
     Referring now to  FIGS. 17-18 , embodiments of the present disclosure consistent with mutation coverage process  13  are provided. Mutation coverage process  13  may be configured to provide a guided solution that addresses the limitations identified above. 
     As is shown in  FIG. 17 , this particular embodiment explains how mutation coverage may be generated using formal verification. The coverage may be generated for assertions that are proven correct by the formal verification. This particular example explains the coverage generation for one assertion. 
     Embodiments of mutation coverage process  13  may include generating ( 1702 ) one or more mutants in the cone of influence of the assertion. For each mutant, process  13  may further include placing ( 1704 ) a constraint that models the original behavior of the design for that mutant. Formal verification may be initiated ( 1706 ) by ignoring all of the constraints. In other words, all of the mutants may be active. Mutation coverage process  13  may include determining ( 1708 ) whether or not the assertion passes. If the assertion does not pass, mutation coverage process  13  may include temporarily activating ( 1710 ) the original design by adding all of the constraints and replaying ( 1712 ) the trace on the original design. Mutation coverage process  13  may then determine ( 1714 ) if there is a deadend. If not, the process may stop ( 1716 ) and report the failure of the assertion. No coverage data may be generated. If there is a deadend, process  13  may obtain ( 1718 ) the mutant for the deadend and permanently activate the original design for this mutant. The mutant may be collected in a mutant list. The remaining mutants may be activated as well. The assertion passes by activating ( 1720 ) the original design for certain mutants. The process may also refine the list of mutants. For each mutant in the list, mutation coverage process  13  may activate ( 1722 ) a single mutant from the list one by one. Once all the mutants are tried, the new mutant list may be the mutation coverage reported. Mutation coverage process  13  may determine ( 1724 ) if the assertion passes. If not, the process may add ( 1726 ) this mutant to the new mutant list. If the assertion does pass, this mutant may be discarded ( 1728 ). 
     Accordingly, in some embodiments, mutation coverage process  13  may address each mutation category separately. For each category, embodiments of the present disclosure may apply mutation on all the design elements. For example, if flop cut based mutation is used, a cut may be applied on all the flops. At this point, an automatic iterative scheme may be employed to determine which of those mutations are actually covered by the set of assertions. Once the analysis is over the solution may provide a comprehensive coverage for all the assertions. Since this process began with all the mutations, this differs from existing solutions as the traditional solution works on a single mutation/fault mode. Embodiments of mutation coverage process  13  may be configured to handle multiple mutations/faults. In this way, the iterative scheme described herein may ensure that the solution is guided. 
     In some embodiments, mutation coverage process  13  may include, for each passing assertion and/or for each mutation category, applying mutation on each design element. Mutation coverage process  13  may further include running the iterative approach described herein to determine which of the mutations are covered. 
     One example of the mutation is a flop cut. Accordingly, embodiments of mutation coverage process  13  may be used to model this mutation by cutting the flop and applying a constraint which rejoins the flop. In some embodiments, mutation coverage process  13  may work in conjunction with the iterative constraint minimization scheme described herein and in U.S. Pat. No. 8,316,332, which is incorporated herein by reference in its entirety. Accordingly, the process may start by applying a cut on every flop (this may imply that all mutations are active) and iteratively bring in those mutations that are relevant. 
     Referring to  FIG. 18 , an embodiment of mutation coverage process  13  is provided. Mutation coverage process  13  may include providing ( 1802 ), via a computing device, an electronic design associated with the integrated circuit and generating ( 1804 ) one or more faults in a cone of influence of an assertion. Mutation coverage process  13  may further include placing ( 1806 ) a constraint configured to model an original design for the one or more faults and initiating ( 1808 ) formal verification on the electronic design while ignoring all electronic design constraints. Mutation coverage process  13  may further include determining ( 1810 ) if the assertion is passing, wherein determining includes activating an original design for a subset of faults. If the assertion is passing, mutation coverage process  13  may include activating ( 1812 ) a single fault from the subset and determining ( 1814 ) if the assertion is passing. If the assertion does pass, mutation coverage process  13  may include deleting ( 186 ) the single fault from the subset. 
     Embodiments of mutation coverage process  13  may provide a single step solution whereas traditional solutions require multiple steps. As discussed above, the traditional solution requires one verification run to figure out the passed assertions. Only after that can the mutation coverage begin. Embodiments of mutation coverage process  13  may be applied upfront which generates coverage numbers along with the verification. Moreover, the iterative constraint minimization scheme may bring in the mutations in a guided manner. In contrast, the traditional solutions apply mutations randomly and gather the coverage information. Additionally and/or alternatively, embodiments of mutation coverage process  13  may be capable of handling a multi-fault model whereas the traditional approach can only handle a single fault model. In this way, embodiments of mutation coverage process  13  may be configured to handle cases where more than one mutation causes the assertion to fail. 
     It will be apparent to those skilled in the art that various modifications and variations can be made in the constraint minimization scheme and debugging process of embodiments of the present disclosure without departing from the spirit or scope of the invention. Thus, it is intended that embodiments of the present disclosure cover the modifications and variations of this invention provided they come within the scope of the appended claims and their equivalents.