Artificial intelligence-based constrained random verification method for design under test and non-transitory machine-readable medium for storing program code that performs artificial intelligence-based constrained random verification method when executed

An artificial intelligence (AI)-based constrained random verification (CRV) method for a design under test (DUT) includes: receiving a series of constraints; obtaining a limited constraint range according to the series of constraints; generating a series of stimuli according to the limited constraint range; and verifying the DUT by the series of stimuli; wherein at least one of the step of obtaining the limited constraint range according to the series of constraints and the step of generating the series of stimuli according to the limited constraint range employs an AI algorithm.

BACKGROUND

The present invention relates to constrained random verification (CRV), and more particularly, to an artificial intelligence (AI)-based CRV method and a non-transitory machine-readable medium for storing a program code that performs the AI-based CRV method when executed.

The CRV is a simulation-based method for register transfer level (RTL) design verification, and is arranged to generate multiple stimuli according to multiple constraints to interact with a design under test (DUT), wherein the multiple stimuli is arranged to verify performance of the DUT in a testbench. As the design becomes more and more complex, the stimulus space of the CRV becomes larger. The CRV will be harder to generate specific stimuli to hit a corner case. Hence, the coverage closure becomes a challenging problem for design verification engineers and requires a lot of laborious human effort to complete. Take a memory management unit (MMU) with at least one first in first out (FIFO) as an example. The corner case can be to verify whether a functional operation is working or not when the at least one FIFO in the MMU is full. The problem is how to determine a sequence of constraints to trigger PUSH behaviors of the at least one FIFO as possible as it could to fill up the at least one FIFO. As a result, a novel framework for the CRV to increase a hit rate of the corner case in design verification without human expert guidance is urgently needed.

SUMMARY

It is therefore one of the objectives of the present invention to provide an AI-based CRV method and a non-transitory machine-readable medium for storing a program code that provides the AI-based CRV method when executed, to address the above-mentioned issues.

According to an embodiment of the present invention, an AI-based CRV method for a DUT is provided. The AI-based CRV method includes: receiving a series of constraints; obtaining a limited constraint range according to the series of constraints; generating a series of stimuli according to the limited constraint range; and verifying the DUT by the series of stimuli; wherein at least one of the step of obtaining the limited constraint range according to the series of constraints and the step of generating the series of stimuli according to the limited constraint range employs an AI algorithm.

According to an embodiment of the present invention, a non-transitory machine-readable medium for storing a program code is provided, wherein when loaded and executed by a processor, the program code instructs the processor to perform an AI-based CRV method for a DUT, and the AI-based CRV method includes: receiving a series of constraints; obtaining a limited constraint range according to the series of constraints; generating a series of stimuli according to the limited constraint range; and verifying the DUT by the series of stimuli; wherein at least one of the step of obtaining the limited constraint range according to the series of constraints and the step of generating the series of stimuli according to the limited constraint range employs an AI algorithm.

One of the benefits of the present invention is that, by the two-stage framework combined with the CRV of the present invention, the hit rate of the corner case for the DUT in the design verification can be increased, which can greatly improve the verification quality and shorten the time-to-market. In addition, in some embodiments, the two-stage framework of the present invention can be modified to only include the constraint selected stage or only include the stimuli generated stage, which can also increase the hit rate of the corner case for the DUT in the design verification.

DETAILED DESCRIPTION

Certain terms are used throughout the following description and claims, which refer to particular components. As one skilled in the art will appreciate, electronic equipment manufacturers may refer to a component by different names. This document does not intend to distinguish between components that differ in name but not in function. In the following description and in the claims, the terms “include” and “comprise” are used in an open-ended fashion, and thus should be interpreted to mean “include, but not limited to . . . ”.

FIG.1is a diagram illustrating an electronic device10according to an embodiment of the present invention. Byway of example, but not limitation, the electronic device10may be a portable device such as a smartphone or a tablet. The electronic device10may include a processor12and a storage device14. The processor12may be a single-core processor or a multi-core processor. The storage device14is a non-transitory machine-readable medium, and is arranged to store computer program code PROG, multiple models (e.g. a selector and a generator), and a series of constraints CONS for the constrained random verification (CRV). The processor12is equipped with software execution capability. The computer program code PROG may include multiple artificial intelligence (AI)-based algorithms (e.g. a transformer-based algorithm and a reinforcement learning (RL)-based algorithm). When loaded and executed by the processor12, the computer program code PROG instructs the processor12to train the selector according to the transformer-based algorithm and/or train the generator according to the RL-based algorithm. The electronic device10may be regarded as a computer system using a computer program product that includes a computer-readable medium containing the computer program code PROG. Regarding an AI-based CRV method for a design under test (DUT) as proposed by the present invention, it may be embodied on the electronic device10.

Please refer toFIG.2.FIG.2is a flow chart of an AI-based CRV method according to an embodiment of the present invention, wherein the concept of CRV and the concept of machine learning are combined in the AI-based CRV method for increasing a hit rate of a corner case in design verification of a DUT without human expert guidance. For example, the DUT may be a memory management unit (MMU) with at least one first in first out (FIFO), and the corner case may be to verify whether a functional operation is working or not when the at least one FIFO in the MMU is full, but the present invention is not limited thereto. The AI-based CRV method of the present invention can also be applied to other product verification according to design requirements. In some embodiments of the present invention, the computer program code PROG instructs the processor12to perform the AI-based CRV method when loaded and executed by the processor12.

In Step S200, the series of constraints CONS that are arranged to determine a stimulus space may be stored in the storage device14. For example, a series of virtual addresses that are transmitted to an MMU may be arranged to trigger PUSH behaviors or POP behaviors of the at least one FIFO in the MMU, and may be regarded as the series of constraints CONS. It is assumed that the length of the series of constraints CONS is 30, and for each element in the series of constraints CONS, one of 16 constrained parameters (e.g. 0-15) may be selected to represent the amount of adjustment between the corresponding virtual address and the previous virtual address. As a result, the stimulus space for the series of constraints CONS may be 1630.

In Step S202, a range of the series of constraints CONS (labeled as “CR” inFIG.2) is refactored in ascending order to obtain multiple sub-ranges SRS, a selector200is trained according to a transformer-based algorithm and the multiple sub-ranges SRS, and a selected constraint range SCR is selected from the multiple sub-ranges SRS by the selector. For example, since the more the PUSH behaviors of the at least one FIFO are triggered, the easier it is to make the FIFO full, one of the multiple sub-ranges SRS with the highest number of PUSH counts is selected from the multiple sub-ranges SRS as the selected constraint range SCR by the selector200. It is assumed that a specific range (e.g. 2-5) is selected as the selected constraint range SCR, the stimulus space for the selected constraint range SCR may be 430, which is much smaller than that for the series of constraints CONS (e.g. 1630).

In Step S204, a generator202is trained according to an RL-based algorithm and the selected constraint range SCR, to generate a series of stimuli STI, wherein the hit rate of the corner case of the DUT is increased through the series of stimuli STI.

In detail, please refer toFIG.3.FIG.3is a diagram illustrating a two-stage framework300for the CRV according to an embodiment of the present invention, wherein the two-stage framework300is designed to replace manual tuning of constraints and random generation of stimuli in CRV. The two-stage framework300may include a constraint selected stage302and a stimuli generated stage304, the Step S202shown inFIG.2may be implemented by a selector301in the constraint selected stage302, and the Step S204shown inFIG.2may be implemented by a generator303in the stimuli generated stage304.

In the constraint selected stage302, simulations between a DUT305(e.g. an MMU with at least one FIFO) and one of the multiple sub-ranges SRS are performed to obtain a target value. For example, the target value may be a number of PUSH counts of the at least one FIFO in the MMU. The selector301may be trained according to said one of the multiple sub-ranges SRS and the target value through the transformer-based algorithm. After the selector301fits said one of the multiple sub-ranges SRS, target values of others of the multiple sub-ranges SRS may be predicted according to the selector301. In this way, the selector301can learn patterns between the multiple sub-ranges SRS and select one of the multiple sub-ranges SRS that can trigger the highest number of PUSH counts as the selected constraint range SCR. Under a condition that an original range set of the series of constraints CONS is a range set {0, 1, 2, . . . , M−1}, the selected constraint range SCR may be a range set {m1, m1+1, . . . , m2−1}, wherein M is a positive integer (i.e. M≥1), m1is an integer (i.e. m1≥0), and m2is smaller than or equal to M (i.e. m2≤M).

In detail, please refer toFIG.4.FIG.4is a diagram illustrating a selector400according to an embodiment of the present invention, wherein the selector301shown inFIG.3may be implemented by the selector400. As shown inFIG.4, a structure of the selector400may include a multilayer perceptron (MLP)402, a transformer encoder404, and an MLP406. In the beginning, since all of the multiple sub-ranges SRS are in ascending order, a positional encoding is performed upon the multiple sub-ranges SRS, to obtain position information PI for subsequent process. Regarding the transformer encoder404, the transformer encoder404is an encoder in a transformer model, and a self-attention mechanism of the transformer encoder404is utilized to build the selector400for automatically selecting the selected constraint range SCR. Finally, the output of the transformer encoder404is transmitted to the MLP406for predicting the target values of others of the multiple sub-ranges SRS. In this way, the selector400may be arranged to select the selected constraint range SCR from the multiple sub-ranges SRS, and transmit the selected constraint range SCR to the generator303in the stimuli generated stage304.

Please refer back toFIG.3. In the stimuli generated stage304, after receiving the selected constraint range SCR from the selector301/400, the generator303may learn to generate the series of stimuli STI according to the selected constraint range SCR through the RL-based algorithm, to achieve the corner case of the DUT305(e.g. fill up the at least one FIFO of the MMU). In detail, please refer toFIG.5.FIG.5is a diagram illustrating a generator500according to an embodiment of the present invention, the generator303shown inFIG.3may be implemented by the generator500. As shown inFIG.5, the generator500and the DUT305may act as an agent and an environment for RL, respectively. For each constraint in the selected constraint range SCR, the generator500may interact with the DUT305to obtain a target value (e.g. a number of PUSH counts) as a reward506for RL. The generator500may be trained according to an action502, a state504, and the reward506through the RL-based algorithm (i.e. the generator500may generate the action502to interact with the DUT305, and the DUT305may return the state504and the reward506back to the generator500), and may learn how to generate an action with the maximum reward.

Regarding the action502, under a condition that the length of the series of stimuli STI is N and the length of the selected constraint range SCR is T, N cycles are required to be executed for the series of stimuli STI. For each cycle, the generator500may select a constraint from the selected constraint range SCR to generate a stimulus for interacting with the DUT305. As a result, the action space for the selected constraint range SCR may be TN. Under a condition that an original range set of the series of constraints CONS is a range set {0, 1, 2, . . . , M−1}, the selected constraint range SCR may be a range set {m1, m1+1, . . . , m2−1}, the original action space may be MN, and the action space corresponding to the selected constraint range SCR may be (m2-m1)N, wherein M is a positive integer (i.e. M≥1), m1is an integer (i.e. m1≥0), m2is smaller than or equal to M (i.e. m2≤M), and the action space corresponding to the selected constraint range SCR may be reduced to be smaller than the original action space. In addition, an actor-critic method is applied to the generator500. As shown inFIG.5, the generator500may include an actor model508and a critic model510, wherein the actor model508may be arranged to predict action of N cycles every iteration, the critic model510may be arranged to judge every action predicted by the actor model508, and a proximal policy optimization principle (PPO2) is utilized as an objective function to optimize the actor model508and the critic model510.

Regarding the state504, the initial state may be encoded as a list of 0 with the same length of the series of stimuli STI. For each cycle, the state504will be updated to the collection of the previous actions. Since operations of the RL through the action502, the state504, and the reward506are well known to those skilled in the art, the details of the RL will not be described in the specification of the present invention.

After the series of stimuli STI are generated by the generator303, a scoreboard307may be arranged to verify the functionality of the DUT305according to the series of stimuli STI, to generate a report309, wherein the hit rate of the corner case of the DUT305is increased through the series of stimuli STI. Since operations of the CRV are well known to those skilled in the art, the details of the CRV will not be described in the specification of the present invention.

It should be noted that, in some embodiments, a single constraint selected stage (e.g. the constraint selected stage302) combined with the CRV can also increase the hit rate of the corner case for the DUT. That is, the two-stage framework300shown inFIG.3can be modified to only include the constraint selected stage302. Please refer toFIG.6.FIG.6is a diagram illustrating a single constraint selected stage600for the CRV according to an embodiment of the present invention. As mentioned above, in the single constraint selected stage600, a selector601is trained according to the transformer-based algorithm and the multiple sub-ranges SRS, and the selected constraint range SCR is selected from the multiple sub-ranges SRS by the selector601. Afterwards, the CRV method is utilized for further simulations toward a DUT602, to generate multiple stimuli, wherein the hit rate of the corner case for the DUT602can also be increased through the multiple stimuli. After the series of stimuli STI are generated by the CRV, a scoreboard603may be arranged to verify the functionality of the DUT602according to the series of stimuli STI, to generate a report604. For brevity, similar descriptions for these embodiments are omitted here.

In addition, in some embodiments, a single stimuli generated stage (e.g. the stimuli generated stage304) can also increase the hit rate of the corner case for the DUT. That is, the two-stage framework300shown inFIG.3can be modified to only include the stimuli generated stage304. Please refer toFIG.7.FIG.7is a diagram illustrating a single stimuli generated stage700for the CRV according to an embodiment of the present invention. In the beginning, the range of the series of constraints CONS (labeled as “CR” inFIG.7) stored in the storage device14can be tuned and shrunk to obtain a limited constraint range LCR, wherein the limited constraint range LCR may be transmitted to a generator701in the single stimuli generated stage700. Afterwards, the generator701may learn to generate the series of stimuli STI according to the limited constraint range LCR through the RL-based algorithm, to achieve the corner case of a DUT702(e.g. fill up the at least one FIFO of the MMU). In this way, the hit rate of the corner case for the DUT702can also be increased through the series of stimuli STI. After the series of stimuli STI are generated by the generator701, a scoreboard703may be arranged to verify the functionality of the DUT702according to the series of stimuli STI, to generate a report704. For brevity, similar descriptions for these embodiments are omitted here.

In summary, by the two-stage framework300combined with the CRV of the present invention, the hit rate of the corner case for the DUT in the design verification can be increased, which can greatly improve the verification quality and shorten the time-to-market. In addition, in some embodiments, the two-stage framework300of the present invention can be modified to only include the constraint selected stage302or only include the stimuli generated stage304, which can also increase the hit rate of the corner case for the DUT in the design verification.