# Patent

## Title
Fault support in an emulation environment

## Abstract

An emulator is disclosed that allows for diagnoses of failures or defects within the emulator. A map of faulty resources is generated to identify which resources should be avoided during compilation. Thus, in a transparent and automated manner, defects found during diagnostics are stored in a database of unusable emulator resources. A compiler has access to the database and compiles the design taking into account unusable resources. In another embodiment, the defects of an emulator board are stored on the emulator board itself. This allows each board to store its own maintenance information that can be used at the manufacturing site for changing defective chips. Defects stored on the board itself allow the defects to be obtained independent of a position of a board within the emulator to simplify identification of the faulty resource.

## Background
FIELD
The present disclosure generally relates to hardware emulators, and more particularly to monitoring faults in a hardware emulator.
BACKGROUND
Today's sophisticated SoC (System on Chip) designs are rapidly evolving and nearly doubling in size with each generation. Indeed, complex designs have nearly exceeded 50 million gates. This complexity, combined with the use of devices in industrial and mission-critical products, has made complete design verification an essential element in the semiconductor development cycle. Ultimately, this means that every chip designer, system integrator, and application software developer must focus on design verification.
Hardware emulation provides an effective way to increase verification productivity, speed up time-to-market, and deliver greater confidence in the final SoC product. Even though individual intellectual property blocks may be exhaustively verified, previously undetected problems can appear when the blocks are integrated within the system. Comprehensive system-level verification, as provided by hardware emulation, can test overall system functionality, IP subsystem integrity, specification errors, block-to-block interfaces, boundary cases, and asynchronous clock domain crossings. Although design reuse, intellectual property, and high-performance tools all help by shortening SoC design time, they do not diminish the system verification bottleneck, which can consume 60-70% of the design cycle. As a result, designers can implement a number of system verification strategies in a complementary methodology including software simulation, simulation acceleration, hardware emulation, and rapid prototyping. But, for system-level verification, hardware emulation remains a favorable choice due to superior performance, visibility, flexibility, and accuracy.
A short history of hardware emulation is useful for understanding the emulation environment. Initially, software programs would read a circuit design file and simulate the electrical performance of the circuit very slowly. To speed up the process, special computers were designed to run simulations as fast as possible. IBM's Yorktown “simulator” was the earliest (1982) successful example of this as it used multiple processors running in parallel to run the simulation. Each processor was programmed to mimic a logical operation of the circuit for each cycle and could be reprogrammed in subsequent cycles to mimic a different logical operation. This hardware ‘simulator’ was faster than the current software simulators, but far slower than the end-product ICs. When Field Programmable Gate Arrays (FPGAs) became available in the mid-80's, circuit designers conceived of networking hundreds of FPGAs together in order to map their circuit design onto the FPGAs and the entire FPGA network would mimic, or emulate, the entire circuit. In the early 90's the term “emulation” was used to distinguish reprogrammable hardware that took the form of the design under test (DUT) versus a general purpose computer (or work station) running a software simulation program.
Soon, variations appeared. Custom FPGAs were designed for hardware emulation that included on-chip memory (for DUT memory as well as for debugging), special routing for outputting internal signals, and for efficient networking between logic elements. Another variation used custom IC chips with networked single bit processors (so-called processor based emulation) that processed in parallel and usually assumed a different logic function every cycle.
Physically, a hardware emulator resembles a large server. Racks of large printed circuit boards are connected by backplanes in ways that most facilitate a particular network configuration. A workstation connects to the hardware emulator for control, input, and output. Before the emulator can emulate a DUT, the DUT design must be compiled. That is, the DUT's logic must be converted (synthesized) into code that can program the hardware emulator's logic elements (whether they be processors or FPGAs). Also, the DUT's interconnections must be synthesized into a suitable network that can be programmed into the hardware emulator. The compilation is highly emulator specific and can be time consuming. After the compilation is complete, the design is downloaded into the emulator.
Some integrated circuits in the emulator can be faulty or become faulty over time (i.e., malfunctioning or defective), which can lead to an unacceptable emulation of a downloaded design. Correction through re-compilation is time consuming and wasteful.
SUMMARY
An emulator is disclosed that allows for diagnoses of failures or defects within the emulator. A map of faulty resources can be generated to identify which resources should be avoided during compilation. Thus, desirably in a transparent and automated manner, emulator defects found during diagnostics can be stored in a database or in formatted output files. A compiler has access to the database or formatted output files and can compile the design taking into account (e.g., avoiding the use of) defective or unusable resources.
In another embodiment, the defects of an emulator board can be stored on the emulator board itself. This allows each board to store its own maintenance information that can be used, for example, at the manufacturing site for changing defective chips. Defects stored on the board itself allow the defects to be obtained independent of the position of a board within the emulator to simplify identification of the faulty resource.
In another embodiment, the emulator can check the current state of faulty resources on an emulator board and compare the faults to a compiled design to be downloaded. If the compiled design uses faulty resources an error condition can be indicated.
In another embodiment, a unique database is created when powering on the emulator, built from fault information received from the emulator boards. When new faults are detected during diagnostics or input by a user, such faults are sent to the appropriate emulator boards containing those faults for being stored onboard in non-volatile memory.
The foregoing features and advantages of embodiments will become more apparent from the following detailed description, which proceeds with reference to the accompanying figures.



BRIEF DESCRIPTION OF THE DRAWINGS
FIG. 1 is a system diagram of an exemplary hardware emulator environment.
FIG. 2 is a more detailed system diagram for an exemplary system comprising multiple host computers coupled to the emulator through an intermediate platform maintenance board.
FIG. 3 is an exemplary high-level system diagram showing various servers connected through a messaging bus.
FIG. 4 is a three-dimensional physical view of an embodiment of a system of FIG. 1 including multiple printed circuit boards coupled together.
FIG. 5 is a detailed view of an exemplary emulator printed circuit board of FIG. 4.
FIG. 6 is a flowchart of an embodiment of a method for using fault information in compiling a design to be emulated.
FIG. 7 is a flowchart of an embodiment of a method for using a fault avoidance technique in compiling a design to be emulated.
FIG. 8 is a flowchart of a method providing further details of an example that uses a fault avoidance technique of FIG. 7 in compiling a design to be emulated.
FIG. 9 is a flowchart of an embodiment of a method for checking if the design being loaded into the emulator is using fault-free resources.
FIG. 10 is a flowchart of an embodiment of a method for storing fault information on an emulator board.



DETAILED DESCRIPTION
Disclosed below are representative embodiments of electronic circuit testing techniques and associated apparatus that should not be construed as limiting in any way. Instead, the present disclosure is directed toward all novel and nonobvious features and aspects of the various disclosed methods, apparatus, and equivalents thereof, alone and in various combinations and subcombinations with one another. The disclosed technology is not limited to any specific aspect or feature, or combination thereof, nor do the disclosed methods and apparatus require that any one or more specific advantages be present or problems be solved.
As used in this application and in the claims, the singular forms “a,” “an” and “the” include the plural forms unless the context clearly dictates otherwise. Additionally, the term “includes” means “comprises.” Moreover, unless the context dictates otherwise, the term “coupled” means electrically or electromagnetically connected or linked and includes both direct connections or direct links and indirect connections or indirect links through one or more intermediate elements.
Although the operations of some of the disclosed methods and apparatus are described in a particular, sequential order for convenient presentation, it should be understood that this manner of description encompasses rearrangement, unless a particular ordering is required by specific language set forth below. For example, operations described sequentially may in some cases be rearranged or performed concurrently. Moreover, for the sake of simplicity, the attached figures may not show the various ways in which the disclosed methods and apparatus can be used in conjunction with other methods and apparatus.
Any of the methods described herein can be performed (at least in part) using software comprising computer-executable instructions stored on one or more computer-readable media. Furthermore, any intermediate or final results of the disclosed methods can be stored on one or more computer-readable media. For example, a software tool can be used to determine and store one or more control signals used to control any of the disclosed apparatus. Any such software can be executed on a single computer or on a networked computer (for example, via the Internet, a wide-area network, a local-area network, a client-server network, or other such network). For clarity, only certain selected aspects of the software-based implementations are described. Other details that are well known in the art are omitted. For the same reason, computer hardware is not described in further detail. It should be understood that the disclosed technology is not limited to any specific computer language, program, or computer. For instance, a wide variety of commercially available computer languages, programs, and computers can be used.
FIG. 1 shows an exemplary emulator environment 10 including a hardware emulator 12 coupled to one or more hardware emulator hosts 14. The emulator host 14 may be any desired type of computer hardware and generally includes a user interface through which a user can load, compile and download a design to the emulator 12. Additionally, the user can visualize physical parameters associated with the emulator through a graphical user interface (GUI) on any of the emulator hosts 14, as further described below. The illustrated emulator 12 includes a monitoring portion 16 and an emulation portion 18. The emulation portion 18 includes multiple printed circuit boards 20 coupled to a midplane 22. The midplane 22 allows physical connection of the printed circuit boards into the emulator 12, in this example, on both sides of the midplane. A backplane may also be used in place of the midplane, the backplane allowing connection of printed circuit boards on one side of the backplane. Any desired type of printed circuit boards may be used. For example, programmable boards 24 generally include an array of FPGAs, VLSIs or ICs, or other programmable circuitry, that may be programmed with the user's design downloaded from the emulator host 14. One or more I/O board interfaces 26 can allow communication between the emulator 12 and hardware external to the emulator. For example, the user may have a preexisting processor board that is used in conjunction with the emulator and such a processor board connects to the emulator through I/O board interface 26. Clock board 28 can generate any number of desired clock signals. Interconnect boards 30 can allow integrated circuits on the programmable boards 24 to communicate together and with integrated circuits on the I/O board interface 26.
FIG. 2 shows a more detailed view of an exemplary system. The multiple host computers 14 are coupled together through a network 40, such as a LAN, but other networks can also be used. The host computers 14 can be equipped with a high-speed-link PCI board coupled to a platform maintenance board (PMB) 42, which acts as an exemplary form of the monitoring portion 16. The PMB 42 can monitor various physical parameters in the emulator portion 18 and can create the interface between the emulator portion 18 and the host computers 14. The PMB 42 on a periodic basis (e.g., 10 seconds) transmits communication and monitoring reports to the host workstations 14 for display in the GUI. Similarly, the PMB 42 may receive information regarding the physical parameters of the emulator portion 18, such as periodically. For example, hardware (e.g., an FPGA) on each printed circuit board 20 typically has intelligence for monitoring physical parameters on its respective board and for sending this physical information to the PMB (e.g., every 5 seconds). Other changes, such as a detected error, can be transmitted immediately upon and in response to the detection. Thus, the PMB 42 may instantaneously (as opposed to periodically) detect any changes in the emulation environment 10 and generate real-time state change messages to the host stations 14. All of the physical parameters obtained through the PMB desirably can be obtained while the emulator portion 18 is performing emulation. Thus, several emulations may be separately running and the physical parameters of the emulator may separately be viewed on the GUI of the host computers. However, there need not be a link between the number of simultaneous emulations and the number of workstations. For example, several emulations can be simultaneously run through one workstation. The printed circuit boards 20 can be grouped in a one-to-one correspondence with the number of host computers. This grouping allows one host computer to be associated with a group of boards 20 so that multiple high-speed links can be used in parallel. The grouping used can be easily be modified based on the design or not used at all. For example, all boards can be chained together into one group that correspond with one host computer. IO boxes 46 can allow the connection of other user boards to the system. The IO boxes 46 can also be coupled to the PMB 42 and monitored thereby.
FIG. 3 shows a view of a form of the emulator system including various servers 60 that communicate through a messaging bus 62. Emulator server 64 can be in charge of managing one physical host connection to the emulator and provide a way to transfer data between the emulator messaging bus 62 and the emulator 12. If there is more than one emulator, then additional servers 64, 66, 68 can be used for each emulator. The maintenance server 66 can be in charge of diagnostics, and storing maintenance information collected from other applications, servers, and/or emulator boards. The resource server 68 can be in charge of managing the different emulator resources provided to the applications. The virtual maintenance panel (VMP) 67, in this example, is the main user interface through which the user can control the system, monitor and control the emulators. The virtual maintenance panel can also be coupled to a faults database 72 that stores information about components (e.g., integrated circuits) on printed circuit boards (e.g., programmable boards 20) in the emulator 18 that are not properly functioning. Thus, the faults may be stored externally to an emulator board. A compiler 74 can be coupled to a database 76 containing the user's design to be emulated. The compiler 74 is also desirably coupled to the resource server 68 and the faults database 72 (or the compiler may read fault files generated by the virtual maintenance panel instead of the database 72). When compiling the design from database 76, the compiler can access either the faults database 72, the resource server 68 or both in order to obtain a list of components (e.g., integrated circuits) that are malfunctioning and should not be used. The compiler 74 may also obtain defective IC information from the maintenance server 66 instead of the resource server by coupling the compiler to the maintenance server. In addition, the compiler 74 may receive user input regarding fault avoidance 80. The fault avoidance options can allow a user to control what resources or components the compiler considers accessible. For example, under one fault avoidance option that can be called “flat”, there is a direct one-to-one correspondence between components that are faulty and the components that the compiler avoids using. For example, if IC0 is the only IC not working on PCB0, then only IC0 on PCB0 is not used. In a second option that can be called “generic slot”, if IC0 is the only IC not working on PCB0, then all IC0s on the other PCBs are also not used. This gives the user assurance that they can load their design in any PCB in the emulator and it will function. In a third option that can be called “quad”, the PCBs are grouped in groups of four. If a particular IC in the first PCB of one member of the quad is defective, the first PCB of each of the other quads will also be treated as if faulty and are not used. Thus if IC0 on PCB0 is not functioning, then IC0 on PCB4, PCB8, PCB12, and PCB16 are considered not working. This ensures that the user can download their design into any quad and it will function properly. Other fault avoidance techniques can be used depending on the design. Another example can be where there is a combination of faulty components on multiple PCBs. In such a case, the faults can be combined to create a combined fault list. For example, if IC0 on PCB0 is bad and IC1 on PCB1, the combined faults are that both IC) and IC1 can be considered faulty on all boards, even if those ICs are not necessarily faulty on the other boards.
FIG. 4 shows a physical three-dimensional view of an embodiment of the emulator portion 18 including the midplane 22 having horizontal printed circuit boards 80 coupled to one side of the midplane, and vertical printed circuit boards 82 coupled to the opposite side of the midplane. The physical integrated circuits are shown at 84. The IO boxes 46 sit separately and are not generally considered part of the emulator. Other connection schemes can be used such as a motherboard or backplane.
FIG. 5 shows an embodiment of an exemplary emulator PCB 20 in more detail. For this particular example, the PCB is a programmable board 24, but other PCBs may have a similar or different structure. The illustrated PCB 24 includes multiple programmable ICs 100 (e.g., FPGAs) that are used to emulate the user's design. A controller 102 is coupled to the programmable ICs 100 and controls diagnostics, for example upon power up. Any programmable ICs or other resources on the PCB 24 that are malfunctioning are desirably stored locally on the emulator, such as stored in a non-volatile fault memory 104. The non-volatile memory may be flash memory or any other type of non-volatile memory. The faults of each PCB are therefore stored locally in the memory 104. The diagnostics may also be controlled by communication between the maintenance server 66 and the controller 102.
FIG. 6 is a flowchart of an embodiment of a method for compiling the design to be emulated using stored emulator faults, such as emulator faults stored locally, on each PCB. In process block 120, faults can be stored locally on the PCBs within the emulator. The faults indicate components (e.g., ICs) that are not functioning on that particular PCB. In process block 122, the user's design is compiled for emulation taking the faults into account during the compilation. Thus, the compiler uses only available, non-faulty resources on each PCB in order to compile the design. If IC0 on PCB0 is not functioning as indicated in the stored faults, then the compiler will compile the design and not use this particular IC. Desirably, ICs that are functioning and not listed in the faults list are used. In order to determine which components on the PCB are faulty, the faults stored in memory 104 can be read. In process block 124, the design can then be downloaded (i.e., programmed) into the emulator. However, an additional check can be performed. In particular, a check can be performed to ensure that the design being loaded does not use logic included in the fault list. To accomplish this, the resource server checks the faults stored in memory 104 before downloading the design. If the compiled design includes faults, an error is indicated on the workstation 14. On the other hand, if the compiled design is using only available, fault-free resources, then the design is programmed into the PCB for emulation.
FIG. 7 is a flowchart of an embodiment of a method for using a fault avoidance type in the design compilation process. In process block 150, the user inputs into one of the workstations 14 the desired type of fault avoidance. By allowing the user to choose the fault avoidance type, the user maximizes flexibility by allowing the design to be downloaded into different printed circuit boards in the emulator. For example, the user can choose to compile the design such that a fault on one PCB does not affect available resources on other PCBs. For example, failure of IC0 on PCB0 can result in bypassing the use of IC0 on PCB0 without limiting the use of non-faulty ICs on other PCBs (e.g., IC0 on PCB1 is still used if not faulty.) Alternatively, the user can choose that faults on one PCB affect the use of resources on other PCBs. For example, a failure of IC0 on PCB0 can result in not using IC0 on PCB1 and IC0 on the other PCBs. In other words, the compiler will not consider IC0 on PCB0 or IC0 on PCB1 to be available resources, despite that IC0 on PCB1 is fully functioning. This allows the user to ensure that the design can be compiled and run on different PCBs in the system because faults are taken into account across PCBs. The user can also choose which boards or how many. For example, the user can choose that all PCBs are affected by a fault on any PCB. Alternatively, the user can choose that a smaller group of PCBs are affected by a fault on any PCB in that group (e.g., any board in a quad affects the same numbered board in another quad). In any event, faults on one PCB affect what are considered available resources on other PCBs in the emulator. In process block 152, based on the type of fault avoidance, a determination is made by the compiler as to which components in the emulator are available. Thus, the compiler considers the type of fault avoidance and if a fault occurs on an IC on one PCB, the compiler should determine the other PCBs in the system are affected by that fault, as already described above. In process block 154, the design is compiled using only the available components in the emulator as determined in process block 152.
FIG. 8 is a flowchart of an embodiment of a method providing further details of process block 152 in FIG. 7. In process block 200, PCBs that are affected by the selected type of fault avoidance are identified. For example, if PCB1 has a fault and the fault avoidance type is quad, the similarly positioned board in each quad is also affected by the fault. In process block 202, the faults are effectively combined across each identified PCB. To combine the faults, the malfunctioning IC number can be used and applied to the same IC number on the other identified PCBs. Thus, other PCBs can be considered to have a fault at the same IC number as the faulty IC. In process block 204, the combined faults are stored to be used by the compiler in compiling the design. This ensures that the design can be emulated on different, but logically equivalent, PCBs within the emulator. If multiple emulators are used, faults may also be combined across emulators.
FIG. 9 is a flowchart of an exemplary method for performing additional checks to ensure a compiled design is properly loaded into the emulator. In process block 250, faults are identified by running diagnostics on each PCB locally using the controller 102 in a well-known manner. The faults can be then stored on the emulator PCB itself, locally in memory. Additionally, it is desirable to store the faults in a faults database 72 so that if the emulator is powered down, the compiler can still access the fault information of all the PCBs. To accomplish this, the resource server 68 can read the locally stored faults and passes such faults to the faults database 72 through the virtual maintenance panel 67. In process block 252, a check can be made by the virtual maintenance panel 67 through the resource server 68 to see if the faults database 72 and the locally stored faults are consistent. If not, an error indication can be given to the user on workstation 14. The user can then instruct the resource server 68 to update the faults database either immediately or periodically. In process block 254, a check can be made by the resource server 68 if the compiled design being downloaded is using faulty components. Such a check can be accomplished by comparing ICs considered defective as stored locally in the memory 104 to ICs used in the compiled design. If the compiled design is using faulty components, an error is indicated to the user. In process block 256, the design is loaded into the emulator. Another check may also be performed if desired. Specifically, the resource server can check a board's serial number to ensure that boards have not swapped locations. If the faults database 72 stores information about an emulator board in a particular slot, checking the board's serial number ensures that the faults database properly informed the compiler 74 of faults on that board.
FIG. 10 is a flowchart of an embodiment of a method for storing fault information on the emulator boards. In process block 270, fault information is received through diagnostics on power-up and/or user input. For example, the user can input fault information that a known fault exists on a particular board through the virtual maintenance panel 67. The fault information is then stored in non-volatile memory on the emulator board associated with the fault information. In process block 272, after powering on the emulator, the virtual maintenance panel 67 receives fault information from the emulator boards. In process block 274, the virtual maintenance panel builds a database including the fault information. In addition, or alternatively, the virtual maintenance panel can generate a set of fault files used when compiling a design and used by the resource server prior to loading the design for a compatibility check.
In view of the many possible embodiments to which the principles of the disclosed invention may be applied, it should be recognized that the illustrated embodiments are only desirable examples of the invention and should not be taken as limiting the scope of the invention. Rather, the scope of the invention is defined by the following claims. We therefore claim as my invention all that comes within the scope of these claims.

## Claims

1. A method of loading a design into a hardware emulator, comprising:
storing faults, associated with components on an emulator board, in memory on the emulator board;
compiling the design to be emulated by avoiding use of components with the faults; and
loading the compiled design into the emulator.

2. The method of claim 1, further including storing the faults external to the emulator board and wherein compiling further includes reading the faults stored external to the emulator board or the faults stored in the memory on the emulator board.

3. The method of claim 2, further including comparing the faults stored external to the emulator board to the faults in the memory and issuing an indication if they are different.

4. The method of claim 1, wherein the memory is non-volatile memory and wherein storing faults on the emulator board includes running diagnostics on the emulator board to determine components with faults and wherein storing includes storing results of the diagnostics in the non-volatile memory on the emulator board.

5. The method of claim 1, further including reading a fault avoidance type and wherein compiling includes determining components that are available for compilation using the faults and the fault avoidance type.

6. The method of claim 1, further including combining faults across multiple emulator boards and compiling the design based on the combined faults.

7. The method of claim 1, further including combining faults across multiple emulators and compiling the design based on the combined faults so that the design can run on any one of the emulators.

8. The method of claim 1, wherein storing faults on the emulator board includes running diagnostics on the emulator board and storing results of the diagnostics in a non-volatile memory on the emulator board and also storing the results in a database located externally of the emulator board.

9. The method of claim 8, further including comparing the faults stored in the database to the faults stored in the non-volatile memory and issuing an error indication if they are different.

10. The method of claim 1, further including storing a board serial number on the board and checking the board serial number prior to downloading the compiled design to the board.

11. An emulator for emulating a user design, comprising:
multiple printed circuit boards having programmable logic thereon for emulating the user design in hardware;
at least one non-volatile memory located on one of the printed circuit boards in the emulator for storing faults locally on the circuit board;
wherein the faults indicate malfunctioning components on the circuit board.

12. The emulator of claim 11, further including a resource server coupled to the multiple circuit boards and a database, the resource server coupled to retrieve faults from the different circuit boards and store the faults in the database.

13. The emulator of claim 11, further including a compiler that uses the stored faults in order to compile the design.

14. The emulator of claim 11, wherein the non-volatile memory is flash memory.

15. The emulator of claim 11, further including a resource server coupled to the multiple circuit boards, the resource server checking a serial number of a printed circuit board prior to downloading a design to the circuit board to ensure that the design is loaded on the correct circuit board.

16. An emulator comprising:
means for storing faults associated with components on an emulator board in the emulator; and
means for compiling the design to be emulated by reading the stored faults and avoiding the use of components with the faults.

17. The emulator of claim 16, further including means for running diagnostics on the emulator board and storing the results in a non-volatile memory on the emulator board.

18. The emulator of claim 16, further including means for combining faults across multiple emulator boards and compiling the design based on the combined faults so that the design can run on any of the boards.

19. A method of loading a design into a hardware emulator, comprising:
storing faults, associated with components on at least a first emulator board and a second emulator board, in memory in the hardware emulator;
combining faults across the first and second emulator boards; and compiling the design using the combined faults.

20. The method of claim 19, before loading the design on a printed circuit board in the emulator, checking faults stored on the emulator board to ensure that the compiled design is not using faulty components.

21. A method of loading a design into a hardware emulator, comprising:
storing fault information, associated with components on an emulator board, in memory on the emulator board;
receiving fault information from the emulator board; and
building a database including the fault information.

22. The method of claim 21, wherein the storing fault information includes receiving the fault information through diagnostics and/or user input.