Multi-frequency debug network for a multiprocessor array

A debug network on a multiprocessor array having multiple clock domains includes a backbone communication channel which communicates with information nodes on the channel. The information nodes store and access information about an attached processor. The nodes are also coupled to registers within the attached processor, which operate at the speed of the processor. A master controller solicits information from the information nodes by sending messages along the backbone. If a message requires interaction with a processor register, the node performs the action by synchronizing to the local processor clock.

TECHNICAL FIELD

This disclosure relates to a system debugger, and, more particularly, to a system debugger structured to operate on a multiprocessor platform in which individual processors run at different frequencies.

BACKGROUND

Debugging software that executes on hardware systems is the process of recognizing, identifying and fixing or isolating software and/or hardware errors. An error occurs when an actual result does not match an expected result, and can be caused by errors in the software and/or hardware.

Developing software applications for a new computer processor typically uses a software simulation of the new processor design where software being developed is run on the software model of the chip being developed.

Debugging mixed software/hardware systems is easier if either the software or the hardware has been verified as accurate in another system. Debugging mixed software/hardware systems where neither has been verified to be accurate is difficult, and this difficulty scales as the number of interrelated processes increases.

The difficulty of debugging a software/hardware system that is based on an architecture of dozens or hundreds of individual processors does not scale linearly from experience in single processor systems. Not only must the operation of each processor be verified, but communication paths and buffers between the processors must be exposed for analysis. No tool currently exists that provides the type of exposure, depth, and flexibility necessary to adequately debug such multi-processor systems.

Embodiments of the invention address these and other limitations in the prior art.

DETAILED DESCRIPTION

FIG. 1illustrates an example tessellated multi-element processor platform100according to embodiments of the invention. Central to the processor platform100is a core112of multiple tiles120that are arranged and placed according to available space and size of the core112. The tiles120are interconnected by communication data lines122that can include protocol registers as described below.

Additionally, the platform100includes Input/Output (I/O) blocks114placed around the periphery of the platform100. The I/O114blocks are coupled to some of the tiles120and provide communication paths between the tiles120and elements outside of the platform100. Although the I/O blocks114are illustrated as being around the periphery of the platform100, in practice the blocks114may be placed anywhere within the platform100. Standard communication protocols, such as Peripheral Component Interface Express (PCIe), Dynamic Data Rate Two Synchronous Dynamic Random Access Memory interface (DDR2), or simple hardwired input/output wires, for instance, could be connected to the platform100by including particularized I/O blocks114structured to perform the particular protocols required to connect to other devices.

The number and placement of tiles120may be dictated by the size and shape of the core112, as well as external factors, such as cost. Although only sixteen tiles120are illustrated inFIG. 1, the actual number of tiles placed within the platform100may change depending on multiple factors. For instance, as process technologies scale smaller, more tiles120may fit within the core112. In some instances, the number of tiles120may be purposely be kept small to reduce the overall cost of the platform100, or to scale the computing power of the platform100to desired applications. In addition, although the tiles120are illustrated as being equal in number in the horizontal and vertical directions, yielding a square platform100, there may be more tiles in one direction than another, and may be shaped to accommodate additional, non tiled elements. Thus, platforms100with any number of tiles120, even one, in any geometrical configuration are specifically contemplated. Further, although only one type of tile120is illustrated inFIG. 1, different types and numbers of tiles may be integrated within a single processor platform100.

Tiles120may be homogeneous or heterogeneous. In some instances the tiles120may include different components. They may be identical copies of one another or they may include the same components packed differently.

FIG. 2Aillustrates components of example tiles210of the platform100illustrated inFIG. 1. In this figure, four tiles210are illustrated. The components illustrated inFIG. 2Acould also be thought of as one, two, four, or eight tiles120, each having a different number of processor-memory pairs. For the remainder of this document, however, a tile will be referred to as illustrated by the delineation inFIG. 2A, having two processor-memory pairs. In the system described, there are two types of tiles illustrated, one with processors in the upper-left and lower-right corners, and another with processors in the upper-right and lower-left corners. Other embodiments can include different component types, as well as different number of components. Additionally, as described below, there is no requirement that the number of processors equal the number of memory units in each tile210.

InFIG. 2A, an example tile210includes processor or “compute” units230and “memory” units240. The compute units230include mostly computing resources, while the memory units240include mostly memory resources. There may be, however, some memory components within the compute unit230and some computing components within the memory unit240. In this configuration, each compute unit230is directly attached to one memory unit240, although it is possible for any compute unit to communicate with any memory unit within the platform100(FIG. 1).

Data communication lines222connect units230,240to each other as well as to units in other tiles. Detailed description of components with the compute units230and memory units240begins withFIG. 4below.

FIG. 2Billustrates an example tile260, which could be the same or similar to one of the tiles210ofFIG. 2A. The tile260includes two interconnected pairs of processor/memory groups. Specifically, one pair includes a processor group272coupled to a memory group274through a set of communication channels262. The channels262inFIG. 2Aare functional illustrations and there may be any number of physical or logical communication channels between the processor group272and the memory group274. A second pair includes a processor group282coupled to a memory group284.

The first pair,272,274operates in a first clock domain, domain A, while the second pair282,284operates in a second clock domain, domain B. Operating in different clock domains means that a local clock may have a different operating frequency than another domain's local clock. For instance, a system clock290may be modified by a clock rate controller,292,294, to create a different local clock for the clock domains A and clock domain B. For example, a system clock may operate at 600 MHz, while the clock rate controller292generates a local clock of 300 MHz for the clock domain A, and the clock rate controller294generates a local clock of 123 MHz for clock domain B.

In a multiprocessing system, individual processor groups or even individual processors may operate at different clock rates. Typically the clock rate is based on the function of the processors, how quickly it can be performed, and how the performance time relates to the requirements of passing data to other processors. In this embodiment, the local clock rate is controlled by a value set in a particular register. Thus the register can be set once at initialization: for instance, if a first set of processors is performing a complex filtering procedure, the processors may need to run at a high clock rate; but processors in a second group that manage the filtering procedure may only require a much lower rate. The register may also be written during operation of a processor, allowing the code to run at different speeds; for instance a tight loop may need to run very fast, but error handling code may only require a much lower rate. Lowering the operating frequency of any circuitry, even for a limited period of time, has many benefits, such as reducing dissipated power and reducing the need for external buffers to store data that has been produced that cannot yet be consumed.

As described above, communication lines262couple a processor272,282to its associated memory block274,284. Because the components coupled to the lines262synchronously using the same clock, no special clock considerations are necessary. Communication lines264are coupled between processors272,282that may operate in different clock groups, and therefore may require special logic to ensure data is neither lost nor duplicated when crossing into a different clock rate. Clock crossing logic is covered in detail below.

FIG. 3is a block diagram illustrating a data/protocol register300, the function and operation of which is described in U.S. application Ser. No. 10/871,347, referred to above. The register300includes a set of storage elements between an input interface and an output interface.

The input interface uses an accept/valid data pair to control the flow of data. If the valid and accept signals are both asserted, the register300moves data stored in sections302and308to the output datapath, and new data is stored in302,308. Further, if out_valid is de-asserted, the register300continues to accept new data while overwriting the invalid data in302,308. This push-pull protocol register300is locally self-synchronizing in that it only moves data if the data is valid and the subsequent register is ready to accept it. Likewise, if the protocol register300is not ready to take data, it de-asserts the in_accept signal, which informs the previous stages that the register300cannot take the next data value.

In some embodiments, the packet_id value stored in the section308is a single bit and operates to indicate that the data stored in the section302is in a particular packet, group or word of data. In a particular embodiment, a LOW value of the packet_id indicates that it is the last word in a message packet. All other words in the packet would have a HIGH value for packet_id. Thus the first word in a message packet can be determined by detecting a HIGH packet_id value that immediately follows a LOW value for the word that precedes the current word. Alternatively stated, the first HIGH value for the packet_id that follows a LOW value for a preceding packet_id indicates the first word in a message packet.

The width of the data storage section302can vary based on implementation requirements. Typical widths would include powers of two such as 4, 8, 16, and 32 bits.

With reference toFIG. 2A, the data communication lines222could include a register300at each end of each of the communication lines. Because of the local self-synchronizing nature of register300, additional registers300could be inserted anywhere along the communication lines without changing the operation of the communication.

FIG. 4illustrates a set of example elements forming an illustrative compute unit400which could be the same or similar to the compute230ofFIG. 2A. In this example, there are two minor processors432and two major processors434. The major processors434have a richer instruction set and include more local storage than the minor processors432, and are structured to perform mathematically intensive computations. The minor processors432are more simple compute units than the major processors434, and are structured to prepare instructions and data so that the major processors can operate efficiently and expediently.

In detail, each of the processors432,434may include an execution unit, an Arithmetic Logic Unit (ALU), a set of Input/Output circuitry, and a set of registers. In an example embodiment, the registers of the minor processors432may total64words of instruction memory while the major processors include256words, for instance. Additionally, a debug unit (DB) may be instanced in each of the processors432,434.

Communication channels436may be the same or similar to the data communication lines222ofFIG. 2A, which may include the data registers300ofFIG. 3.

In embodiments of the invention, the processors432,434may operate at any clock rate, even rates different from those processors adjacent to one another. Communication channels that cross different clock domains require special clock crossing circuits to ensure that data is neither lost nor duplicated. A specific example of a clock crossing circuit is described in detail below.

FIG. 5illustrates an example processor500that could be an implementation of either the minor processor432or major processor434ofFIG. 4.

Major components of the example processor500include input channels502,522,523, output channels520,540, an ALU530, registers532, internal RAM514, and an instruction decoder510. The ALU530contains functions such as an adder, logical functions, and a multiplexer. The RAM514is a local memory that can contain any mixture of instructions and data. Instructions may be 16 or 32 bits wide, for instance.

The processor500has two execution modes: Execute-From-Channel (channel execution) and Execute-From-Memory (memory execution), as described in the '036 application referred to above.

In memory execution mode, the processor500fetches instructions from the RAM514, decodes them in the decoder510, and executes them in a conventional manner by the ALU530or other hardware in the processor500. In channel execution mode, the processor500operates on instructions sent to the processor500over an input channel502. A selector512determines the source of the instructions for the processor500under control of a mode register513. A map register506allows any physically connected channel to be used as the input channel502. By using a logical name for the channel502stored in the map register506, the same code can be used independent of the physical connections.

Numeric operations are performed in the ALU530, and can be stored in any of a set of registers532. One or two operands may be sent to the ALU530from the selectors564and566. Specialized registers include a self-incrementing register534, which is useful for counting, and a previous register526, which holds the output from the previous ALU530computation.

Input channels522,523supply data and/or instructions for the processor500.

To ease the difficulty of design, registers or other components that are clocked within the processor500typically use the same local clock signal and therefore run synchronously. The local clock is known as the “processor clock”.

A debug slave570is an independently operating unit that may be included in each processor and memory of the entire system100, including the core112and I/Os114(FIG. 1). Including an interactive debug network on the system100allows software to thoroughly examine and test the hardware as it runs. The debug slave570is unique in that it performs most operations at the speed of the debug network and only some operations at the local speed of its processor when required to do so, such as stepping an instruction and having to wait for a result value. Detailed description of the debug slave570and how it relates to an entire debug system follows.

FIG. 6illustrates a debug network600according to embodiments of the invention. The debug network includes a debug system controller610, which directly controls a debug master controller612and communicates with a debug slave controller614.

A debug datapath622connects through a series of slaves620in a ring. In one embodiment, the datapath622is formed of data/protocol registers300illustrated inFIG. 3. In the embodiment described with reference toFIG. 6, the width of the data register302can be a single data bit. Additionally, the valid, accept, and packet-id registers304,306,308, can also be a single bit wide. Thus, the datapath622can be four bits wide. The master controller612and slave614control the valid and accept signals for the datapath622as described with reference toFIG. 3above. By controlling these valid and accept signals, data is sent around the datapath622in a predetermined manner. In an example embodiment, the entire datapath622is formed of a single shift register, where the output of one slave620is bitwise shifted to the next location in the datapath. Each slave620is identified with a particular bit location in the shift register. By stepping the exact number of times as there are slaves620on the network600, a data bit makes a complete circuit around the datapath622. In the illustrated embodiment, the master controller612generates data to be placed on the datapath622and the slave controller614removes data from the datapath after it has completed the entire datapath622ring. In other embodiments, the master controller612performs both functions of placing data on the datapath622ring as well as removing data from the ring. In embodiments of the invention, the datapath622is a standalone datapath that no other object within the system100uses for any other purpose.

The slaves620may be resident in the processors, as illustrated above, or may otherwise may be connected to retrieve data from and write data to the processors. In an example embodiment, there is only one debug master controller612active for each datapath622, although more than one master could be operating if desired. For example, one master controller612could be monitoring a particular aspect of all the slaves620, while another master controller could be controlling the ring. In other embodiments, each system100could include several or even many separate datapaths622, each under the control of at least one master controller612. Further, in some embodiments, the datapath622may include dedicated portions of data communication lines222ofFIG. 2. In addition, to being resident in processors, the slaves620can be resident in any piece of state logic, such as a memory controller. For instance, with reference toFIG. 2, several slaves620(not shown) may be coupled to or resident in processors in the processor group230, while one or more slaves may also be coupled to or resident in memory controllers in the memory group240.

Still referring toFIG. 6, a debugger640can reside separate from the core112(FIG. 1), and can connect to it through a host to chip communication interface630, which may be embodied by one of the I/Os114ofFIG. 1. For instance, the communication interface could be embodied by a Joint Test Action Group standard, JTAG), also known as IEEE 1149.1, PCI, general I/O, or other acceptable interface as is known in the art. The debugger640could be a hardware and/or software process running on conventional hardware.

The debugging network600ofFIG. 6is straightforward to implement on a multiprocessing platform, such as the platform100ofFIG. 1. In this embodiment, one or more processors in the core112, such as a major or minor processor434,432ofFIG. 4could operate as the system controller610, the master controller612, and the slave controller614. In other embodiments, these duties could be shared across one or more processors in the platform100. In some embodiments, each processor432,434(FIG. 4) includes hardware to be the system controller670and the master controller672, as well as a slave676, and any of the processors could be selectively driven to perform any or all of the functions. Because this would be expensive to implement in hardware, other embodiments include specialized hardware in only some of the processors, such as one or two processors in every tile210(FIG. 2). Partitioning the components of the debug network600is left to the implementation engineer.

One way to implement the debug system controller610is to use an operating kernel that accepts commands from the off-chip debugger640or on-chip debugger650. The commands are translated into one or more debug packets according to a predetermined protocol used by the master controller612, slaves620, and the slave controller614. The system controller610generates the debug packets and the master controller612, and places them on the datapath622. After one of the slaves620responds to the request from the debug packet, the slave controller614(or master controller612) removes the packet from the datapath622and transfers it to the system controller610for further analysis and operation. In the event that no slave620responds, e.g., the packet comes back unchanged, the system controller610can determine that no slave620had a valid response.

In another embodiment, the debugger640,650itself generates and interprets the debug packets, which would make the system controller610easier to implement, at the expense of a more complicated debugger.

FIG. 7is a functional block diagram of a slave700, which is an example embodiment of the slave620ofFIG. 6or slave570ofFIG. 5. Of course, other embodiments are possible.

The slave700couples to the debug network622(FIG. 6), referred to here as the debug channel. The slave700is divided into two clock domains: a debug clock domain730and a processor clock domain720. In the debug domain730, all read and write actions occur synchronously to the debug system clock. In particular, local state such as the address register712or other state information714, such as various processor states, and the import/export to the debug channel all occur using the debug clock. Similarly, all reads and writes within the processor logic occur synchronously in the processor clock domain720. In particular, the processor (not shown inFIG. 7) can read and write all the register sets722,728,724,740,742and744synchronously without hazard.

The processor register sets,722,728,724,740,742and744are also accessed by the debug controller710. An instruction register728, a previous result register724, and a control register set722have both read and write access by the debug controller710, and so are accessed through bidirectional clock-crossing channels. A watchdog register740, a status register set742, and an exception register set744only has read access to the debug controller, and so registers740,742and744are simply sampled asynchronously.

Typically, the debug channel has a much simpler functionality than a processor, and so the logic will operate at a faster clock rate than the processor clock. Thus splitting the clock domains720and730as shown inFIG. 7allows debug operations in domain730that do not have to cross into the processor clock domain720to execute faster. In particular, debug commands not used by the particular slave controller will be quickly passed on without affecting the execution of the associated processor.

An address unique to each slave700is stored in an address register712. A channel controller710accepts debug packets (described below) from the debug channel, determines the requested information or action, performs the request (or denies the request based on a state of the processor), and places packets back on the channel.

An instruction register728may be the same as the instruction register511ofFIG. 5or a slaved copy. If requested to do so, the controller710can load an instruction into the instruction register728, or read an instruction from the instruction register728. This read or write must be synchronous to the processor clock domain720, and thus occurs at the clock rate of the attached processor. In one embodiment the clock-crossing channel is 1-bit wide to reduce hardware, and so loading and unloading is performed by a serial bit shift register.

Similar to the instruction register728, the previous result register may be the same as the previous register526ofFIG. 5. Similarly, any reads or writes by the controller710to the previous result register724also must happen at the clock rate of the attached processor.

The single-bit operation of loading or unloading the instruction register728and previous result register724matches the bit-wise operation of the datapath622ofFIG. 6, so that the debug network and slaves are always synchronized.

The slave700also includes specialized data storage, which is used to control or read relevant data from its host processor. A watchdog bit740can be written by the host processor when instructed to do so. The watchdog bit740is initialized to zero on startup of the host processor. Executing a watchdog command in the processor writes a 1 in the watchdog bit740. The debug network can then query the watchdog bit740and report it over the debug channel to the debug system controller610. If the watchdog bit740contains a 1, the debug master determines that the host processor is operational, or at least has executed the watchdog command since startup or the last time it was reset. The watchdog bit740can also be cleared by the debug system controller610by sending an appropriate debug message to the particular slave700, as described below.

A set of control data is stored in a control register722used by the slave700to control its host processor. Similar to an instruction register728and a previous result register724, a control register722is accessed via a clock-crossing channel and so reads and writes are accessed at the slowest speed of the two clock domains720and730. For instance, a “keep” command is effected by storing a “1” in the K register of the control register722. Other commands include “step” (S), “execute” (E), and “abort” (A). These commands and their operation are described below.

A set of status information in a status register742provides status information about particular data and control signals on the host processor. For example, status information can include whether particular flags are asserted, if a conditional branch is present, whether any of the input or output channels of the processor have stalled, whether there is an instruction in the instruction register728or if the execution of the processor is blocked. Additional status information can include the mode the processor is operating in, such as memory execution or channel execution mode. A copy of the program counter (such as the program counter508inFIG. 5) may also kept in the status register728.

Exception information is stored in an exception register744. Exceptions occur when particular instructions or behavior is executed on the host processor. For example, when a trap instruction is executed by the host processor, relevant trap data is stored in a trap ID section of the exception register744. Channel identification and exception identification can also be stored upon similar commands. Description of commands to store and use such data follows.

All information that is stored or retrieved from the status register742, the watchdog bit740, and the exception register744can be accessed and updated without crossing the processor clock domain720. Thus, reading this data occurs at the full rate of the clock signal of the debug clock domain730.

FIG. 8is a flow diagram illustrating an example operation flow800of the slave700according to embodiments of the invention. Debug message packets can be addressed to specific slaves700, or, in some cases, sent out generally for any slave to answer. The first bit of the debug message packet indicates whether the message is destined for a particular processor or is sent out to all slaves. Each slave700matches its own address to the data stored in its address register712. In the embodiment described, each slave700need not know the address of any other slave.

In a process810, the first slave700downstream of the master controller612of the debug system600inspects the global bit of the current debug packet. If the global bit is set and the slave700has a response that can be given in response to the global request, the process800exits the query814in the YES direction. Then, a process820de-asserts the global bit820and overwrites the address portion of the debug packet with its own address, so that no subsequent slave700can respond. Next, in a process824, the slave700modifies the debug packet with the response data. The modification can be simply changing bits in the existing debug packet, or can involve appending data generated by the slave700to the end of the original debug packet as its response. Some processes of the slave700can be performed within the debug clock domain730, while others must cross into the processor clock domain720. After the modification to the current debug packet has been made, the process800transmits the debug packet to the next stage out on the debug channel.

If the global bit of the current debug packet is not set (or the slave700has no response to give to a global inquiry), the slave700reads the debug packet destination address in a process830. If the current debug packet is not addressed to the particular slave700in inquiry834, or if the slave does not have a response to the debug packet in inquiry844, the slave simply sends the debug packet, with no modification, out onto the debug channel to the next slave700.

If instead the current debug packet matches the local slave address712and the slave has a response in the inquiry844, the flow800proceeds back to the process824to modify the debug packet with the appropriate response.

Once the slave700has completed the debug packet in the process, the flow800returns to the process810and the slave700waits to receive the next debug packet.

FIG. 9illustrates how the debug slave700can dynamically change the operation of its host processor while the system100(FIG. 1) is in operation.FIG. 9illustrates a typical operating flow in processors, such as the processor500ofFIG. 5. The operating flow of a processor900is divided into three main stages, a fetch stage910, a decode stage940, and an execute stage970.

Between each stage is a set of data/protocol registers, such as the register300ofFIG. 3. Each register300is a master-slave register and at any instant in time holds two (possibly different) values. The data width of the registers300can depend on their application. With reference toFIG. 9, an instruction register can be 16 or 32 bits wide, as can a branch register. A flag register can be as wide as necessary depending on the number of flags used for the particular processor. Because each register comprises two values, separate labels for each side of each register are illustrated inFIG. 9. For instance, the instruction register has a side that is in the fetch stage910, referenced as register912, and a side that is in the decode stage940, referenced as register942. Similar references are made to the branch and flag registers to denote which side is being referred to. Likewise, a decoded instruction register is962/972, and two operand registers964/974and966/976are illustrated.

Feeding the instruction register912is a selector930, which determines whether the processor is in memory execution mode or channel execution mode, as described above. The selector930receives its channel input from an input channel902and its memory input from RAM924. Another selector922feeds the RAM924with the normal incrementing program counter920or one from a value generated by a branch decoder952in the decode branch940. Also within the decode branch is a, decoder950, which may be identical to the decoder510ofFIG. 5. In the execute stage970, an ALU980receives instructions from the instruction register972and is connected to two operand registers974,976. The output of the ALU is fed to an output register984, which further feeds the output channel904.

In operation, the flow illustrated inFIG. 9begins at the fetch stage, where an instruction is sent to the instruction register912, either from the input channel902or from the RAM924. Because the instruction register comprises two values, a first instruction is propagated from the instruction register912to the instruction register942when there is a valid instruction in the register912and the register942is accepting. Thus the instruction register912/942can be holding 0, 1, or 2 instructions, Further, if the instruction register912/942holds a single instruction, the instruction can be stored in either the instruction register912or942. Precise control of the location of instructions in the instruction register912/942gives the debug network600the power to easily control the processor900.

Such precise control could also be exercised on the border between the decode stage940and execute stage970, but in this embodiment such fine control is typically unnecessary for operation of the debug network600.

The debug network600can change the operation of the processor900under its control by extracting instructions from the instruction register942and writing new instructions into register942. Recall in the description with reference toFIG. 7, that the slave700can remove instructions from, or can insert instructions into the instruction register728one bit at a time, at the speed of the slowest clock domain720or730. The same is true for the instruction register942ofFIG. 9. Similarly, the slave700can extract from and load to the previous result register724. Although no analogue to the previous result register724is illustrated inFIG. 9, it would be located in the execute stage970. Interaction with the previous result register would also be governed by which clock domain720or730is slowest.

If such an extracted instruction is stored where it can be accessed by the debug network600, such as in the debug system controller610(FIG. 6), the debug network could re-insert the extracted instruction back into the instruction register942when it concluded its operations. Thus, the debug network600is able to stop a processor from executing, store the processor's current state, execute its own instructions for testing/verifying/debugging and then, when the debug network has finished, replace the processor to its original state and re-start the processor.

Operation of the debug network620will now be described with reference toFIGS. 5-11.

A master controller612generates debug packets and places them on the debug datapath622. The debug packets could be any length, but is convenient to make them equal the lowest common multiple of instruction width, 16-bits in this embodiment.

A debug packet is delimited by the packet_id ofFIG. 3. In one embodiment, the packet_id is set to “1” for each bit in the debug packet except for the last bit. When a receiver, such as the controller710ofFIG. 7, detects the packet_id changing from “1” to “0”, it knows it has received the last bit in the debug packet. Because debug packets are always received and sent on (sometimes with modifications), debug packets cannot ever become blocked because slaves are waiting. Keeping the debug channel free from blockage ensures that all packets can get access to the destination slave620.

The debug packet includes a header, which identifies the packet as a global packet (which any slave620can answer) or includes a destination address for a particular slave. Other fields in the packet include a command field and an indication of how detailed of a response it is requesting. For example, the debug packet may instruct that the slave620simply acknowledge the receipt of the command. Alternatively, the slave620may be requested to append a result, copy status bits, or include other information. Additionally, the debug packet may include data, for example values to be loaded into specific registers of the processor. In most cases the packet requests data about the host processor of the slave620, such as operating state, or the packet simply requests that the slave620acknowledge that it has received the command. In some embodiments, the global packet is limited only to particular debug commands. As described with reference toFIG. 7, if the packet only requests state that is stored in the debug clock domain730or can be asynchronously read (register sets740,742and744), the answer can be generated and returned at the speed of the debug network. If instead the packet requests interaction with the processor, through an instruction register728, a previous result register724, or direct control using a control register722, the slave620interacts with those registers at the slowest clock rate of the two domains720and720, typically the processor clock.

All debug packets are returned by the slave620over the debug datapath622to the slave controller614for transfer to the debug system controller610. In some embodiments, a slave cannot create a packet and can only modify the received packet. The slave620can append data by simply changing the packet_id of the former last bit of the current debug packet to “1,” appending the data from the slave, and then inserting a “0” as the packet_id of the new last bit. When the slave controller614receives the new packet, it continues to process until it recognizes the 0 as the packet_id, thus operating on the whole length of the new packet.

Commands are broadly split into two groups: those that are guaranteed to produce a result (so long as the debug network600is operational), and those with contingent success. Those commands that are guaranteed to respond are those that can be completed within the debug clock domain730. The commands that are not guaranteed to complete are those that cause the slave to interact with the attached processor, having to cross into the processor clock domain720, and are thus dependent on a state of the processor. The guaranteed success actions include “watchdog,” “slot”, and “set-state.”

The watchdog command from the debug network600is used in conjunction with a watchdog instruction executed by the processor. At any time a processor can execute a watchdog instruction, which sets to “1” the watchdog bit in the watchdog register740of its attached slave700(FIG. 7). At any time the debug controller system610(FIG. 6) can send a watchdog command to a specific processor by sending it in a debug packet. When responding to the command, a slave700reports the status of its watchdog bit in the watchdog register740, and resets the bit value in the watchdog register740to “0.”

The slot command from the debug network is used in conjunction with a trap instruction executed by the processor. The trap instruction stops the processor pipeline by not allowing the next instruction to execute. The processor on which the trap instruction just completed notifies its attached slave700that the trap has occurred, such as by sending an “except” signal. This causes the slave700to load a trap ID into the exception information register744(FIG. 7). Then, when the debug system controller610issues a slot command, which may be a globally issued or directed to a specific slave700, the slave appends the trap ID from register744in response to the request.

Channel exceptions follow the same pattern. A channel exception occurs when the processor decodes an instruction that is scheduled to receive data from, or output data to, a channel that is on an exception list stored by the processor. If such an exception occurs, similar to how traps are handled above, the processor notifies its slave700, which causes the slave to store (in its exception register744) the channel ID that caused the exception. If more than one channel could have caused the exception, only the highest priority channel ID is loaded into the exception register744. Also similar to the procedure above, when the debug system controller610issues the slot command, the slave700answers by appending the exception ID from register744to the requesting debug packet.

The “set-state” command is used to set or clear the state of the information in the control register722(FIG. 7). Recall that the control register722stores states for “keep,” “step,” “execute,” and “abort,” which control operation of the slave700, and thus the attached processor. The states may be set or cleared by sending appropriate debug commands, through the debug network600to the appropriate slave700. There are also debug commands that may not be guaranteed to complete successfully, which include “load-previous,” “extract,” “insert,” and “insert-execute.”

Because they depend on the state of the processor when the command is received and attempted, the commands below are not guaranteed to complete. With reference toFIG. 7, the load-previous command causes the slave700to load data from the debug channel into the previous result register724. Recall that in some embodiments loading the previous result register occurs one bit at a time, in the processor clock domain720. Similarly, the insert command loads data from the debug channel into the instruction register728, also in the processor clock domain720. The insert-execute command first loads data from the debug channel into the instruction register728, then causes the loaded instruction to execute. The extract command loads data from the instruction register728back onto the debug channel.

During operation, the processor can be in one of several states. For example, the processor can be running, or it can be blocked. A block occurs when the protocol signals prevent data from being fetched, decoded, or executed (FIG. 9). For instance, if an object connected to the output channel904is not receiving data, the ALU980cannot process further data. If the ALU980cannot process data, then the instructions and operands fill the registers962,972,964,974,966, and976. This, in turn, causes the decode stage940to stall, which backs up instructions in the instruction register912/942.

For effective debugging to occur, it is best to have the decode and execute stages940,970empty (or know they will be empty), and an instruction held in the instruction register942. This is denoted a “clean-halt” state, which means the processor900is ready to be controlled by the debug system600. The instruction in the register942can be extracted by using the extract command, as described above. Also as described above, at the conclusion of the debugging, the previously extracted instruction can be replaced using the insert command, which would place the processor in the original state.

With reference to the control register722ofFIG. 7, the Keep command halts the processor at the fetch stage and attempts to allow the processor to clear. The Step command allows one fetched instruction to decode and execute. The Execute command allows an inserted instruction to decode and execute, while the Abort input is used to nullify and remove results from the decode940and execute stages970, thereby clearing the processor for operation. In the debug system, using the Abort command is usually a non-preferred choice because it removes data from a processor in an attempt to gain control when the processor is fatally blocked. By combining these commands, a rich dialog can be built between the debug master controller612(FIG. 6), the debug slave620, and the processor attached to the slave.

In principle, the debug network600first uses Keep to stop the processor pipeline at the input of the decode stage940(FIG. 9), by de-asserting the valid signal to the instruction register942. Instructions in the execute stage970are not affected and are completed normally. Operation of the fetch stage910will eventually stop because of de-asserted accept signals flowing back from the instruction register942.

The debug network600can query the slave700to send the value of its status register742, which indicates whether there is an instruction waiting and/or the execution is blocked, as described above.

Once the pipeline is put into a clean-halt state, instructions may be single stepped by executing them from the instruction register942one at a time using the Step control. The slave700could insert its own instruction into the instruction register942, as described above, or can allow the instruction presently stopped in the register942to continue. If the slave700inserts its own instruction into the instruction register942, the instruction stored in the instruction register912remains undisturbed.

After executing the desired instruction, the debug network600could request that the slave700send a copy of its status registers742, which allows the debug system master610to determine how the processor is operating. Also, the debug network600could request that the slave700send the previous result register724. The system master610would need to recognize that the previous result register724is potentially invalid until the processor has completed a number of cycles because of the pipeline created by the execution logic.

The debug system master610can cause the processor to execute many different instructions by using the insert-execute command. When the system master610is ready to return the processor to the original instruction stream, it can put the saved instruction back into the instruction register728, then cause it to execute, returning the processor back to its original condition before debug started.

If instead the processor is not in the clean-halt state, any attempt at executing the commands insert, insert-execute, and extract will be unsuccessful. The slave700indicates this by modifying a bit in the debug packet containing the instruction before sending it along the debug network to the debug system master610.

FIG. 10is a block diagram that illustrates a debug network1000for a multiprocessor system that includes multiple clock domains according to embodiments of the invention. A backbone debug network1010operates at the system clock, which in this example can be 1 GHz. A debug domain1020is similar to the debug domain730ofFIG. 7, and includes a controller1012. The debug domain1020is coupled through a clock crossing network (not shown) to a processor domain1022, which is similar to the processor clock domain720of FIG.7. The processors may operate at various frequencies as illustrated, such as 123 MHz, 221 MHz, 400 MHz, and 1 GHz, etc. The particular clock frequencies driving the pairs120is unimportant, other than for illustration.

In operation, the backbone debug network1010transmits a debug message packet to the first local controller1012in the network1000at the debug clock frequency. The local controller then determines if it is addressed for itself, or has a global address set. If the message packet has a local destination, the local controller1012reads the debug message packet. If the debug message packet contains a command that can be answered by reporting or changing any of the data registers solely within the debug clock domain, such as those illustrated inFIG. 7, then the local controller1012immediately performs the requested operation and reports the action by modifying the debug packet, as described above. The local controller1012then sends the debug message packet back to the debug master along the backbone debug network1010.

If instead the debug message packet contains a command that requires a read or write in the processor domain1022, the local controller uses a clock crossing channel to access the processor domain registers. In this case, the entire debug network1010temporarily (for the time to process the packet) is throttled to operate at the (slowest of the processor or debug clock) speed. Because many of the commands or operations that the debug system controller610(FIG. 6) may request a particular processor can be answered by the local controller1012without ever having to read or write elements in the processor domain1022, the debug network1000simply runs at full network speed much of the time. In the case where the processor clock in the processor domain1022equals the debug clock in the debug domain1020, the debug network operates at full speed, regardless of whether the particular command specifies action to be performed in the processor domain1022.

If the debug message packet received by the local controller1012is not addressed locally, the controller1012passes the debug message packet to the next traffic controller1012on the backbone debug network1010.

In contrast to a debug network that is serially coupled through every slave unit in an entire network, a system such as that described with reference toFIG. 10only (potentially) slows to a local clock rate when debug message packets need to access registers that operate in the processor domain. Otherwise the message packets operate at the full debug clock rate.

FIG. 11illustrates a clock crossing circuit1100that can be used for the clock crossing networks when a debug message packet passes from the debug clock domain730to the processor clock domain720and back again. The clock crossing circuit1100includes three clock domains: an input clock domain1160, a clock crossing domain1180, and an output clock domain1190. Within each domain, components operate at the clock speed of the domain. With reference toFIG. 10, the input clock domain1160is circuitry included in the debug domain1020, while the output clock domain is circuitry included in the processor clock domain1022.

Each of the domains1160,1190may run from a master clock having the same frequency or different frequencies.

The local clock frequencies may be generated locally from a master clock, as described in the U.S. patent application Ser. No. 11/460,231, filed Jul. 26, 2006, entitled CLOCK GENERATION FOR MULTIPLE CLOCK DOMAINS, the teachings of which are incorporated by reference herein. As taught in the '231 application, the master clock for each domain can be made from a power-of-two divider, which means that the rising edge of any slower clock always aligns with a rising edge of faster clocks. Additionally, each of the domains1160,1190may mask particular clock cycles of its own master clock, using clock enable signals, i_ape and o_cpe to generate its own final frequency.

In operation, the clock crossing domain1180operates at a rate equal to or an integer multiple above the higher of the clock rate of the input clock domain1160and the output clock domain1190. In other words, whichever clock domain has the highest master clock frequency, the input clock domain1160or the output clock domain1190, the clock crossing domain1180runs at that clock frequency or an integer multiple above that clock frequency. Although the clock domain1160is referred to as an input domain, and the clock domain1190is described as an output domain, protocol information in the form of data actually flows in both directions, as illustrated inFIG. 3.

In the input clock domain1160, data is stored in flip-flops or registers1164and side registers1162. A selector1163, such as a multiplexer, controls the origination of the data stored in the register1164. Data operating as the packet_id is stored in the same manner and will not be differentiated with respect to the description of the clock crossing circuit1100. A similar configuration also stores an input valid signal, i_valid, in either register1168or side register1166, controlled by a selector1167. Output of an i_accept signal, which indicates that a successive stage is able to accept data, controls the selectors1163and1167. Additionally, an output of the side register1166, which indicates whether the data stored in the side registers1162is valid, is combined with an output of a register1170in a logic gate1174. Such a configuration allows the data in the side registers1162to be updated when the data is invalid, regardless of a state of an output from a register1170. A logic gate1172operates in the same way to allow data in the main registers1164and1168to be updated as well, based on a state of the output of the logic gate1172.

The output clock domain1190includes only a single additional gate when compared to a non clock-crossing system. A logic gate1192combines an accept signal with a clock pulse enable signal for the output clock domain, o_cpe. In operation, the o_cpe signal is combined with the master clock signal of the output clock domain1190to generate the actual clocking signal for the output clock domain1190. The output of the logic gate1192is sent to the clock crossing domain1180. The logic gate1192ensures that only one accept signal is ever generated within one tick of the master clock signal that is used to drive the output clock domain1190. This avoids multiple accept signals in a single output clock tick.

The clock crossing domain1180includes circuitry that ensures that data passes correctly from the input clock domain1160to the output clock domain1190, no matter what clock speed the domains1160,1180are operating, and no matter how many of the master clock signals are masked to generate the domains' final operating frequency. In this context, correctly passing data means that only one set of data passes from the input domain1160to the output domain1190for each data transfer cycle.

In a system where different domains may have different clock rates, a data transfer cycle is measured by the slowest master clock. Thus, a data transfer cycle means that only one set of data will pass from the input clock domain1160to the output clock domain1190per single cycle of the slowest clock, assuming that the protocol signals authorize this data transfer.

The circuitry in the clock crossing domain1180allows the data in the register1181to be set only once per data transfer cycle, and then prevents further data transfers in that cycle by negating the o_valid (forward protocol) signal. In particular, when the o_valid signal is negated, data transfer halts, as described above. The data in the register1181cannot be set again until after the rising edges of both of the slow and fast domains next occur at the same time. Note that the circuitry in the clock crossing domain1180operates correctly no matter which of the clock domains1160or1190is the fastest domain, and no matter which of the domains has the highest master clock frequency. When the clock domains1160and1190are clocked at the same frequency, the clock crossing domain1180has almost no affect on the clock crossing circuit1100. In particular, if both clocks of the input clock domain1160and output clock domain1190have the same frequency (the synchronous case), o_cpe=i_cpe=1, the logic gates1184and1192are always enabled, and therefore the clock rate of such a synchronous system would perform at full rate, as if the circuitry in the clock crossing domain1180didn't exist, other than a minimal logic gate delay.

From the foregoing it will be appreciated that, although specific embodiments of the invention have been described herein for purposes of illustration, vanous modifications may be made without deviating from the spirit and scope of the invention.

More specifically, the debug network could be implemented in a variety of ways. For example, a wider network could access data more quickly from the processors and would also allow instructions or data to be loaded with less delay. Multiple debug networks could be present on the platform. Instead of a ring, the network could be implemented by directly connecting channels to a central hub. Different data could be stored by the slave and requested by the debug master. A different command protocol could be used by the debug network to control the slaves and host processors. Accordingly, the invention is not limited except as by the appended claims.