Guaranteed edge synchronization for multiple clocks

A method and apparatus for guaranteeing clock edge synchronization is disclosed. In one embodiment, a system for synchronizing clock signals includes a clock unit and a synchronization unit. Both the clock unit and the synchronization unit may be configured to receive a reference clock signal. The clock unit may be configured to drive a plurality of domain clock signals to various clock domains. The synchronization unit may be configured to assert a synchronization pulse once every N reference clock cycles. Clock edges of the domain clock signals may be aligned with each other responsive to asserting the synchronization pulse.

BACKGROUND OF THE INVENTION

1. Field of the Invention

This invention relates to the clocking of electronic circuits, and more particularly, to the synchronization of multiple clock signals.

2. Description of the Related Art

In a typical computer system, one or more processors may communicate with input/output (I/O) devices over one or more buses. The I/O devices may be coupled to the processors through an I/O bridge which manages the transfer of information between a peripheral bus connected to the I/O devices and a shared bus connected to the processors. Additionally, the I/O bridge may manage the transfer of information between a system memory and the I/O devices or the system memory and the processors.

With multiple processors, multiple I/O devices, and multiple I/O bridges may come multiple clock domains. Often times, the clock frequencies required for operation of the processor, the I/O bridges, and the peripheral devices (and buses that couple these elements together) are different. Furthermore, it is often times possible the within a processor or an I/O bridge, multiple functional units may be present which require differing clock frequencies. In such cases, multiple clocks may be provided, with the clock signals being routed to their respective domains.

In systems having multiple clock domains, it is often necessary to transmit information from one domain to another. However, this is not a straightforward process when the clock frequencies from one domain to another are different. Often times a technique known as handshaking is used. Handshaking may involve a communication between two clock domains wherein an agreement is reached on when data may be sent. This may ensure that a faster domain does not overwhelm a slower domain by exceeding the maximum bandwidth of the slower domain. Similarly, handshaking may ensure that a slower device may utilize as much of the bandwidth of a faster domain as possible when the faster domain is receiving information.

Regardless of the technique used to transfer information between clock domains having different clock speeds, the relationship between the respective clock signals is an important consideration. In particular, it may be important to know the relationship between the rising or falling edges of the respective clock signals. Often times, it is difficult to ascertain this relationship. Thus, it may not always be possible to transmit information while utilizing the maximum available bandwidth on a link between two different clock domains.

SUMMARY OF THE INVENTION

A method and apparatus for guaranteeing clock edge synchronization is disclosed. In one embodiment, a system for synchronizing clock signals includes a clock unit and a synchronization unit. Both the clock unit and the synchronization unit may be configured to receive a reference clock signal. The clock unit may be configured to drive a plurality of domain clock signals to various clock domains. The synchronization unit may be configured to assert a synchronization pulse once every N reference clock cycles. Clock edges of the domain clock signals may be aligned with each other responsive to asserting the synchronization pulse.

In one embodiment, a system for synchronizing clock signals may include a plurality of phase locked loops (PLLs), although embodiments having delay locked loops (DLLs) are possible and contemplated). Each of the PLLs may include a divide-by-n counter. Each PLL may be configured to produce a domain clock signal that is greater or less than the frequency of reference clock signal. When the synchronization pulse is asserted, it may cause the divide-by-n counters in each of the PLLs to be reset. Following the reset, each of the PLLs may achieve a lock on the next rising edge of the reference clock signal. Thus, at the next rising edge of the reference clock signal, the rising edge of each of the domain clock signals will be aligned with each other, as well as being aligned with the reference clock signal.

DETAILED DESCRIPTION OF THE INVENTION

Turning now toFIG. 1, a block diagram of one embodiment of a computer system is shown. The computer system includes processors10A-10D each interconnected by a coherent packet bus15. Each section of coherent packet bus15may form a point-to-point link between each of processors10A-D. While four processors are shown using point-to point links it is noted that other numbers of processors may be used and other types of buses may interconnect them. The computer system also includes three I/O nodes numbered20,30and40each connected together in a chain by I/O packet buses50B and50C respectively. I/O packet bus50A is coupled between host node/processor10A and I/O node20. Processor10A is illustrated as a host node which may include a host bridge for communicating with I/O packet bus50A. Processors10B-D may also include host bridges for communication with other I/O packet buses (not shown). The communication links formed by I/O packet bus50A-C may also be referred to as a point-to-point links. I/O node20is connected to a pair of peripheral buses25A-B. I/O node30is connected to a graphics bus35, while I/O node40is connected to an additional peripheral bus45.

Processors10A-10D are each illustrative of, for example, an x86 microprocessor such as an Athlon™ microprocessor. In addition, one example of a packet bus such as I/O packet bus50A-50C may be a non-coherent HyperTransport™. Peripheral buses25A-B and peripheral bus45are illustrative of a common peripheral bus such as a peripheral component interconnect (PCI) bus. Graphics bus35is illustrative of an accelerated graphics port (AGP), for example. It is understood, however, that other types of microprocessors and other types of peripheral buses may be used.

It is noted that while three I/O nodes are shown connected to host processor10A, other embodiments may have other numbers of nodes and those nodes may be connected in other topologies. The chain topology illustrated inFIG. 1is shown for its ease of understanding.

In the illustrated embodiment, the host bridge of processor10A may receive upstream packet transactions from downstream nodes such as I/O node20,30or40. Alternatively, the host bridge of processor10A may transmit packets downstream to devices such as peripheral devices (not shown) that may be connected to peripheral bus25A for example.

During operation, I/O node20and40may translate PCI bus transactions into upstream packet transactions that travel in I/O streams and additionally may translate downstream packet transactions into PCI bus transactions. All packets originating at nodes other than the host bridge of processor10A may flow upstream to the host bridge of processor10A before being forwarded to any other node. All packets originating at the host bridge of processor10A may flow downstream to other nodes such as I/O node20,30or40. As used herein, “upstream” refers to packet traffic flow in the direction of the host bridge of processor10A and “downstream” refers to packet traffic flow in the direction away from the host bridge of processor10A. Each I/O stream may be identified by an identifier called a Unit ID. It is contemplated that the Unit ID may be part of a packet header or it may be some other designated number of bits in a packet or packets. As used herein, “I/O stream” refers to all packet transactions that contain the same Unit ID and therefore originate from the same node.

To illustrate, a peripheral device on peripheral bus45initiates a transaction directed to a peripheral device on peripheral bus25. The transaction may first be translated into one or more packets with a unique Unit ID and then transmitted upstream. It is noted that each packet may be encoded with specific information which identifies the packet. For example the Unit ID may be encoded into the packet header. Additionally, the type of transaction may also be encoded into the packet header. Each packet may be assigned a Unit ID that identifies the originating node. Since I/O node20may not forward packets to a peripheral device on peripheral bus25from downstream, the packets are transmitted upstream to the host bridge of processor10A. The host bridge of processor10A may then transmit the packets back downstream with a Unit ID of the host bridge of processor10A until I/O node20recognizes and claims the packet for the peripheral device on peripheral bus25. I/O node20may then translate the packets into peripheral bus transactions and transmit the transactions to the peripheral device on peripheral bus25.

As the packet transactions travel upstream or downstream, the packets may pass through one or more I/O nodes. The pass-through is sometimes referred to as a tunnel and the I/O node is sometimes referred to as a tunnel device. Packets that are sent from upstream to downstream or from downstream to upstream are referred to as “forwarded” traffic. Additionally, packet traffic that originates at a particular I/O node and is inserted into the upstream traffic is referred to as “injected” traffic.

As will be described in greater detail below, to preserve the ordering rules of the various buses that may be connected to an I/O node, the I/O node may provide transaction reordering as well as packet buffering. The I/O node may also include control logic which controls the flow of packets into and out of the tunnel by both forwarded and injected traffic.

Referring toFIG. 2, a block diagram of one embodiment of an input/output (I/O) node is shown. This particular description will refer to I/O node20for simplicity, but it should be noted that the description may apply as well to any of the I/O nodes shown inFIG. 1. Input/output node20includes an upstream packet bus interface and a downstream packet bus interface, implemented in this embodiment as transceiver circuit110and transceiver circuit120, respectively. Transceiver circuits110and120are coupled together through an I/O tunnel140. Transceiver circuits110and120may each include a transmitter and a receiver (not shown). The transmitters and receivers may be connected through I/O tunnel140such that the receiver of transceiver110may be coupled to the transmitter of transceiver120and the receiver of transceiver120may be coupled to the transmitter of transceiver110. I/O tunnel140may include command and data buses used to forward packet traffic through I/O node20. In addition, I/O node20includes a peripheral bus interface, implemented here as peripheral interface circuit150. Peripheral interface circuit150is coupled to transceivers110and120through I/O tunnel140. However as described above, transactions which originate at peripheral interface circuit150and then enter I/O tunnel140may be referred to as injected transactions. I/O node20also includes a tunnel control unit130which is coupled to control the transactions between transceivers110and120and peripheral interface150. I/O node20is also coupled to a peripheral bus25through the peripheral interface circuit150.

In the illustrated embodiment, transceivers110and120are part of two uni-directional communication paths through I/O tunnel140. Since each of the communication paths is uni-directional, either path may be connected as the upstream or downstream path. Thus, the injected traffic from peripheral interface circuit150may be provided to either of transceivers110and120. Transceivers110and120may each receive packet transactions into a receive buffer (not shown). As each transaction is received, a control command may be generated containing a subset of the information contained in the received command. The control command may include the Unit Id of the originating node, destination information, a data count and transaction type, for example. It is noted that the control command may include other information or may not include some of the information listed here. The control command may be sent from transceivers110and120to tunnel control unit130. Tunnel control unit130may further determine which of the received transactions may be sent from a respective transceiver and to indicate to the transceiver to send the transaction to the transaction's destination.

In the illustrated embodiment, peripheral interface circuit150may be thought of as having a downstream section and an upstream section. The downstream section may handle downstream transactions whose destination may be a device connected to peripheral bus25. Packets received by the downstream section may allow peripheral bus interface circuit to generate cycles on the peripheral bus, enabling communication with peripheral devices coupled to the bus. The upstream section may handle transactions which originate from devices that may be connected to peripheral bus25. Thus, peripheral interface circuit150includes a downstream command buffer160and a downstream data buffer170, each coupled to I/O tunnel140via tunnel interface logic155. Downstream command buffer160and downstream data buffer170are each coupled to peripheral bus25via bus interface circuit210. Peripheral interface circuit150also includes an upstream command buffer160and an upstream data buffer190, each coupled to I/O tunnel140via tunnel interface logic155. Upstream command buffer160and upstream data buffer170are also each coupled to peripheral bus25via bus interface circuit210. Peripheral interface circuit150further includes a control logic unit200which is coupled to both upstream command buffer180and downstream command buffer160.

It is noted that in the illustrated embodiment, peripheral bus25is illustrative of a variety of common peripheral buses such as a PCI bus, a PCI-X bus and an AGP bus for example. Additionally, bus interface circuit210may include circuitry for translating peripheral bus commands from such buses. Further, it is contemplated that in one particular embodiment, bus interface circuit210may be a bus interface such as a Compaq™ Rapid Enabler for PCI-X (CREX) interface. Lastly, it is contemplated that in certain embodiments, I/O node20may include more than one peripheral interface circuit (not shown) and that I/O node may include arbitration logic (not shown) which may arbitrate between commands sent from each of the peripheral interface circuits

Transactions may be sent from transceiver circuit110or120depending on which transceiver is connected as the downstream receiver. The command portion of the transaction may be stored in downstream command buffer160and the data portion of the transaction may be stored in downstream data buffer170. Likewise, when a transaction is sent from bus interface circuit210, the command portion of the transaction may be stored in upstream command buffer180and the data portion of the transaction may be stored in upstream data buffer190. Control logic unit200may be configured to control the conveyance of the transactions to and from bus interface210and I/O tunnel140.

In response to peripheral interface circuit150receiving upstream transactions from peripheral bus25, control logic unit200may generate control commands similar to the control commands generated by transceivers110and120. Control logic unit200may also send those control commands to tunnel control unit130, thus allowing upstream commands to be scheduled into I/O tunnel140. In addition, control logic unit200may provide downstream circuitry with signals corresponding to upstream commands which may allow the downstream circuitry to maintain the relative order of downstream response transactions.

FIG. 3is an alternate block diagram of the embodiment of the I/O node illustrated inFIG. 2, illustrating multiple clock domains. I/O node20includes a plurality of clock domains, wherein each of the clock domains may be synchronized to a clock having a different frequency than adjacent clock domains. Clock domain1402serves as the clock domain for I/O tunnel140. Switching circuitry1302may reside within clock domain1402, and may be a component of tunnel control unit130shown inFIG. 2.

Data to be transmitted in packets from I/O node20may be conveyed from switching circuitry1302into one of transmit (TX) construction circuits1112or1212. In the embodiment shown, TX construction circuit1112is a component of transceiver circuit110, while TX construction circuit1212is a component of transceiver circuit120. TX construction circuits1112and1212may be configured to organize data into packets or perform other functions necessary for transmitting information from I/O node20. In embodiments where the width of the packet bus is less than that of buses internal to I/O node20, TX construction circuit may perform a wide-to-narrow conversion of the packets.

Once packets are ready for transmission, it may be necessary to transfer them to the clock domain of TX circuitry1114or1214. Since these transmit circuits may be in clock domains different from I/O tunnel clock domain1402, it may be necessary to route the packets through a corresponding clock domain synchronizer500. Clock domain synchronizers500may be configured to “shift” the packets into a clock domain with a faster or slower clock frequency than that of clock domain1402. Clock domain synchronizers500will be discussed in further detail below.

Packets may be transmitted onto a packet bus link through TX circuitry1114or1214, depending upon the link. TX circuitry1114/1214may include circuitry for transmitting packet signals onto a packet bus link, as well as any necessary timing circuitry. Control circuitry may also be present, as transmissions onto the packet bus may be subject to arbitration. In addition, TX circuitry1114/1214may further include buffers or registers in order to store packets awaiting transmission onto the packet bus.

Packetized data and commands from a packet bus link coupled to I/O node20may be received through receive (RX) circuitry1102/1202, data synchronization units (shown here as CFF units1104/1204), and RX paths1106/1206. Switching circuitry1302may receive the packets from either of RX paths1106/1206, and route the packets to a TX construction circuit1112/1212(for packets passing through I/O node20to another packet bus link) or to a peripheral bus interface, such as peripheral interface circuit150ofFIG. 2.

In one embodiment, RX circuitry1102/1202may perform narrow-to-wide bus conversions. The packet bus links coupled to I/O node20may be narrower than the signal width of buses internal to the I/O node. Thus, RX circuitry1102/1202may convert packets received on a narrow packet bus link to a format suitable for a wider internal bus link.

After conversion, data and/or commands may be forwarded by RX circuitry1102/1202to a corresponding data synchronization unit. In the embodiment shown, the data synchronization units are clock-forwarding FIFO (first in first out) units1104/1204. CFF units1104/1204may be configured to control the flow of information from the packet bus link into I/O node20by providing buffering and clock synchronization between clock domains. CFF units1104/1204may be sized to absorb dynamic variation between a transmitter's clock (external to I/O node20) and the receiver's clock (generated internal to I/O node20). Because of variations in manufacturing process, voltage, and temperature, it is possible for clock cycle slippage to occur between different I/O nodes in a computer system. This may occur when the transmitter clock and the receiver clock are synchronized in frequency but not in phase. CFF units1104/1204may absorb these variations by temporarily storing information received from RX circuitry1102/1202in a first in first out (FIFO) memory. This may allow for the receiving of information by I/O node20from a packet bus link despite any accumulated phase difference between external and internal clocks.

Each of CFF units1104/1204may also, in some embodiments, perform a function similar to that of the clock domain synchronizers500. The presence of a FIFO memory in CFF units1104/1204may allow information to be received by a slower clock domain from a faster clock domain.

Information from each of the CFF units may pass through an RX path1106/1206into switching circuitry1302. RX paths1106/1206may be a bus configured to convey information into the switching circuitry. The bus may be sized to match the data width of a FIFO output from CFF units1104/1204.

I/O node20may also include bus clock domains1505and1506. Bus clock domains1505and1506may each include a plurality of clock domains which correspond to bus interfaces. Such bus interfaces may be configured to support peripheral buses such as a peripheral component interface (PCI) bus, an advanced graphics port (AGB) bus, a general purpose instrument bus (GPIB), and any other type of peripheral bus. Internal buses may also be coupled to a bus interface in one of bus clock domains1505/1506. Corresponding clock domain synchronizers500, as shown in the drawing, may allow information to be transmitted between bus clock domains1505/1506and clock domain1402. In the embodiment shown, bus clock domains1505/1506may each correspond to a peripheral bus interface such as peripheral interface circuit150ofFIG. 2. It should also be noted that multiple clock domains may be present within each of bus clock domains1505/1506.

Switching circuitry1302may perform a variety of switching functions related to the routing of information that may pass through I/O node20. Switching circuitry1302may include circuitry which routes information between the bus clock domains and the transmit and receive paths shown in the drawing (made up of RX circuitry1102/1202, TX circuitry1114/1214, etc.).

Turning now toFIG. 4, a block diagram of one embodiment of a circuit configured to perform edge synchronization for multiple clock signals is shown. Clock circuit501may be a component of clock domain synchronizer500shown inFIG. 3. Clock circuit501may include a clock unit502and a synchronization circuit520. Clock unit501may include a plurality of phase locked loops (PLLs)503. The output of each of PLLs503may be a clock signal, which may be conveyed to a clock domain in the system in which clock circuit501is implemented (such as I/O node20). Clock unit502may output n clock signals, wherein n is an integer value. There is no theoretical limit to the number of PLLs503that may be present in clock unit502, and consequently, the number of clock signals which it may produce. It should also be noted that other embodiments utilizing delay locked loops (DLLs) are possible and contemplated.

Each of PLLs503may receive a reference clock signal, and as previously noted, produce an output clock signal. In embodiments such as I/O node20ofFIGS. 2 and 3, the reference clock signal may be a signal produced external to the I/O node, while the output clock signals (clock1, clock2, etc.) may be local clock signals for one of the clock domains. The output clock signal from each of PLLs503may have a frequency that is greater than or less than the frequency of the reference clock signal. It may also be possible for the output clock signal of a given PLL503to have a frequency that is equal to the frequency of the reference clock signal.

In the embodiment shown, each of PLL503may be a typical PLL including a phase detector and a voltage controlled oscillator (VCO). Each PLL503may also include a divide-by-n counter, shown here as counter512in each of PLLs503. Counters512may each be placed in a feedback path between the VCO and phase detector. This may enable a given PLL503to output a clock signal having a frequency that is a multiple of the reference clock signal. Conversely, depending upon the counter, the frequency of the output clock signal may be a fraction of the input clock signal. Thus, multiple clock signals having different frequencies may be produced by clock unit502using the reference clock signal as an input to each of the PLLs503.

Synchronization unit520may also be configured to receive the reference clock signal, and may be further configured to drive a synchronization pulse to each of PLLs503. In the embodiment shown, synchronization unit520includes counter522and synchronization pulse logic524. Counter522may be configured to receive the reference clock signal, and may cycle through a plurality of states.

Synchronization pulse logic524may be coupled to counter522and may assert a synchronization pulse each time the counter cycles through a number of states N. In one embodiment, the assertion of the synchronization pulse may occur once every N clock cycles of the reference clock signal. The beat frequency may be defined as the frequency in which a given pattern of all of the clock cycles repeats itself relative to the reference clock signal. The beat frequency of the repeating clock patterns may be the frequency of the reference clock signal divided by N. For example, if the reference clock signal is 200 MHz, and the pattern repeats itself at 4 times relative to a cycle of the reference clock signal, then N=4 and the beat frequency is 50 MHz. Other embodiments are possible and contemplated wherein the beat frequency may be less than the frequency of the reference clock signal.

The outputs of counter522may be coupled to synchronization pulse logic524. In one embodiment, synchronization pulse logic524may be a combinational logic circuit. When the outputs of counter522reach a certain state, it may cause the synchronization pulse to be asserted. The synchronization pulse may be driven to each of the PLLs503. In addition, the synchronization pulse may be fed back to a reset input of counter522, which may cause the counter to be reset each time the synchronization pulse is asserted.

The synchronization pulse may also be received by each counter512within its respective PLL503. In particular, each counter512may receive the synchronization input through a reset input. Receiving the asserted synchronization pulse at the reset input may cause each of counters512to synchronously reset. When each of counters512is reset, it may cause its respective PLL503to attain a phase lock on a subsequent clock edge of the reference clock signal. In one embodiment, each PLL503may attain a phase lock on the next rising edge of the reference clock signal subsequent to the resetting of counters512. Thus, each PLL503may achieve a phase lock at approximately the same time, thereby aligning the rising edges of all of the clock signals.

It should be noted that other embodiments are possible and contemplated wherein the falling edges of the clock signals are aligned instead of the rising edges. This may be true for systems having synchronous logic circuits that are triggered by a falling edge of a clock signal instead of the rising edge. Furthermore, systems having a mix of synchronous circuits triggered by either the rising edges or the falling edges of a clock signal may be configured to perform alignment of either rising edges or falling edges.

By achieving phase lock for each of PLLs503subsequent to the assertion of the synchronization pulse, the relationship between the clock edges may be known at a given instant. Knowledge of this relationship may be critical when information is to be transferred between clock domains wherein the respective clock signals involved have different frequencies. When information is transferred between clock domains synchronized to clock signals of different frequencies, only certain “safe” clock cycles may be suitable for information transmission. Thus, to ensure that the transmissions occur during safe clock cycles, it may be of critical importance to know the relationship between the edges of the clock signals involved. Thus, by periodically asserting the synchronization pulse, the clock edges may be aligned in a repetitious fashion during the operation of a system in which clock circuit501is implemented.

Moving now toFIG. 5, a timing diagram illustrating edge synchronization of multiple clock signals that may be performed by one embodiment of clock circuit501is shown. In this particular example, three clock signals (clock1, clock2, and clock3) of different frequencies are generated based on a reference clock signal that may be received by a plurality of PLLs503. Clock1, clock2, and clock3may be clock signals that are used for synchronization in a local clock domain. Although each of the clock signals shown here has a higher frequency than the reference clock signal, it is possible and contemplated that some embodiments will include clock signals having a frequency lower than the reference clock signal.

The frequency at which the synch pulse is asserted may be dependent upon the relationship between the reference clock signal and the beat frequency. In this example, the beat frequency is ⅓ that of the reference clock frequency, and thus, N=3. Therefore, counter522in synchronization unit520(as shown inFIG. 4) may cycle through three different states for each synchronization pulse assertion. When the third state is reached, synchronization pulse logic524may assert the synchronization pulse. It should be noted that synchronization pulse logic524may include additional circuitry which may limit the width of the synchronization pulse. This may prevent the resetting and suppression of clock signals having a higher frequency than the reference clock signal.

When the synchronization pulse is asserted, each of the PLL counters512may be synchronously reset. After resetting each of the counters512, each PLL503will attempt to achieve a phase lock with the reference clock signal. Typically, each PLL503will attain the phase lock at the as the reference clock reaches its rising edge. As a result, the rising edge of each of the output clock signals may be aligned with the rising edge of the reference clock signal, as the reset and subsequent phase lock forces these clock edges into alignment. As such, the phase relationship between each of the clock signals may be known at that particular instance. Furthermore, the rising edges of the output clock signals may be aligned in a repeated fashion (every third rising edge in this example). This may allow the phase relationship of the clock signals to be known for a majority of the time a system is operating.

While the present invention has been described with reference to particular embodiments, it will be understood that the embodiments are illustrative and that the invention scope is not so limited. Any variations, modifications, additions, and improvements to the embodiments described are possible. These variations, modifications, additions, and improvements may fall within the scope of the inventions as detailed within the following claims.