Data transfer between asynchronous clock domains

Some implementations disclosed herein provide techniques and arrangements for transferring data between asynchronous clock domains. A synchronization signal may be generated by a first of the clock domains, and data may be transferred between the domains in response to the synchronization signal. Clock cycles of the second of the clock domains may be monitored in comparison to the synchronization signal to report the number of second clock domain cycles occurring per occurrence of the synchronization signal. This information may be recorded by testing and validation equipment to facilitate error analyses.

CROSS-REFERENCE TO RELATED APPLICATIONS

This patent application is a U.S. National Phase Application under 35 U.S.C. § 371 of International Application No. PCT/US2011/067612, filed Dec. 28, 2011, entitled DATA TRANSFER BETWEEN ASYNCHRONOUS CLOCK DOMAINS.

TECHNICAL FIELD

Some embodiments of the invention relate generally to asynchronous clock domains in systems or devices such as processors. More particularly, some embodiments of the invention relate to system or device configurations that facilitate testing of systems having asynchronous clock domains.

BACKGROUND ART

Integrated circuits traditionally use synchronous protocols for data transfer. Existing testing and validation technologies rely heavily on cycle-by-cycle, deterministic, synchronous models.

In a massively parallel architecture or a platform-level design, the number and diversity of interacting clock domains increases. Synchronizing all of the clock domains can be prohibitive because of engineering costs, power consumption, and project-level risks. Accordingly, such architectures and designs increasingly utilize multiple asynchronous clock domains. However, it is difficult to validate or emulate a fully asynchronous architecture with industry-standard validation hardware and software.

DETAILED DESCRIPTION

Large-scale integrated circuits and other systems, including processors and other devices, increasingly use multiple asynchronous clock domains. Asynchronous clock domains often communicate with each other through an intermediate buffer that can be accessed from the different clock domains.

During development, devices and logic can be monitored to confirm that they operate as intended and produce expected results. However, available testing and diagnostic equipment is typically not designed for operation with asynchronous clock domains. Rather, such testing and diagnostic equipment typically relies on deterministic relationships between components and subsystems. Determinism in this context refers to the certainty of clock cycle on which a processor event occurs. It ensures repeatability of the event, which is fundamental to most high-volume manufacturing experiments involving that event. The non-deterministic nature of asynchronous clock domains presents a challenge, particularly when a developer wants to record and subsequently reproduce conditions and events leading to a detected fault or error.

The examples described below provide a way for test equipment to record and reproduce the runtime interactions between asynchronous clock domains. In certain embodiments, a first clock domain and a second clock domain transfer data in response to a synchronization signal that is generated synchronously with the clock of the first clock domain. During each period of the synchronization signal, referred to herein as a synchronization cycle, the number of clock cycles in the second clock domain may vary. However, the implementations described below generate a reporting output to indicate the correspondence between synchronization cycles and cycles of the second clock domain. The reporting output may be recorded by testing equipment, and allows the testing equipment to determine, in response to actual operating conditions, the number of clock cycles that occur in both the first and second clock domains during each synchronization period. This allows subsequent emulation of device operation, and recreation of recorded events.

FIG. 1illustrates a device or system100having a first clock domain102and a second clock domain104. As an example, the system100may be a processor or similar type of device.

The first clock domain102has or is associated with a first clock106that generates a first clock signal CLOCK1. The second clock domain104has or is associated with a second clock108that generates a second clock signal CLOCK2. The first and second clock signals CLOCK1and CLOCK2are asynchronous, and therefore may not have deterministic frequency or phase relationships.

The first and second clock domains102and104may each have various components and/or elements, not shown, that operate in response to the respective clock signals CLOCK1and CLOCK2.

The system100may have a synchronization clock110that is responsive to the first clock signal CLOCK1to generate a synchronization signal or pulse SYNC. In some embodiments, the synchronization clock110may be implemented as a frequency divider, so that the synchronization pulse SYNCoccurs at a lower frequency than that of the first clock signal CLOCK1while having a fixed phase relationship with the first clock signal CLOCK1. For example, the synchronization clock110may produce a single synchronization pulse corresponding to every N cycles of the first clock signal CLOCK1, where N is an integer. The periods defined by the synchronization signal will be referred to herein as synchronization periods or cycles.

The system100may also have data transfer logic112to transfer data between the first and second clock domains102and104. The data transfer logic112may comprise one or more intermediate registers or buffers, and may be responsive to the synchronization pulse SYNCto initiate and/or perform data transfers or exchanges. During an exchange cycle, data may be transferred from the first clock domain102to the second clock domain104, and/or from the second clock domain104to the first clock domain102. A more specific implementation of the transfer logic112will be described below, in conjunction with the description ofFIG. 4.

The system100may include reporting logic114that produces a clock count signalCOUNTcorresponding to each synchronization cycle. The clock count signalCOUNTmay be generated in response to the synchronization pulse SYNCand the second clock signal CLOCK2. The clock count signalCOUNTmay indicate, for every SYNCpulse or corresponding synchronization cycle, the number of occurring cycles of the second clock signal CLOCK2. Note that because of the asynchronous relationship between the first and second clock domains, the number of CLOCK2cycles occurring between SYNCpulses may be indeterminate, and may vary over time. The clock count signalCOUNTindicates the number of actually occurring CLOCK2cycles corresponding to individual synchronization periods.

The clock count signalCOUNTmay be produced and output in synchronization with the first clock signal CLOCK1, the second clock signal CLOCK2, and/or the synchronization pulse SYNC.

The clock count signalCOUNTprovides a mechanism for correlating and reporting the runtime operations of the first clock domain102and the second clock domain104, which can be recorded and used by testing and validation equipment to reproduce conditions and sequences that precede error conditions. Because the SYNCpulse is generated from the first clock signal CLOCK1, the number of CLOCK1cycles per SYNCpulse is known. Because the second clock domain104is asynchronous with the first clock domain102, the number of CLOCK1cycles per SYNCpulse is potentially variable. However, the reporting logic114monitors actual observed performance, and provides the clock count signalCOUNTas an indication of how many CLOCK2cycles actually occur for each SYNCpulse. Thus, the two clock domains can be allowed to run asynchronously, while their operations can be monitored, recorded, and correlated to each other by validation and testing equipment: for every SYNCpulse, it is possible to determine and record the number of corresponding CLOCK1and CLOCK2cycles that actually occurred.

FIG. 2illustrates signal relationships in the system100. Note that in this illustration, signals are considered to become active on their rising edges.

The CLOCK1signal may be a cyclical, repetitive, and/or periodic signal such as the square wave illustrated. The synchronization pulse SYNCmay similarly be a cyclical, repetitive, or periodic signal, such as a repetitively or periodically occurring pulse. As described above, the synchronization pulse SYNCmay be synchronized with the CLOCK1signal, and may be at a lower frequency than that of the CLOCK1signal. In the illustrated example, the synchronization pulse SYNCis repeated once for every two cycles of the CLOCK1signal. More generally, the synchronization pulse SYNCmay occur once for every N cycles of the CLOCK1signal, where N is an integer. Each synchronization pulse SYNCcorresponds to a respective synchronization cycle.

The CLOCK2signal may be a cyclical, repetitive, and/or periodic signal such as the square wave illustrated. The CLOCK2signal may have a different frequency than that of the CLOCK1signal, and may have an indeterminate or variable phase relationship with both the CLOCK1signal and the SYNCsignal.

In this example, the frequency of the CLOCK2signal is such that it occurs either once or twice for each repetition, period, or cycle of the SYNC pulse. At the rising edge of each SYNCpulse, theCOUNTsignal is updated to indicate the actual number of CLOCK2cycles (indicated by x's inFIG. 2) that occurred during the previous SYNCperiod, where synchronization periods are defined by the rising edges of the SYNCsignal (indicated by dashed vertical lines inFIG. 2). A low value of theCOUNTsignal in this example corresponds to one cycle of the CLOCK2signal, and a high value of theCOUNTsignal corresponds to two cycles of the CLOCK2signal. TheCOUNTsignal may of course be used to indicate different CLOCK2counts, in situations where the nominal frequencies of the CLOCK1and CLOCK2signals are different than shown. In addition, theCOUNTsignal may in some situations comprise a multi-bit signal or value.

FIG. 3illustrates an example device or system300in which an intermediate storage element or buffer302is used for asynchronously transferring data between a first clock domain304and an asynchronous second clock domain306. The example device or system300uses the techniques described above to transfer buffer pointers between the first and second clock domains304and306.

The intermediate storage buffer302may comprise an elastic buffer such as a first-in-first-out (FIFO) buffer. In the given example, the first clock domain304writes data to the FIFO buffer302in synchronization with a first clock signalCLOCK1. The second clock domain306reads data from the FIFO buffer in synchronization with a second clock signalCLOCK2. The first and second clock signalsCLOCK1andCLOCK2may be asynchronous.

In order to coordinate writing and reading between the first and second clock domains304and306, write and read pointers are maintained within the first and second clock domains304and306. More specifically, the first clock domain304maintains a write pointer308, indicating the address of the next position of the FIFO buffer302to be written. After the first clock domain304writes to this position of the FIFO buffer302, the write pointer308is incremented.

The second clock domain306maintains a read pointer310, indicating the address of the next position of the FIFO buffer302to be read. After the second clock domain306reads from this position of the FIFO buffer302, the read pointer310is incremented.

The first clock domain304may also have a shadow or duplicate read pointer312, which is updated from time to time to reflect the value of the read pointer310of the second clock domain306. To prevent overwriting data that has not yet been read by the second clock domain306, the first clock domain304does not perform writes to locations beyond the address indicated by the shadow read pointer312.

Similarly, the second clock domain306may have a shadow or duplicate write pointer314, which is updated from time to time to reflect the value of the write pointer308of the first clock domain304. To avoid reading invalid data, the second clock domain306does not perform reads from locations beyond the address indicated by the shadow write pointer314.

The system100may have a synchronization clock316, which is configured similarly to the synchronization clock110ofFIG. 1to generate a synchronization signal or pulse SYNCbased on the first clock signal CLOCK1. In this embodiment, the SYNCpulse is used to update the shadow read pointer312and the shadow write pointer314. More specifically, the SYNCpulse is used to clock or latch data from the read pointer310into the shadow read pointer312, and from the write pointer308into the shadow write pointer314.

The shadow read pointer312may comprise a latch or register that receives the current value of the read pointer310from the second clock domain306. This value is captured by the latch or register312upon or in response to receiving the SYNCpulse. Similarly, the shadow write pointer314may comprise a latch or register configured to receive the current value of the write pointer308from the first clock domain304. This value is captured by the latch or register314upon or in response to receiving the SYNCpulse.

Similar to the embodiment ofFIG. 1, the system300may have reporting logic318to report the number of CLOCK2cycles that occur during every repetition or period of the SYNCsignal. The reporting logic318may generate a clock count signalCOUNTto indicate correlation between clock cycles of the first clock domain304and clock cycles of the second clock domain306. As described above, theCOUNTsignal can be used by testing and validation equipment to record and later emulate conditions and sequences that precede error conditions.

The shadow read pointer312and the shadow write pointer may be clocked directly by the SYNCpulse, or may be clocked in synchronization with the respective clock domains in response to respective SYNCpulses. For example, the read pointer312may be clocked by the CLOCK1signal in response to each SYNCpulse. Similarly, shadow the write pointer314may be clocked by the CLOCK2signal in response to each SYNCpulse.

FIG. 4illustrates an example implementation of a system400that demonstrates the concepts described above. The system400has a first clock domain402and a second clock domain404. The first clock domain402operates in response to a first clock signal CLOCK1. The second clock domain404operates in response to an asynchronous second clock signal CLOCK2.

The first and second clock domains402and404transfer data using an intermediate elastic buffer or FIFO (not shown), which may be similar to the FIFO buffer302ofFIG. 3. The buffer is addressed by pointers as described with reference toFIG. 3. Specifically, the first clock domain402maintains a write pointer406and a shadow read pointer408. The second clock domain404maintains a read pointer410and a shadow write pointer412. The shadow write pointer412is updated periodically to reflect the value of the write pointer406. The shadow read pointer408is updated periodically to reflect the value of the read pointer410.

In this example, a synchronization clock is implemented as a divide-by-N frequency divider414within the first clock domain402. The frequency divider414generates SYNCsignal as a function of the first clock signal CLOCK1. The SYNCsignal may comprise a periodic or repetitive pulse that occurs at a lower frequency than the CLOCK1signal, while also being synchronous with the CLOCK1signal.

The SYNCsignal is communicated or propagated from the first clock domain402, to the second clock domain404, and then back to the first clock domain402. More specifically, SYNCsignal is transmitted from the first clock domain402to the second clock domain404through a first metastable-hardened flip-flop416. The flip-flop416produces a WRITESYNCsignal, which may be a delayed version of the SYNCsignal. The WRITESYNCsignal is transmitted from the second clock domain404back to the first clock domain402through a second metastable-hardened flip-flop418. The flip-flop418produces a READSYNCsignal, which may be a delayed version of the WRITESYNCsignal.

In this embodiment, DQ latches are used to transfer pointers between the first and second clock domains402and404, in response to the various versions or instances of the synchronization signal, which include the SYNC signal, the WRITESYNCsignal, and the READSYNCsignal.

A first latch or transfer register420, within the first clock domain402, receives at its input the value of the write pointer406of the first clock domain402. A synchronization cycle is initiated by the SYNCsignal, which latches the write pointer value into the latch420so that it can be received and read by the second clock domain404.

A second latch or transfer register422, within the second clock domain404, receives the write pointer value from the first latch420, and is responsive to the WRITESYNCsignal to latch this value.

A third latch or transfer register424, within the second clock domain404, receives at its input the value of the read pointer410of the second clock domain404. This value is latched into the third latch424in response to the WRITESYNCsignal, so that the value can be received and read by the first clock domain402.

A fourth latch or transfer register426, within the first clock domain402, receives the read pointer value from the third latch424, and is responsive to the READSYNCsignal to latch the read pointer value from the third latch424.

These events can be summarized as the following sequence of actions, which together may be referred to as a synchronization cycle in this embodiment:SYNCinitiates the synchronization cycle and latches the write pointer into the output latch420;WRITESYNClatches the write pointer from the output latch420into the input latch422;WRITESYNCalso latches the read pointer410into the output latch424; andREADSYNClatches the read pointer from the output latch424into the input latch426.

The second clock domain404may have reporting logic428, similar to the reporting logic described above. The reporting logic428is responsive to the WRITESYNCsignal and to the second clock CLOCK2, and produces a COUNToutput indicating, for each cycle or pulse of WRITESYNC, the number of corresponding CLOCK2cycles that occurred during the previous WRITESYNCcycle or synchronization cycle.

The value of N can be chosen based on the ratio of the first clock signal CLOCK1and the second clock signal CLOCK2, in a manner that minimizes the potential for data overwrites and/or data starvation. If the synchronization pace defined by the SYNCsignal is too fast, the first clock domain may initiate a synchronization cycle before the second clock domain has had a chance to process a previous synchronization cycle. If the synchronization pace defined by the SYNCsignal is too slow, the second clock domain may at times be starved for data, even though there is unread data in the FIFO buffer.

The elements within the dashed box430may be considered or referred to as write pointer transfer logic. The elements within the dashed box432may be considered or referred to as read pointer transfer logic. The write and read pointer logic430and432represent an example implementation of the transfer logic112shown inFIG. 1.

FIG. 5illustrates an example embodiment500, which is a variation of the embodiment ofFIG. 4. InFIG. 5, the write and read pointer logic430and432are each replicated K times to produce a multi-register write buffer502and a multi-register read buffer504. Each of these buffers may be configured to operate in FIFO fashion, allowing the first and second clock domains402and404to process synchronization cycles at different rates. For example, this may allow the first clock domain402to initiate synchronization cycles at a rate that is faster than the rate at which the second clock domain404is able to process the synchronization cycles.

The embodiment ofFIG. 5includes a K-Phase clock generator506that is responsive to the single-bit SYNCsignal to create a multi-bit SYNCK signal. The SYNCK signal comprises K signal bits, which are used to clock respective instances of write pointer logic430and read pointer logic432.

FIG. 6illustrates the relationship of the individual bits of the SYNCK signal to the single-bit SYNCsignal. This example assumes K=4. The SYNCK signal comprises K individual bits or signals, referred to as SYNC1, SYNC2, SYNC3, and SYNC4. Each of the individual SYNCK signals is generated by dividing the SYNCsignal by K. In addition, the individual SYNCK signals are staggered from each other, so that a single SYNCK signal is generated for each occurrence of the SYNCsignal.

Referring again toFIG. 5, each of the write transfer logic instances430may correspond to and be responsive to a different one of the individual SYNCK signal bits. Similarly, each of the read transfer logic instances432may be responsive to a different one of the individual SYNCK signal bits. This arrangement effectively implements FIFO logic for the transferred pointers, allowing them to be written and read at different paces by the first and second clock domains402and404.

FIG. 7illustrates an example of a method700for transferring data between asynchronous clock domains. An action702comprises clocking a first clock domain with a first cyclical clock signal. An action704comprises clocking a second clock domain with a second cyclical clock signal. As discussed above, the first and second cyclical clock signals may be asynchronous.

An action706comprises generating a synchronization signal. The synchronization signal may be synchronous with the first clock signal, and may be generated by dividing the frequency of the first clock signal by an integer N. Thus, the first clock signal may have a frequency that is an fixed integer multiple of the synchronization signal.

An action708may comprise propagating the synchronization signal from the first clock domain to the second clock domain, and then from the second clock domain back to the first clock domain.

An action710may comprise aligning the pointers between the first and second clock domains in response to the synchronization signal. A first buffer pointer may be transferred from the first clock domain to the second clock domain response to propagating the synchronization signal from the first clock domain to the second clock domain. A second buffer pointer may be transferred from the second clock domain to the first clock domain in response to propagating the synchronization signal from the second clock domain back to the first clock domain.

An action712may comprise indicating and/or reporting the correspondence between the synchronization signal and cycles of the second clock signal. For example, the action710may comprise indicating, for each occurrence of the synchronization signal, the number of corresponding cycles of the second clock signal.

FIG. 8is a block diagram of an illustrative architecture of a system800in which the techniques described above may be implemented. The system800may include one or more processors802-1, . . . ,802-N (where N is a positive integer ≥1), each of which may include one or more processor cores804-1, . . . ,804-M (where M is a positive integer ≥1). In some implementations the processor(s)802may be a single core processor, while in other implementations, the processor(s)802may have a large number of processor cores, each of which may include some or all of the components illustrated inFIG. 8.

The processor(s)802and processor core(s)804can be operated, via an integrated memory controller (IMC)810in connection with a local interconnect816, to read and write to a memory812. The processor(s)802and processor core(s)804can also execute computer-readable instructions stored in the memory812or other computer-readable media. The memory812may include volatile and nonvolatile memory and/or removable and non-removable media implemented in any type of technology for storage of information, such as computer-readable instructions, data structures, program modules or other data. Such memory may include, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology. In the case in which there are multiple processor cores804, in some implementations, the multiple processor cores804may share a shared cache814, which may be accessible via the local interconnect816.

Storage818may be provided for storing data, code, programs, logs, and the like. The storage818may include solid state storage, magnetic disk storage, RAID storage systems, storage arrays, network attached storage, storage area networks, cloud storage, CD-ROM, digital versatile disks (DVD) or other optical storage, magnetic cassettes, magnetic tape, or any other medium which can be used to store desired information and which can be accessed by a computing device. Depending on the configuration of the system800, the memory812and/or the storage818may be a type of computer readable storage media and may be a non-transitory media.

In various embodiments, the local interconnect816may also communicate with a graphical controller or graphics processing unit820to provide graphics processing. Additionally, in some embodiments, the local interconnect816may communicate with a system agent822. The system agent822may be in communication with a hub824, which connects a display engine826, a PCIe828, and a DMI830.

The memory812may store functional components that are executable by the processor(s)802. In some implementations, these functional components comprise instructions or programs832that are executable by the processor(s)802. The example functional components illustrated inFIG. 8further include an operating system (OS)834to manage operation of the system800.

The system800may include one or more communication devices836that may include one or more interfaces and hardware components for enabling communication with various other devices over a communication link, such as one or more networks838. For example, communication devices836may facilitate communication through one or more of the Internet, cable networks, cellular networks, wireless networks (e.g., Wi-Fi, cellular) and wired networks. Components used for communication can depend at least in part upon the type of network and/or environment selected. Protocols and components for communicating via such networks are well known and will not be discussed herein in detail.

The system800may further be equipped with various input/output (I/O) devices840. Such I/O devices840may include a display, various user interface controls (e.g., buttons, joystick, keyboard, touch screen, etc.), audio speakers, connection ports and so forth. An interconnect824, which may include a system bus, point-to-point interfaces, a chipset, or other suitable connections and components, may be provided to enable communication between the processors802, the memory812, the storage818, the communication devices836, and the I/O devices840.

Although the subject matter has been described in language specific to structural features and/or methodological acts, the subject matter defined in the appended claims is not limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as example forms of implementing the claims. This disclosure is intended to cover any and all adaptations or variations of the disclosed implementations, and the following claims should not be construed to be limited to the specific implementations disclosed in the specification. Instead, the scope of this document is to be determined entirely by the following claims, along with the full range of equivalents to which such claims are entitled.