Multi-channel synchronization for programmable logic device serial interface

A serial interface for a programmable logic device substantially eliminates skew across multiple channels both in the receiver and in the transmitter. Even when the channels are independent (e.g., are in different quads), skew is substantially eliminated by monitoring to determine when all channels have reached their active states (i.e., in the case of receiver channels when all channels have achieved byte alignment and have received an alignment character, and in the case of transmitter channels when all transmit PLLs have locked), and only then allowing data to flow between the serial and parallel domains.

BACKGROUND OF THE INVENTION

This invention relates to synchronization across multiple channels in a high-speed serial interface, especially in a programmable logic device. More particularly, this invention relates to synchronization of either multiple receiver channels or multiple transmit channels, especially in a programmable logic device when not all channels may be used.

Recently PLDs have begun to incorporate high-speed serial interfaces to accommodate high-speed (i.e., greater than 1 Gbps) high-speed serial I/O standards—e.g., the XAUI (10 Gbps Extended Attachment Unit Interface) standard. In accordance with the XAUI standard, a high-speed serial interface includes transceiver groups known as “quads,” each of which includes four transceivers and some central logic. Within each transceiver, the receiver portion typically includes a phase-locked loop (“PLL”), primarily for the purpose of enabling clock data recovery from a received high-speed serial signal. In addition, the central logic typically includes a PLL, primarily for the purpose of generating a transmit clock to be used by the transmitter portion of each of the four transceivers, and in some cases for generating a reference clock for the receiver PLLs.

In most cases, the individual receivers or transmitters in a quad are intended to be used together, for multi-channel reception or transmission of related signals. However, because of skew on the device on which the various receivers or transceivers are provided, there may be difficulties in synchronizing across the various channels. For example, in the case of reception on multiple channels under the XAUI standard, successive bytes or words of a message are sent on successive channels in a “round robin” scheme. Align characters preferably are sent substantially simultaneously on each channel. Although the multiple channels may have been aligned at the source, data converted from the serial domain back to the parallel domain on the received channels may be misaligned as a result of skew between the channels. Similarly, in the case of transmission from the parallel domain to the serial domain, it is desirable for the serial output clocks of the various channels to be aligned within one period or “unit interval (UI).” However, if different channels are ready to transmit at different times, and each channel simply begins transmitting when it is ready, the clocks can easily be misaligned by more than one UI.

While it is known to provide solutions for such synchronization problems when all receivers or transmitters are in a single quad, there may be applications in which receivers or transmitters are spread across multiple quads. It would be desirable to be able to provide synchronization across multiple channels, even across quad boundaries.

SUMMARY OF THE INVENTION

The present invention provides high-speed serial interface circuitry on a programmable logic device that is more flexible than previously known high-speed serial interfaces in that different channels can be synchronized, even across quad boundaries. This is accomplished by providing synchronization circuitry that determines when all desired ones of the channels are active, or ready, and, when all desired channels are active or ready, initiates communication on each channel between the serial domain and the parallel domain (i.e., from the serial domain to the parallel domain in the case of a receiver, and from the parallel domain to the serial domain in the case of a transmitter).

In the case of receivers, in one embodiment each receiver preferably includes a FIFO storage device that holds or stores the parallel words or bytes converted from the incoming serial bit data. In accordance with an embodiment of the present invention, the data words for each channel preferably are held in the respective FIFO for that channel until it is determined that all channels have locked on their respective clocks and are holding data words in their FIFOs, whereupon the data words preferably are released, for use “round robin” style if appropriate, under control of the respective clocks, starting with the respective align characters.

A preferred embodiment of a mechanism for determining that all receiver channels are locked includes a synchronization status output from the byte alignment circuitry of each receiver. For each receiver, that “sync status” signal is ANDed with a cascade of ANDed such signals from neighboring channels. The result of that AND operation in the last of the channels preferably is used as a read-enable signal for the FIFOs of all of the channels.

In a particularly preferred embodiment, the cascade input of each sync status AND gate is selectable (e.g., via a multiplexer or OR gate) between the cascade signal or a permanent “high” signal. Selecting the high signal effectively ignores all sync status signals further up the cascade, thereby providing the ability to eliminate those channels from the grouping being synchronized. It should similarly be noted that the output of any sync status AND gate can be tapped as the read-enable signal, allowing channels further down the cascade to be eliminated from the grouping. Thus, according to this embodiment, any contiguous subset of channels can be synchronized. In fact, a plurality of contiguous subsets of channels can be grouped, and each subset can be synchronized separately from the others.

In another preferred embodiment, even a noncontiguous subset of channels can be synchronized. This embodiment is similar to the previous embodiment that allows a contiguous subset of channels to be synchronized, except that not only does the cascade input to the sync status AND gate have a selectable input, but so does the input for the status of the channel of which the AND gate is a part. If a channel's status input is selected to be permanently high, that channel's status is ignored and the channel is effectively removed from the cascade, without removing the channels above it. In this way, any noncontiguous group of channels can be synchronized, but this embodiment can also be used for a contiguous group of channels. However, unless a plurality of cascade chains is provided, the ability of this arrangement to provide complete freedom in selecting channels to be included in multiple cascades is limited. In fact, a plurality of groupings of noncontiguous channels using a single cascade chain is possible only if the channels in each grouping are not interleaved with the channels of other groupings—i.e., each grouping is a contiguous or noncontiguous subset of a collection of contiguous channels, with each collection of contiguous channels being a contiguous subset of the total number of channels.

In both of the foregoing embodiments, the enable signal—i.e., the output of the last AND gate in the cascade—must be communicated to all channels in the grouping as the read-enable signal for the FIFO of each channel in the group.

In one embodiment, the read-enable signal is conducted by a repeater line, which is a conductor broken at each channel boundary by a repeater that can be set to pass a signal in only one of two directions, or not at all. This embodiment is useful only with one or more contiguous channel groupings, because each repeater can stop the read-enable signal at a channel boundary, but cannot be used to skip a channel. Therefore this embodiment is useful with the embodiment described above for providing a contiguous channel grouping or groupings, and also is useful for a noncontiguous grouping or groupings as long as skipped channels are not being used and as long as, in the case of multiple groupings, the channels of the multiple groupings are not interleaved as defined above.

In another embodiment, a routing network having a plurality of conductors which can be switchably connected to the read-enable signal output of the AND-gate cascade, and to the read-enable signal inputs of the FIFOs of the individual channels, is provided. This embodiment works with both interleaved and non-interleaved groupings, and the number of groupings that can be accommodated is limited only by the number of conductors in the routing network.

In the case of transmitters, in a preferred embodiment of the type used with the XAUI standard as described above, synchronization of the transmit clock as among individual channels in a single quad is not normally of concern, insofar as normally in such an arrangement there is only one transmit PLL per quad providing the serial transmit clock and one clock divider deriving the parallel transmit clock from the serial transmit clock, and all channels within the quad use the same transmit clocks. However, if channels are grouped across quad boundaries (whether or not all channels in each involved quad are part of the grouping), then synchronization of the transmit clocks generally is required. For groupings across quad boundaries, it normally would be expected that all involved quads are in the same clock domain—i.e., that the transmit PLLs in the grouped quads share a common reference clock.

In a preferred embodiment for transmitter synchronization, the lock status output of the transmit PLL in each quad in which there is at least one involved channel is monitored. The clock dividers in each of the involved quads are not enabled until the transmit PLLs in all of the involved quads are locked. In this way, the clock dividers preferably are enabled within one serial period, or serial unit interval, of each other.

Preferably, in such an embodiment the circuitry for monitoring the lock status of the transmit PLLs in the various quads is constructed from programmable logic in the logic core of the PLD. For example, an n-input AND gate, where n is the number of quads involved, can be used to monitor the PLL lock status, with the output of the n-input AND gate being the clock-enable signal for the various clock dividers.

In order for the serial clocks of the various grouped quads to be within one serial UI as desired, the clock-enable signal preferably should reach the clock dividers of all quads as close to the same time as possible. One way to accomplish this is to conduct the clock-enable signal to a plurality of registers, each adjacent one of the grouped quads, and then to clock the clock-enable signal out of those registers substantially simultaneously using a single low-skew global clock of the PLD. Preferably the clock-enable signal is conducted to the various registers with as little skew as possible, and in event it should reach all registers within the same cycle of the global clock. Another way to accomplish this is to distribute the clock-enable signal using a low-skew balanced distribution network similar to a low-skew balanced clock tree.

Therefore, in accordance with the present invention, there is provided a serial interface for use in a programmable logic device. The serial interface includes a plurality of channels, each of the channels communicating between a parallel domain clocked by a parallel clock and a serial domain clocked by a serial clock that is a multiple of the parallel clock. The serial interface further includes synchronization circuitry that, when all desired ones of the channels are in an active state, initiates communication in each of the desired ones of the channels between the parallel domain and the serial domain. The interface can be either a transmitter or a receiver, or can include both. A programmable logic device incorporating such an interface is also provided.

DETAILED DESCRIPTION OF THE INVENTION

As described above, the present invention synchronizes multiple channels of a serial interface in a PLD by monitoring each channel to determine when all channels have achieved an active state, and only then allowing communication between the serial domain and the parallel domain. The invention is useful in both the receiver portion (serial domain to parallel domain) and the transmitter portion (parallel domain to serial domain) of such a serial interface. In the case of a receiver, the indication of an active state is detection of a channel alignment character after byte alignment has occurred, while in the case of a transmitter, the indication of an active state is detection of a lock signal from the transmitter PLL. Similarly, in the case of a receiver, allowing communication between the serial domain and the parallel domain refers to enabling the reading out from a FIFO memory of data bytes converted from input serial data, while in the case of a transmitter, allowing communication between the serial domain and the parallel domain refers to enabling a clock divider on the transmitter PLL output to provide a parallel clock so that data can be transmitted from the parallel domain.

The invention will now be described with reference toFIGS. 1-11.

PLD10, shown schematically inFIG. 1, is one example of a device incorporating a serial interface20according to the invention. PLD10has a the programmable logic core including programmable logic regions11accessible to programmable interconnect structure12. The layout of regions11and interconnect structure12as shown inFIG. 1is intended to be schematic only, as many actual arrangements are known to, or may be created by, those of ordinary skill in the art.

PLD10also includes a plurality of other input/output (“I/O”) regions13. I/O regions13preferably are programmable, allowing the selection of one of a number of possible I/O signaling schemes, which may include differential and/or non-differential signaling schemes. Alternatively, I/O regions13may be fixed, each allowing only a particular signaling scheme. In some embodiments, a number of different types of fixed I/O regions13may be provided, so that while an individual region13does not allow a selection of signaling schemes, nevertheless PLD10as a whole does allow such a selection.

For example, each I/O region20preferably is a high-speed serial interface as described above, similar to an interface capable of implementing the XAUI standard. Thus, as shown inFIG. 2, each interface20preferably includes a plurality of groupings200having four channels21-24, each including a transmitter25and a receiver26, as well as central logic27. As discussed above, because each such grouping includes four channels, it may be referred to as a “quad.” However, it should be understood that in accordance with the present invention, which is not linked to any particular high-speed serial standard, each grouping200can include any number of channels. Similarly, while each region20is shown to contain two groupings200, each region20may contain any number of groupings200greater than or equal to two.

As shown inFIG. 1, PLD10includes five interfaces20. However, PLD10may include any desired number of interfaces20, with a corresponding number of channels.

Within each interface20, all transmitters25preferably are identical, and all receivers26preferably are identical, and preferably are substantially similar to known high-speed serial interface transmitters and receivers such as those used with the XAUI standard. It should be noted that any differences between transmitter25or receiver26and known high-speed serial transmitters and receivers preferably maintain compatibility with existing standards such as the XAUI standard, while adding capabilities as described herein.

FIG. 3shows the affect of skew on four channels (channel0, channel1, channel2and channel3) of data that originally are aligned but are misaligned after being processed in different receivers that do not incorporate the present invention. As shown, each channel n includes a plurality of fields Xn+1, Yn+1, Zn+1, as well as a channel alignment character30. Although channel alignment character30is identified inFIG. 3by the legend “COMMA,” it may be any predetermined character. As seen in the left-hand portion31ofFIG. 3, all of fields Xn+1, Yn+1, Zn+1, as well as channel alignment character30originate in a state of alignment across all four channels at their source, but as seen in right-hand portion33, after passage through whatever route (indicated by arrow32) by which they are conducted to receivers26, the various fields and the alignment character are no longer aligned. In the particular example shown, channels0and2remain aligned, while channels1and3are late relative to channels0and2, although channels1and3remain aligned relative to each other. It will be appreciated that in practice, each channel that is delayed could be delayed by a different amount, and in many cases all channels would be affected by delay. As stated above, for a standard, such as the XAUI standard, in which bytes are read “round-robin” style from all channels, channel alignment is required for proper reconstruction of the data.

One embodiment of the receiver portion40of a serial interface that solves the alignment problem is shown inFIG. 4, in which three channels41-43of an eight-channel receiver are shown. Each channel preferably includes a CDR/Deserializer stage400which recovers a clock signal401from the incoming serial data402and converts serial data402to parallel data403. Recovered clock signal401is used by subsequent stages404,405of the channel, and also is made available to clock tree44in the programmable logic core of PLD10for whatever purpose the user configuration may require it as well as for the specific purpose described below.

In byte alignment stage404, the byte boundaries of parallel data403are identified using techniques such as those described in copending, commonly-assigned U.S. patent application Ser. No. 10/454,626, filed Jun. 3, 2003, which is hereby incorporated by reference in its entirety. When the byte boundaries have been aligned, and alignment character30has been detected, byte alignment stage404preferably asserts SYNC STAT (i.e., synchronization status) signal406. SYNC STAT signal406serves as the write-enable input407to FIFO storage device405. Thus, when SYNC STAT signal406is asserted, meaning that byte alignment stage404has identified the boundaries of parallel data403and detected alignment character30, FIFO405begins to store data, with alignment character30in the first position under control of recovered clock signal401which functions as the write clock for FIFO405. This occurs separately in each channel41-43(as well as the remaining channels that are not shown).

In this preferred embodiment, SYNC STAT signal406of each channel41-43(as well as the remaining channels that are not shown) is input into synchronization cascade circuitry45which facilitates a determination that all channels of interest have achieved synchronization, so that reading of the data in the various FIFOs405may begin. In this embodiment, synchronization cascade circuitry45preferably includes a respective two-input AND gate450in each of channels41-43(as well as the remaining channels that are not shown). One input (“the channel input”) of AND gate450preferably is the SYNC STAT signal406for the respective channel of which gate450is a part, while the other input (“the cascade input”) of AND gate450preferably is the output of AND gate450of the immediately preceding channel. For the first channel, the cascade input is held high.

It can be seen that the output of AND gate450of the last channel will be high—i.e., a logic “1”—only if the SYNC STAT signals406of every channel in the cascade are high. If any one of them is low—i.e., a logic “0”—the output of the of AND gate450of the last channel will be low. Thus, the output of AND gate450in the last channel—i.e., the output of synchronization cascade circuitry45—is an indicator of whether or not each channel has achieved synchronization. If each channel has achieved synchronization, then it is appropriate to begin reading the contents of the various FIFOs405.

This is accomplished by conducting the cascade output to repeater line410through synchronizer408and tristate buffer409. Each channel preferably has a synchronizer408and tristate buffer409because, as discussed below, any channel could be the last channel in the cascade, so that the output of that channel's AND gate450would be the cascade output. Therefore, the facility to conduct that output to repeater line410preferably should be available. For whichever channel is the last in the cascade—here, channel43—the cascade output is allowed to propagate onto repeater line410by enabling tristate buffer409of that channel at input411. In addition, the recovered clock signal401of that channel preferably is connected (see conductor segment412) to a clock tree419, preferably in the programmable logic core of PLD10. That clock401thus propagated preferably is used not only to read the various FIFOs405but also to clock the cascade output itself onto repeater line410through synchronizer408.

By this arrangement, the cascade output is conducted by repeater line410and connections413to the read enable inputs of each FIFO405in the cascade. While each FIFO405is clocked for writing by the respective recovered clock401of its respective channel, the clocks401of the different channels are asynchronous with one another. Therefore, for synchronous reading of FIFOs405, a single clock preferably is used. Specifically, clock401of the last channel in the cascade, conducted to FIFO405of each channel by clock tree419, preferably is used as the single FIFO read clock for all channels.

Repeaters414in repeater line410, and multiplexers415on the respective cascade input of each AND gate450, are provided to allow selection of which channels to group in the cascade. Because serial interface20is part of a PLD, the way in which it will be used may differ from one user configuration to another. In any particular configuration, only certain channels may be used. Repeaters414can be configured to pass or block signals in either direction, allowing propagation of the cascade output to be confined to the selected channels, even if the selected channels are a group of channels bounded on both sides by other channels that are not included in the group. Similarly, by configuring one of multiplexers415, using configuration bit418, to select its permanently high input416, rather than its cascade input417, all channels “above” the channel in which the multiplexer is so configured are effectively eliminated from the cascade, because their cascade output will be ignored.

As far as channels below the selected channels are concerned, the cascade output will continue to propagate to those additional channels. However, if those channels are not in use, it will not matter. And if those channels are in use, multiplexer415in the first of those channels can be configured to ignore the cascade output of the grouping above. Thus it is seen that any set of contiguous channels may be grouped in this way. Indeed, it will be appreciated that more than one set of contiguous channels may be grouped into separate groups if desired.

As seen inFIG. 5, instead of multiplexer415, an OR gate50may be used, with configuration bit418as one of its inputs51and the cascade output of the previous channel as its other input52.

Although the embodiment shown inFIGS. 4 and 5allows more than one set of channels to be grouped, any grouping must be of contiguous channels. A second preferred embodiment60shown inFIGS. 6 and 7allows noncontiguous channels to be grouped. In this embodiment, each AND gate450not only has multiplexer415on its cascade input, but also has multiplexer615on its channel input. Multiplexer615may be configured by configuration bit61to select either SYNC STAT signal406or a permanently high input616. Selection of input616would allow that particular channel to be ignored in the cascade, thereby excluding that channel from the group. Alternatively, multiplexer615may be replaced by an OR gate70having as inputs SYNC STAT signal406and configuration bit61(seeFIG. 7).

In addition, in embodiment60tristate buffers409, and repeater line410and repeaters414, preferably are replaced by a routing network62, in which conductors620are switchably connected to the read-enable inputs of FIFOs405and to the outputs of synchronizers408. This allows any synchronizer output to be connected to any FIFO read-enable input, thereby allowing noncontiguous groupings. Moreover, if multiple cascades (not shown) of AND gates450are provided, those multiple cascades, along with multiple conductors620, would allow multiple noncontiguous groupings. However, with only a single cascade, as shown, only one noncontiguous grouping is possible.

FIG. 8shows the effect of the transmitters of various quads in a serial interface being completely asynchronous. Waveforms80and81show, respectively, the high-speed serial clock from the transmit PLL of a first quad, and the slower parallel clock divided down from the serial clock. The first rising edge810of parallel clock81occurs substantially simultaneously with the first rising edge800of serial clock80after at least one of the transmitters in that first quad is ready to transmit. Transmission begins on rising edge810. Similarly, waveforms82and83show, respectively, the high-speed serial clock from the transmit PLL of a second quad, and the slower parallel clock divided down from the serial clock. The first rising edge830of parallel clock83occurs substantially simultaneously with the first rising edge820of serial clock82after at least one of the transmitters in that second quad is ready to transmit. As can be seen, because the transmitters of the two quads are completely asynchronous, the second quad does not begin transmitting until almost a full parallel period (unit interval) after the first quad begins transmitting. In applications where a plurality of quads are used together, this is not acceptable.

FIG. 9shows a first embodiment90of a transmitter portion of an interface according to the present invention in which several quads begin transmitting within one serial unit interval of each other. InFIG. 9, the central logic blocks91,92of two quads93,94are shown to contain, inter alia, a transmit PLL95that produces a high-speed serial clock950, and a clock divider96that produces from high-speed serial clock950a slower parallel clock960whose period, or unit interval, preferably is an integral multiple of the serial unit interval of clock950. In addition, each transmit PLL95has a reference input multiplexer951that selects an input reference952from elsewhere in PLD10, such as from the programmable logic core, or from an off-chip source953common to both (or all, in the case of more than two) quads. Where the two quads are being used independently, it is not important that the transmit PLL reference clocks be the same, but where they are to be used together, it is preferable, if the quads are to operate synchronously, that the reference clocks be the same.

Each transmit PLL95preferably has a lock output97that signals that the PLL95has locked. Preferably, the lock outputs97of all quads that are to be synchronized are input to an n-input AND gate98, where n is the number of quads. Preferably, AND gate98is configured from programmable logic in the programmable logic core of PLD10. The output of AND gate98preferably is used as a clock-enable signal99to enable all clock dividers96to provide each quad with its parallel transmit clock, which would allow each quad to begin transmitting.

Thus in operation, the transmit PLL95of each quad would begin operating when conditions in its quad called for such operation, and a lock signal97would be output from that quad to AND gate98. The PLL would continue to provide serial clock950, but the transmitters in that quad would not yet transmit for lack of parallel clock960. As PLL95of each quad became active, eventually, all inputs to AND gate98would be high, and clock-enable signal99would go high, enabling all clock dividers96to provide parallel clocks960so that all quads could begin transmitting.

In order for the respective quads to receive clock-enable signal99as simultaneously as possible, so that they begin transmitting as simultaneously as possible, registers900preferably are provided in the programmable logic core as close as possible physically to the respective quads and clock-enable signal99preferably is registered in registers900. Registers900preferably are clocked by a low-skew global clock901of PLD10so that all registers900are triggered to release clock-enable signal99to their respective clock dividers96substantially simultaneously. Care need only be taken that all of registers900are close enough to AND gate98that clock-enable signal99reaches all registers900within one period of clock901.

However, the various quads are still asynchronous, and the various PLLs95will begin providing serial clocks950asynchronously, although preferably with a common serial unit interval as a result of the common reference clock953. Therefore, although all clock dividers96preferably receive clock-enable signal99substantially simultaneously, each clock divider96will wait until the next rising edge of its respective serial clock950before beginning to provide its respective parallel clock960. Therefore, as illustrated inFIG. 10, the various quads may not begin transmitting perfectly simultaneously, but nevertheless will all begin transmitting within one serial unit interval.

FIG. 11shows an alternative embodiment110of a transmitter portion of an interface according to the present invention in which several quads begin transmitting within one serial unit interval of each other. Embodiment110is substantially similar to embodiment90, except that instead of delivering clock-enable signal99to clock dividers96by clocking registers900with a low-skew clock signal, clock-enable signal99is delivered through a balanced clock tree111spanning all involved quads, which may be similar to the multiple interface reference clock distribution network described in copending, commonly-assigned U.S. patent application Ser. No. 10/455,773, filed Jun. 4, 2003, which is hereby incorporated by reference in its entirety.

As described, the various quads arranged in accordance with the present invention will begin transmitting within 1 serial unit interface (1 UI). In addition, there may be up to 0.03 UI of skew introduced by each of (1) PLD clock tree skew giving rise to skew in communicating clock-enable signal99to PLLs95, (2) skew in the transmit serializers, and (3) output buffer delay variation between channels, for total additional skew of up to 0.9 UI, or total skew of up to 1.9 UI. Relevant standards tolerate up to 2 UI of total skew.

A PLD10incorporating interfaces20according to the present invention may be used in many kinds of electronic devices. One possible use is in a data processing system120shown inFIG. 12. Data processing system120may include one or more of the following components: a processor121; memory122; I/O circuitry123; and peripheral devices1244. These components are coupled together by a system bus125and are populated on a circuit board126which is contained in an end-user system127.

System120can be used in a wide variety of applications, such as computer networking, data networking, instrumentation, video processing, digital signal processing, or any other application where the advantage of using programmable or reprogrammable logic is desirable. PLD10can be used to perform a variety of different logic functions. For example, PLD10can be configured as a processor or controller that works in cooperation with processor121. PLD10may also be used as an arbiter for arbitrating access to a shared resources in system120. In yet another example, PLD10can be configured as an interface between processor121and one of the other components in system120. It should be noted that system120is only exemplary, and that the true scope and spirit of the invention should be indicated by the following claims.

Various technologies can be used to implement PLDs10as described above and incorporating this invention.

It will be understood that the foregoing is only illustrative of the principles of the invention, and that various modifications can be made by those skilled in the art without departing from the scope and spirit of the invention, and the present invention is limited only by the claims that follow.