Secondary phase compensation assist for PLL IO delay

A line card of a network box receives a SYNC input signal and generates a first time stamp based on receipt of the SYNC input signal. The line card generates a system clock signal in a phase-locked loop and generates a SYNC output signal by dividing the system clock signal in a divider circuit. The SYNC output signal is fed back to an input terminal as a SYNC feedback signal. A time stamp is generated based on receipt of the SYNC feedback signal. The line card determines a time between the SYNC input signal and the SYNC feedback signal based on the first time stamp and the second time stamp. The timing of the SYNC output signal is adjusted based on the time difference using a coarse time adjustment by adjusting a divide ratio of the divider circuit and using a fine time adjustment in the phase-locked loop based on a residue of a remainder of the time difference not accounted for by the coarse time adjustment.

CROSS-REFERENCE TO RELATED APPLICATION(S)

This application relates to the application entitled “Secondary Phase Compensation Assist for PLL IO Delay Aligning SYNC Signal to System Clock Signal”, naming Vivek Sarda as inventor, patent application Ser. No. 16/836,713, filed Mar. 31, 2020, which application is incorporated herein by reference in its entirety.

BACKGROUND

Field of the Invention

This invention relates to network timing and more particularly to reducing errors in network timing signals.

Description of the Related Art

Network communication boxes use timing protocols to ensure time of day (ToD) counters in the network are synchronized. SYNC signals are used to update time of day counters at the same time in the network. Any delay/offset and process, voltage, temperature (PVT) variation between the SYNC lines being supplied to the ToD counters in each line card in the network box results in an error that is classified as Continuous Time Error (CTE). The CTE budget for a network box is 5 ns for Class D type network boxes. Reducing sources of timing error in network boxes would give greater flexibility to designers to meet the CTE budget.

SUMMARY OF EMBODIMENTS OF THE INVENTION

Accordingly, in one embodiment a method includes receiving a SYNC input signal and generating a SYNC output signal, the SYNC output signal indicating when to update a time of day counter. The method further includes feeding back the SYNC output signal to an input terminal as a SYNC feedback signal and determining a time difference between the SYNC input signal and the SYNC feedback signal. A timing of the SYNC output signal is adjusted based on the time difference.

In another embodiment an apparatus includes a first input terminal to receive an input SYNC signal. A phase-locked loop generates a system clock signal. A divider circuit divides the system clock signal and generates a SYNC output signal. A time of day counter is coupled to the SYNC output signal and updates a count value responsive to the SYNC output signal. A second input terminal is coupled to receive the SYNC output signal as a feedback SYNC signal. Arithmetic logic determines a time difference between the SYNC input signal and the SYNC output signal and control logic adjusts a timing of the SYNC output signal based on the time difference.

In another embodiment a method includes receiving a SYNC input signal and generating a first time stamp based on receipt of the SYNC input signal. The method further includes generating a local system clock signal in a phase-locked loop and generating a SYNC output signal by dividing the local system clock signal in a divider circuit. The SYNC output signal is fed back to an input terminal as a SYNC feedback signal. A time stamp is generated based on receipt of the SYNC feedback signal. A time difference is determined between the SYNC input signal and the SYNC feedback signal based on the first time stamp and the second time stamp. The timing of the SYNC output signal is adjusted based on the time difference using a coarse time adjustment by adjusting a divide ratio of the divider circuit and using a fine time adjustment using the phase-locked loop based on a residue of a remainder of the time difference not accounted for by the coarse time adjustment.

DETAILED DESCRIPTION

FIG. 1shows a typical architecture of communication network box100with a slave line card (LC)101, a master timing card (TC)103, and multiple master line cards105. The data_out109from each line card is time stamped using time stamps from local Time of Day (ToD) counters111. One challenge is to keep the ToD on the slave line card in alignment with the network timestamps from the incoming data stream on data_in116. Another challenge is to maintain the ToD counters across different line cards in alignment over process, voltage, and temperature (PVT) variations so that all data_out109with their time stamps are aligned with each other and the incoming network time supplied on data_in116.

The master timing card103supplies a SYNC signal and system clock signal (SYSCLK) to the slave line card101generated using PLL117and dividers (not shown). The SYNC signal is also referred to as the FSYNC (frame sync) signal in certain contexts as the signal has different names (SYNC or F SYNC) at the system level or integrated circuit level inside the network box. The signal will be referred to as the SYNC signal herein for ease of reference. The master timing card103supplies the SYSCLK and SYNC signal to all of the master line cards105over backplane119. The SYNC signal is a global signal inside the network system box100that signifies the right moment/edge for the Time of Day (ToD) counters111to rollover. The SYNC signal has a frequency range of 1 kHz to pp2s (pulse per 2 seconds). In many network systems the SYNC signal is 1 pulse per second (1PPS). SYNC is an integer divided down and edge aligned version of the system clock signal SYSCLK. The SYNC output from the master timing card (TC) is the global SYNC used by all the line cards (LC) for their ToD rollover alignment. The various ToD counters111contain the same value and turnover at the same time based on the SYNC signal. Each of the line cards101and105generate the SYNC signal by dividing the SYSCLK generated by PLL121in a divider (not shown inFIG. 1) to the desired frequency.

The exact position of the SYNC edge is derived using a precision time protocol (PTP) servo loop that uses the time information inside the incoming Synchronous Ethernet (SyncE) packet stream to the slave line card101.FIG. 2illustrates an example of a time stamp exchange201between an upstream PHY and the downstream pHY (e.g. PHY123inFIG. 1). Each of the time stamps t1-t4represents the departure time (t1, t3) or the receive time (t2, t4). The timestamps exchange allows determination of one-way delay (OWD) and error offset between the upstream PHY and the downstream PHY shown at203. That time stamp exchange allows the slave line card to determine the correct time provided by the upstream PHY even with delays between the upstream PHY and the downstream PHY. Note that the high level description of the PTP servo loop is provided as background information to provide context in which various embodiments described herein can be utilized.

The slave line card and the master timing card also have a closed loop PTP servo system in accordance with the IEEE 1588 protocol that corrects the position of the SYNC signal over process, voltage, and temperature (PVT) and aligns the SYNC signals distributed by the master timing card203to the time stamps of the incoming packet stream to the slave line card. The servo loop ensures that the slave line card and the master timing card are synchronized. The slave line card101and the master timing card103exchange information in the closed loop system to adjust the CLK and SYNC pair on the master timing card such that the slave line card ToD is aligned with the network ToD of the chosen incoming data stream on data_in116. The PTP servo loop adjusts the timing of SYNC by adjusting PLL117so that the slave line card ToD is aligned in frequency and phase to the upstream ToD received by the slave line card on data_in116. The distributed SYSCLK is supplied as a reference clock to the PLL121within each of the line cards and the line card PLLs generate a local SYSCLK and SYNC signal that is phase and frequency aligned with the distributed SYSCLK and SYNC signal. The master line cards105are duplicates (up to 64 copies) of the slave line card101but without the closed loop PTP servo loop. In other words, the distribution of the CLK/SYNC pair to the master line cards105is open loop (without the PTP closed loop adjustments).

Referring toFIG. 3additional aspects of the slave line card301and the master time card303are shown. In addition, to generating the SYSCLK305, the master time card generates a Synchronous Ethernet (SyncE) clock signal307. The SyncE clock signal is supplied to SyncE PLL309in the slave line cards so that the local SyncE clock signals are frequency and phase locked to the SyncE clock signal in the master timing card. Embodiments include a slave timing card311that functions as a backup timing card to the master timing card303by providing backup SYNC, SYSCLK, and SyncE signals. The FPGA315is part of the PTP loop and includes an MCU to implement PTP software. PM and SEC are primary and secondary data streams to select from for determining the network time. SEC is a backup of PRI. The PTP PLLs in the slave line card and the master timing card are used to adjust the SYSCLK (and SYNC) based on the PTP servo loop. The various time stamps required for the PTP servo loop are exchanged between the hosts on the slave line card and the master timing card. The hosts are microcontroller units (MCUs) or field programmable gate arrays (FPGAs) with some processing and communication abilities. The digitally controlled oscillator (DCO) in the master timing card303adjusts the phase of the SYNC and SYSCLK in accordance with the calculations of the PTP servo loop so that the SYNC at the ToD counter on the slave line card occurs at the desired time.

Any delay/offset and PVT variation between the SYNC lines to the ToD's in each line card and the slave timing card in the network box results in an error that is classified as Continuous Time Error (CTE). The CTE budget for a network box is 5 ns for Class D type network boxes. One source of error is that the SYNC signal and SYSCLK supplied by the local PLLs in the line cards still have to transit through the circuitry of the PLL to the ToD counters. A mismatch exists in SYNC signal delivery due to PVT differences between the line cards including the slave line card and the master line cards. That mismatch impacts the accuracy of the timestamps in every Master LC and impacts the Continuous Time Error (CTE) budget of 5 ns for a Class D network box.

FIGS. 4A and 4Billustrate the timing relationships that can exist between the SYNC signal and the SYSCLK signal.FIG. 4Ashows a master timing card401supplying a line card403with the SYSCLK signal405and the SYNC signal407through backplane drivers, receivers and PCB traces409. At A, SYNC identifies the SYSCLK period that coincides with ToD rollover. In the example ofFIG. 4B, that period is period 0 with period (−1) and period (+1) before and after period 0. In an embodiment SYNC can be adjusted to any SYSCLK period by aligning the rising edges of SYSCLK and SYNC. In the example ofFIG. 4BSYNC selects the SYSCLK period 0. The Δt boxes shown inFIG. 4Arepresent the adjustments made to account for the delay in the SYNC and SYSCLK signals supplied from the master timing card401. Note that since SYSCLK may be frequency multiplied at the line card output as shown inFIG. 4A(N×SYSCLK), setting this lower frequency SYSCLK period can be considered a coarse adjustment. Backplane drivers, receivers and PCB traces404cause a delay in the SYSCLK period 0 rising edge at C with respect to the alignment reference line421. In addition, there is a mismatch between SYNC and SYSCLK shown at423. At D, input delay adjustments in the line card realign the SYSCLK edges back to the original alignment reference line421but do not realign SYSCLK and SYNC. Even if the SYSCLK and SYNC edges remain misaligned as shown at D, the SYNC signal in the indicated range425will select the SYSCLK period 0. Input to output delay in the illustrated embodiment, causes misalignment of the SYSCLK rising edge with respect to the alignment reference line421. Referring now to E ofFIGS. 4A and 4B, if the misalignment is left without correction (open loop), the misalignment can range between ±0.5 ns around the alignment reference line as shown at425. Alternatively, a zero delay mode with respect to the SYSCLK can achieve ±100 ps as shown at427. As shown at429, the SYNC signal is realigned to the rising edge of SYSCLK period 0.FIG. 4Cshows a larger view of E ofFIG. 4B. The SYNC signal can also be adjusted to any edge of N×SYSCLK (controlled at the line card). That is considered to be a fine adjustment. While open loop and zero delay options provide solutions, if the IO delay is measured and the adjustment is made to SYNC based on the IO delay, the error can be reduced to ±50 ps.

Accordingly, referring toFIG. 5, in order to measure the IO delay, the SYNC output signal (SYNC_OUT) is looped back to the slave line card input buffer501as the SYNC feedback signal (SYNC_FB)503. That allows the IO delay to be tracked and accounted for over PVT variations. The SYNC_FB signal is time stamped and that time stamp is compared to the current valid SYNC time stamp for the SYNC input signal507received on input buffer509. The measured IO delay can include delay caused by input buffers, the PLL511, divide logic517, and other clock tree buffers on the line card. While the slave line card is shown, the same approach is used for the master line cards, each of which feedback their SYNC signal to a master line card input buffer.

The SYNC signal507is received at buffer509and is used, along with SYSCLK515, to adjust the PLL511to ensure the SYNC_OUT and local SYSCLK521generated in the line card is phase and frequency aligned with the input SYNC and SYSCLK from the master timing card. The SYNC_OUT signal is generated by an integer divide in divider517of the local SYSCLK521generated by the PLL511that is aligned in phase and frequency to SYSCLK.

FIG. 6illustrates logic to time stamp and compare the two time stamps. Receive buffer501receives the SYNC_FB signal503and supplies the SYNC_FB signal to time stamp logic601. Time stamp logic603receives the input SYNC signal507through buffer509. The time stamp logic functions as a time to digital converter and converts the transitions of the SYNC_FB signal and the input SYNC signal to digital values based on an available timing reference. Difference logic605receives the two time stamps and determines the difference between the time stamps of SYNC_FB503and the currently valid SYNC signal507. That difference611represents the input/output (TO) delay. The difference logic may be part of the SYNC control block701shown inFIG. 7.

Referring toFIG. 7, in an embodiment the SYNC control block701receives the time stamps, determines the IO delay, and controls both a coarse adjust703and a fine adjust705to remove the IO delay from the SYNC output signal to better align the SYNC output signal with the SYNC input signal. The functionality of the control block may be implemented as a state machine and other discrete control logic, in a programmed microcontroller, FPGA, or in a suitable combination of a programmed microcontroller, FPGA, and/or discrete control logic. Referring toFIG. 8, the coarse adjustment is implemented by adjusting the divider517. For example, if divider517divides by an integer N without IO correction, with IO correction divider517divides by N±M, where M is an integer corresponding to at least a portion of the IO delay as explained further herein. The fine adjustment is made by adjusting an offset to the phase and frequency detector (PFD)901or the feedback divider905of the PTP PLL511shown inFIG. 9in more detail. The combination of the coarse adjustment and the fine adjustment substantially eliminates the measured IO delay associated with the SYNC_out signal (or applies any desired offset to the SYNC_out signal). Still referring toFIG. 9, PLL511receives SYSCLK515as the reference clock signal. Once PLL511is locked to SYSCLK515, divider517generates the SYNC_OUT signal that is looped back as SYNC_FB. In an embodiment, the PTP PLL511has a loop bandwidth of between, e.g., 40 Hz and 100 Hz.

Referring back toFIG. 4A, conceptually, the input SYNC signal can be used to align the output SYNC signal by using the input SYNC signal407to reset the divider415. In that way, SYNC_OUT will then be aligned with the input SYNC signal.

Another approach utilizes the time stamps as discussed herein. Referring toFIG. 10, a flow diagram illustrates the operation of the SYNC control logic701. In1001the control logic determines the IO delay by comparing the time stamps for SYNC_FB503and SYNC507. The SYNC logic701calculates the quantized SYSCLK cycles of IO delay correction and applies that quantized value to the output divider517in the SYNC_OUT path. For example, in an embodiment the OSC903is a voltage controlled oscillator (VCO) providing a VCO output signal with a frequency of 10 GHz. With counter517being clocked by the VCO output signal, that results in the counter counting in 100 picosecond (ps) increments. Assume that the IO delay was measured to be 35.033 ns. The SYNC logic quantizes the IO delay in terms of a number of divider (or counter) cycles to add or subtract and applies the divider correction in1003. The SYNC control logic701then applies the residue left from the coarse IO delay correction to make the fine adjustment in1005using the PLL. For example, assume the divider is implemented as a counter that counts N SYSCLK cycles and then issues a pulse indicating the N count has been reached. The counter then resets and counts again, thus issuing a pulse every N cycles of SYSCLK. The 35 ns can be accounted for using the coarse correction by causing the counter to count 35 ns worth of fewer increments (in counter increments of 100 ps each) to zero out the IO delay (except for the residue). Note that the change in count is made for only one SYNC_OUT cycle. Adjusting the count value of the counter is one way of adjusting the divider value of the divider. While the example shown inFIG. 9utilizes a high speed counter clocked by the VCO, other embodiments utilize a counter clocked by a lower frequency clock signal. Thus, the SYNC control logic may quantize the IO error for one or more lower speed counters (or both low speed and high speed counters) before determining the residue. In the example being considered, the 0.033 ns (33 ps) cannot be accounted for by the coarse adjust. Referring again toFIG. 9, the PLL can be adjusted by adjusting the feedback divider905. In other embodiments in which the PFD901receives digital values, the fine adjustment can be made by adjusting the digital time stamp907of the feedback clock being supplied to the PFD901or even the digital time stamp909of the reference clock signal. The SYNC control logic applies the residue correction slowly, e.g., at less than ⅛ of the loop bandwidth of the PLL so as not to cause an undesirable large perturbation to the VCO output signal. The measured IO delay between SYNC_IN and SYNC_OUT is thereby reduced to zero (or other desired offset). Finally, in1007, the SYNC control logic recalculates the actual IO delay every S cycles of SYNC, where S is an integer, and makes the coarse and fine adjustments to zero out any detected IO delay. Note that embodiments compare the measured IO delay to a threshold amount of IO delay before making the coarse and fine adjustments so that the adjustments are not being continually made in response to random phase noise.

While the embodiment ofFIG. 10, time aligns SYNC_OUT with SYNC, other embodiments align SYNC_OUT directly with SYSCLK. Referring toFIG. 11, the timing diagram illustrates how SYSCLK and SYNC may be misaligned. Ideally, the SYNC input signal is aligned at1101with SYSCLK. However, due to variations in delays associated with transmit and receive buffers and the backplane traces, the actual SYNC input signal received at the line card may be misaligned with respect the SYSCLK as shown at1103. The SYNC_FB IO delay with respect to the actual SYNC input signal is shown at1105. However, rather calculating the IO delay with respect to the time stamped actual SYNC input signal and realigning to the actual SYNC input signal, embodiments realign the SYNC output signal to SYSCLK. Thus, the correction is made to realign SYNC_OUT (SYNC_FB) to SYSCLK instead of the misaligned actual SYNC input signal by calculating the IO delay1107between SYNC_FB and SYSCLK rather than the delay1105between SYNC_FB and SYNC actual. The active edges (e.g., the rising edges) of SYSCLK may be time stamped by the time stamper925(seeFIG. 9) or another time stamper and SYNC and SYNC_FB time stamped as described inFIG. 6.

Referring toFIG. 12A, the functionality of the SYNC control block1201is modified from the SYNC control block701(FIG. 7) to align the SYNC output signal to the input SYSCLK signal rather than the SYNC input signal. The modified control block receives the time stamps1202of SYSCLK, SYNC_FB and the SYNC input signal, determines the closest SYSCLK edge to the SYNC input signal time stamp, determines the IO delay to align the SYNC output signal to the closest input SYSCLK edge, and controls both a coarse adjust1203and a fine adjust705to remove the IO delay from the SYNC output signal to better align the SYNC output signal with the SYNC input signal.

FIG. 12Billustrates the control flow to realign to SYSCLK. The control functionality described inFIGS. 12A and 12Bmay be implemented in a state machine and/or other discrete control logic, in a programmed microcontroller or FPGA, or in a suitable combination of a programmed microcontroller, FPGA, and/or discrete control logic. The control logic in1221receives the time stamp for the SYNC input signal (for the valid SYNC signal) corresponding to the SYNC_FB signal, the time stamp for SYNC_FB, and time stamps of the SYSCLK edges. Then, the control logic determines in1223the closest SYSCLK edge to the SYNC input signal time stamp. In the example ofFIG. 11, the closest edge is edge1111. The closest edge can be determined from SYSCLK time stamps and the SYNC input signal time stamp. A suitable number of SYSCLK time stamps are kept to ensure the closest SYSCLK edge is available. The closest SYSCLK edge is the smallest difference between the time stamps of the SYNC input signal and SYSCLK. Then, the IO delay is determined in1225as the difference between the SYSCLK time stamp (closest edge) and SYNC_FB (equivalent to SYNC_OUT). The IO correction is quantized in terms of the divider increments in1225and the quantized value applied in1227in a coarse correction by adjusting the divider supplying the SYNC_OUT signal. The residue after quantization is applied in1229using a fine correction as described earlier. In1231, the control logic waits for N cycles and then recalculates the IO delay to track changes in the IO delay due to temperature, voltage, or other environmental changes.

In at least one embodiment the IO delay realignment is programmable to be either to the SYNC input signal or to the SYSCLK and thus the SYNC control logic includes the functionality to determine the IO error with respect to both input signals. The programmability may be implemented over a programming interface (not shown) of the integrated circuit implementing the realignment.

The description of the invention set forth herein is illustrative and is not intended to limit the scope of the invention as set forth in the following claims. Variations and modifications of the embodiments disclosed herein may be made based on the description set forth herein, without departing from the scope of the invention as set forth in the following claims.