Patent Publication Number: US-2022216981-A1

Title: Phase transport with frequency translation without a pll

Description:
CROSS-REFERENCE TO RELATED APPLICATION(S) 
     This application is a continuation of U.S. patent application Ser. No. 16/849,036, filed Apr. 15, 2020, entitled “Phase Transport with Frequency Translation Without a PLL”, naming Vivek Sarda as inventor, which application is incorporated herein by reference. 
     This application relates to the application entitled “Secondary Phase Compensation Assist for PLL IO Delay”, naming Vivek Sarda as inventor, patent application number 16/836,706, filed Mar. 31, 2020, which application is incorporated herein by reference in its entirety. 
    
    
     BACKGROUND 
     Field of the Invention 
     This invention relates to network timing and more particularly to transporting and generating clock signals in networks. 
     Description of the Related Art 
     Network communication boxes use timing protocols to ensure that time of day (ToD) counters in the network are synchronized. The synchronization is achieved using SYNC signals to update time of day counters at the same time in the network. Traditionally, a timing card distributes a Synchronous Ethernet (SyncE) clock signal, a SYSCLK clock signal, and a SYNC signal over a backplane to the slave line card and the various master line cards in the network box. 
     SUMMARY OF EMBODIMENTS OF THE INVENTION 
     Embodiments described herein eliminate the need to distribute the SYSCLK clock signal from the master timing card to all of the line cards over the backplane of the network box. Instead, the SYSCLK is generated using a digitally controlled oscillator that receives a timing signal from the SyncE phase-locked loop on the line card and a control signal from control logic on the line card. 
     In an embodiment, a method includes receiving an input clock signal at a line card and generating a first output clock signal using a phase-locked loop and supplying a clock signal from the phase-locked loop to a digitally controlled oscillator. The method further includes receiving a SYNC input signal at the line card and generating a control signal for the digitally controlled oscillator, based in part on the SYNC input signal. The method further includes generating a second output clock signal using the digitally controlled oscillator and dividing the second output clock signal to generate a SYNC output signal. A time of day counter is updated using the SYNC output signal and the second output clock signal. 
     In another embodiment an apparatus includes an input terminal to receive a SYNC input signal. A phase-locked loop is coupled to receive an input clock signal and to generate a first output clock signal. A digitally controlled oscillator is coupled to the phase-locked loop and the digitally controlled oscillator supplies a second output clock signal. A divider circuit divides the second output clock signal to generate a SYNC output signal. A time of day counter is coupled to the SYNC output signal and to the second output clock signal, and updates a time of day count value in synchronism with the SYNC output signal. Compare logic determines a time difference between a SYNC feedback signal and the SYNC input signal and control logic adjusts a timing of the second output clock signal based on the time difference. 
     In another embodiment, a line card includes an input terminal to receive a SYNC input signal and a phase-locked loop coupled to receive an input clock signal and to generate a first output clock signal. A digitally controlled oscillator is coupled to the phase-locked loop, the digitally controlled oscillator to supply a second output clock signal. A divider circuit divides the second output clock signal to generate a SYNC output signal. A time of day counter is coupled to the second output clock signal and is responsive to update a time of day count value in synchronism with the SYNC output signal. Compare logic compares the SYNC input signal and a SYNC feedback signal and provides a time difference and control logic adjusts a timing of the second output clock signal based, at least in part on the time difference. A timing card supplies the input clock signal and the SYNC input signal and a backplane is coupled to the line card and the timing card. The input clock signal and the SYNC input signal are transmitted through the backplane from the timing card to the line card. 
    
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
       The present invention may be better understood, and its numerous objects, features, and advantages made apparent to those skilled in the art by referencing the accompanying drawings. 
         FIG. 1  illustrates a typical communication network box with a slave line card (LC), a master timing card (TC), and multiple master line cards coupled through a backplane. 
         FIG. 2  illustrates an example of a time stamp exchange. 
         FIG. 3  illustrates additional details of a slave line card and a master timing card. 
         FIG. 4  illustrates how SYSCLK and SYNC can be misaligned on receipt at the line card. 
         FIG. 5  illustrates a high level block diagram of a network box that distributes the SyncE clock signal and the SYNC signal over the backplane (without a SYSCLK) and regenerates the SYSCLK clock signal on the line card using a digitally controlled oscillator (DCO). 
         FIG. 6  illustrates a high level block diagram of an interpolative divider that is used as the DCO in one or more embodiments. 
         FIG. 7  illustrates a high level block diagram of a line card PLL used to generate the SyncE clock signal. 
         FIG. 8  illustrates logic to compare the SYNC signal and the SYNC_FB signal. 
         FIG. 9  illustrates a high level block diagram of control functionality to control the interpolative divider to generate the SYSCLK clock signal. 
         FIG. 10  illustrates a network box that includes a slave line card that generates the SYSCLK signal utilizing a digitally controlled oscillator, a master timing card, and backup timing card. 
         FIG. 11  illustrates an embodiment of a network box showing that the master timing card distributes the SYNC signal and the SyncE clock signal to the slave line card and to master line cards (but does not distribute the SYSCLK signal over the backplane). 
         FIG. 12  illustrates an embodiment of a master timing card with a PLL to generate SyncE and a digitally controlled oscillator to generate SYSCLK. 
     
    
    
     The use of the same reference symbols in different drawings indicates similar or identical items. 
     DETAILED DESCRIPTION 
       FIG. 1  shows a typical architecture of communication network box  100  with a slave line card (LC)  101 , a master timing card (TC)  103 , and multiple master line cards  105 . The data_out  109  from each line card is time stamped using time stamps from local Time of Day (ToD) counters  111 . One challenge is to keep the ToD on the slave line card in alignment with the network timestamps from the incoming data stream on data_in  116 . Another challenge is to maintain the ToD counters across different line cards in alignment over process, voltage, and temperature (PVT) variations so that all data_out  109  with their time stamps are aligned with each other and the incoming network time supplied on data_in  116 . 
     The master timing card  103  supplies a SYNC signal and system clock signal (SYSCLK) to the slave line card  101  generated using PLL  117  and dividers (not shown). 
     The SYNC signal is also referred to as the FSYNC (frame sync) signal in certain contexts as the signal has different names (SYNC or FSYNC) at the system level or integrated circuit level inside the network box. The signal will be referred to as the SYNC signal herein for ease of reference. The master timing card  103  also supplies the SYSCLK and SYNC signal to all of the master line cards  105  over backplane  119 . The SYNC signal is a global signal inside the network system box  100  that signifies the right moment/edge for the Time of Day (ToD) counters  111  to rollover. The SYNC signal has a frequency range of 1 kHz to pp 2   s  (pulse per 2 seconds). In many network systems the SYNC signal is 1 pulse per second (1PPS). SYNC is an integer divided down and edge aligned version of the system clock signal SYSCLK. The SYNC output from the master timing card (TC) is the global SYNC used by all the line cards (LC) for their ToD rollover alignment. The various ToD counters  111  contain the same value and turnover at the same time based on the SYNC signal. Each of the line cards  101  and  105  generate the SYNC signal by dividing the SYSCLK generated by PLL  121  in a divider (not shown in  FIG. 1 ) to the desired frequency. 
     The exact position of the SYNC edge is derived using a precision time protocol (PTP) servo loop that uses the time information inside the incoming Synchronous Ethernet (SyncE) packet stream to the slave line card  101 .  FIG. 2  illustrates an example of a time stamp exchange  201  between an upstream PHY and the downstream PHY (e.g. PHY  123  in  FIG. 1 ). Each of the time stamps t 1 -t 4  represents the departure time (tl, t 3 ) or the receive time (t 2 , t 4 ). The timestamps exchange allows determination of one-way delay (OWD) and error offset between the upstream PHY and the downstream PHY shown at  203 . That time stamp exchange allows the slave line card to determine the correct time provided by the upstream PHY even with delays between the upstream PHY and the downstream PHY. Note that the high level description of the PTP servo loop is provided as background information to provide context in which various embodiments described herein can be utilized. 
     The slave line card and the master timing card also have a closed loop PTP servo system in accordance with the IEEE 1588 protocol that corrects the position of the SYNC signal over process, voltage, and temperature (PVT) and aligns the SYNC signals distributed by the master timing card  203  to the time stamps of the incoming packet stream to the slave line card. The servo loop ensures that the slave line card and the master timing card are synchronized. The slave line card  101  and the master timing card  103  exchange information in the closed loop system to adjust the CLK and SYNC pair on the master timing card such that the slave line card ToD is aligned with the network ToD of the chosen incoming data stream on data_in  116 . The PTP servo loop adjusts the timing of SYNC by adjusting PLL  117  so that the slave line card ToD is aligned in frequency and phase to the upstream ToD received by the slave line card on data_in  116 . The distributed SYSCLK is supplied as a reference clock to the PLL  121  within each of the line cards and the line card PLLs generate a local SYSCLK and SYNC signal that is phase and frequency aligned with the distributed SYSCLK and SYNC signal. The master line cards  105  are duplicates (up to 64 copies) of the slave line card  101  but without the closed loop PTP servo loop. In other words, the distribution of the CLK/SYNC pair to the master line cards  105  is open loop (without the PTP closed loop adjustments). 
     Referring to  FIG. 3  additional aspects of the slave line card  301  and the master time card  303  are shown. In addition, to generating the SYSCLK  305 , the master time card generates a Synchronous Ethernet (SyncE) clock signal  307 . The SyncE clock signal is supplied to SyncE PLL  309  in the slave line cards so that the local SyncE clock signals are frequency and phase locked to the SyncE clock signal in the master timing card. Embodiments include a slave timing card  311  that functions as a backup timing card to the master timing card  303  by providing backup SYNC, SYSCLK, and SyncE signals. The FPGA  315  is part of the PTP loop and in an embodiment includes an MCU to implement PTP software. PRI and SEC are primary and secondary data streams to select from for determining the network time. SEC is a backup of PRI. The master timing card  303  receives input SyncE clock signals from the PHYs of line card  301 . The host MCU picks two PHYs and designates one of them as PRI (primary) and another as SEC (secondary) clocks that are supplied to the master timing card and also to the slave backup timing card  311 . SEC is backup for PM. The SyncE clocks are from two different sources but only one is used at any given time (as shown in  FIG. 5 ). Switching from PRI to SEC or vice versa is managed by hitless switching inside the PTP and SyncE PLLs. 
     The PTP PLLs in the slave line card and the master timing card are used to adjust the SYSCLK (and SYNC) based on the PTP servo loop. The various time stamps required for the PTP servo loop are exchanged between the hosts on the slave line card and the master timing card. The hosts are microcontroller units (MCUs) or field programmable gate arrays (FPGAs) with some processing and communication abilities. The digitally controlled oscillator (DCO) in the master timing card  303  adjusts the phase of the SYNC and SYSCLK in accordance with the calculations of the PTP servo loop so that the SYNC at the ToD counter on the slave line card occurs at the desired time. 
     Referring to  FIG. 4 , the timing diagram illustrates that the SYSCLK input signal and SYNC input signal received at the line card may be misaligned. Ideally, the SYNC input signal is aligned at  401  with the SYSCLK input signal. However, due to variations in delays associated with transmit and receive buffers and the backplane traces, the actual SYNC input signal received at the line card may be misaligned with respect to the SYSCLK as shown at  403 . 
       FIG. 5  illustrates one approach to avoid misalignment between the SYSCLK signal and the SYNC signal received at the line card from the timing card. The embodiment has the advantage of fewer routings through the backplane and fewer pins needed on the timing cards and the line cards. The embodiment of the network box  500  illustrated in  FIG. 5  avoids distributing the SYSCLK signal from the master timing card  501  to the line card (slave or master)  503 . Instead, the master timing card supplies the SYNC signal  505  and the SyncE clock signal  507  through the backplane  509  and rebuilds the SYSCLK signal with the correct frequency and phase based on the SYNC signal. In an embodiment master timing card  501  generates the SYNC signal  505  using the PTP loop described earlier to ensure the SYNC signal is synchronized with the upstream SYNC signal. The line card includes a PLL  521  that receives the SyncE clock signal from the master timing card  501  supplied over the backplane as a reference clock signal and generates a SyncE output clock signal  523 . In addition, the PLL  521  supplies a clock signal  525  to digitally controlled oscillator  527 . The digitally controlled oscillator  527  generates the output SYSCLK clock signal  529  with the appropriate phase and frequency. A digitally controlled oscillator supplies an output signal with a phase and frequency determined according to a digital control signal. Divider  531  divides the SYSCLK clock signal to generate the SYNC output signal  533 . The ToD counter  535  receives the SYSCLK and SYNC signals and updates synchronously with the other ToD counters in the network box. The ToD counter  535  is clocked by SYSCLK. The ToD counter increments a fixed amount between two SYNC signals with the fixed amount depending on the frequency relationship between the SYNC and SYSCLK signals. The various ToD counters in the line cards in the network box roll over, e.g., in response to a 1 PPS SYNC signal, in synchronism with respective SYNC signals in the line cards. The ToD counters provides the timestamp (also known as wall time in the line card) for the line cards. The control logic  537  determines the appropriate control signals for the digitally controlled oscillator  527  to generate the SYSCLK signal  529  with the right frequency and phase. Using a digitally controlled oscillator to generate SYSCLK has the added advantage, depending on the implementation, of eliminating a PLL from the line card. 
     In at least one embodiment as illustrated in  FIG. 6 , the DCO  527  is implemented as interpolative divider  600 . The interpolative divider  600  receives a high frequency clock signal  525  from the voltage controlled oscillator (VCO) of the SyncE PLL  521 . The interpolative divider  600  divides the VCO signal  525  to a desired frequency and phase as described below. In an embodiment, the VCO signal is 10 GHz but higher or lower VCO frequencies are used in other embodiments. Digital control circuit  602  generates digital control signals for circuit  604  of interpolative divider  600  based on the control signal  605  from control logic  537  that indicates a divide value that is typically a non integer value N.f, where N is an integer and f is a fractional portion of the divide value. The circuit  604  includes multi-modulus divider  606  and phase interpolator  608 . Digital control circuit  302  generates sequences of corresponding DIVCODE and PICODE control codes that control multi-modulus divider  606  and phase interpolator  608 , respectively. In an embodiment, digital control circuit  602  includes a first order delta signal modulator that generates a stream of integers that approximate the desired frequency of SYSCLK. Thus, digital control circuit  602  provides the integer portion of the divide value to generate SYSCLK to multi-modulus divider  606  and supplies the digital quantization error as control code PICODE to phase interpolator  608 . In an embodiment, multi-modulus divider  606  is an integer frequency divider that counts down an integer number of corresponding edges of input clock signal  525 , as indicated by digital control code DIVCODE, before generating a corresponding output edge of frequency-divided clock signal FDIVCLK  610 . Phase interpolator  608  interpolates between frequency-divided clock signal FDIVCLK  610  and one or more delayed versions of frequency-divided signal FDIVCLK (e.g., one or more equally spaced phases of frequency-divided clock signal FDIVCLK, equally spaced by an entire period of input clock CLKVCO) based on control code PICODE, which corresponds to the phase error, using techniques that are well known in the art. 
     In an embodiment, phase interpolator  608  generates multiple equally spaced phases of frequency-divided clock signal FDIVCLK and interpolates appropriate ones of those phases to generate the output clock signal SYSCLK. Interpolation techniques are well known in the art. Other interpolator implementations may be used based on such factors as the accuracy required, power considerations, design complexity, chip area available, and the number of bits used to represent the digital quantization error. 
     In at least one embodiment, phase interpolator  608  delays frequency-divided clock signal FDIVCLK by selecting from 256 equally spaced phases of the frequency-divided clock signal according to the value of control code PICODE. For example, control code PICODE may have F bits (e.g., F=8), corresponding to P=2 F  (e.g., P=256) different PICODE i  (e.g.,  0 &lt;i &lt;P−1), which correspond to P different delay values. A maximum delay is introduced by control code PICODE (e.g., PICODE P−1 ) corresponding to a target maximum delay of almost one cycle of input clock signal CLKVCO (e.g., a delay of 255/256× the period of input clock signal CLKVCO). The target delay increment (i.e., a delay difference between consecutive PICODES, e.g., the delay difference between control code PICODE and control code PICODE i+1 , where 0&lt;PICODE i &lt;PICODE P−1 ) is one cycle of input clock signal CLKVCO cycle divided by P. An exemplary interpolative divider is further described in U.S. Pat. No. 7,417,510 filed 
     Oct. 17, 2006, entitled “Direct Digital Interpolative Synthesis,” naming Yunteng Huang as inventor, which patent is incorporated herein by reference in its entirety. 
     In at least one embodiment of phase interpolator  608 , target performance is achieved by converting a phase interpolator code to analog phase interpolator control signals using a current digital-to-analog converter. First-order noise-shaping dynamic-element-matching encoding techniques are used to convert control code PICODE, which may be periodic, to a plurality of digital control signals that are provided to circuit  604 . Circuit  604  includes a digital-to-analog converter that converts those control signals to individual analog control signals and provides the analog control signals to current sources in phase interpolator  608 . 
       FIG. 7  illustrates a high level block diagram of an embodiment of the line card SyncE PLL  521 . The PLL  521  receives the SyncE input signal  507  as the reference clock signal. The reference clock  507  is compared to the feedback clock  701  in phase and frequency detector  703 , which supplies the phase difference to the loop filter  705 . The loop filter output controls the VCO  707 . The output  525  of the VCO  707  is supplied to the DCO  527 . One or more divider stages  717  divides the VCO output signal to generate the SyncE clock signal used to drive the PHY. Note that the SyncE clock signal does not utilize the PTP loop utilized by the SYNC signal. In an embodiment, substantial portions of the PLL  521  are implemented digitally. Thus, the SyncE input signal  507  and the feedback clock signal  701  are time stamped and the time stamps compared digitally to determine the phase error. Other embodiments utilize PLLs with more analog circuitry. 
     Referring back to  FIG. 5 , the control logic  537  determines the control signal supplied to the interpolative divider based on the SYNC signal received from the master timing card  501 . In an embodiment, the control logic is implemented in firmware on a microcontroller to achieve the functionality described herein. In other embodiments, the control logic is implemented in discrete control logic, or a combination of firmware and other control logic. In an embodiment, the control logic time stamps the receipt of the SYNC signal, e.g., the active edge or both the rising and falling edges. The SYNC signal frequency is known and can be stored or updated in non volatile memory accessible to the control logic  537 . For example, the SYNC signal frequency may equal to 1 PPS or 1 KHz. In addition, the nominal frequency of the SYSCLK is also known. Based on the nominal frequency of the SYSCLK clock signal, e.g., nominally 1 GHz, the control logic knows how many SYSCLK periods should be contained in a period of the SYNC signal. In that way, the control logic sets the DCO control value to achieve the desired number of SYSCLK periods. The control logic may know the nominal frequency of the SyncE clock signal VCO as well to reduce the time it takes for the interpolative divider to lock to the SYNC signal. The SYNC signal is carrying both frequency and phase information that was adjusted in the PTP loop and the information in the SYNC signal can be used to regenerate the SYSCLK signal with the right phase and frequency information in the line card using the interpolative divider. 
     In addition, one or more embodiments, also remove input/output IO delay. Removal of IO delay in line cards which receive both a SYNC signal and a SYSCLK signal is described in detail in the application entitled “Secondary Phase Compensation Assist for PLL IO Delay”, naming Vivek Sarda as inventor, patent application number 16/836,706, filed Mar. 31, 2020. Referring to  FIGS. 5 and 8 , the generated SYNC output signal  533  is fed back as SYNC_FB  535  to the control logic  537 . By feeding back the SYNC output signal, the control logic can keep the output SYNC signal phase and frequency the same as the input SYNC signal. In embodiments, the SYNC signal is fed back through buffer  801  (not shown in  FIG. 5 ) and the SYNC signal is received through buffer  803 . The two signals are time stamped in time stamp logic  805  and  807 , respectively. The time stamp logic functions as a time to digital converter and converts the transitions of the SYNC_FB signal and the input SYNC signal to digital values based on an available timing reference. Difference logic  809  receives the two time stamps and determines the difference between the time stamps of SYNC_FB  535  and the currently valid SYNC signal  505  and supplies the difference, indicative of the IO delay. The control logic uses the IO delay to adjust the DCO to ensure that the SYNC output signal  533  has zero IO delay with respect to the SYNC input signal  505 . That also ensures that any IO delay that would have been associated with a distributed SYSCLK signal over the backplane is also removed. The divider  531  is adjusted at startup for any phase jam. The divide ratio of divider  531  is fixed to the ratio between the SYNC and SYSCLK frequencies. During run time, the DCO (and not the divider) is adjusted since the SYNC PLL and SyncE PLL in the master timing card are running asynchronously. Since the DCO has the SyncE PLL VCO on the timing card as its timing source, the DCO needs to be corrected by the control logic to match the incoming SYNC frequency and phase. 
     The measured IO delay can include delay caused by input buffers, the DCO  527 , the control logic  537 , divide logic  531 , and other clock tree buffers on the line card. The approach used in line card  503  is used on the slave line card and the master line cards in the network box to reduce mismatch. The approach has the advantage of reducing the number of signals that need to be distributed through the backplane. In addition, the PLL for SYSCLK, such as PLL  121 , in  FIG. 1  is omitted. 
       FIG. 9  illustrates a high level diagram of the control flow for control logic  537 . In  901  the control logic supplies an estimated divider value to the interpolative divider based on expected frequency values of SYSCLK and the SyncE PLL VCO output signal. In  903  the control logic determines the period of SYNC based on successive active edges of the SYNC signal. In embodiments, the SYNC signal is timestamped on receipt and successive time stamps used to determine the period. That period value is then used to adjust the initial (or previous) divide value supplied to the interpolative divider in  905  so the right number of SYSCLK periods occur in the SYNC period. There will be a known relationship between the frequency of SYSCLK and SYNC. For example, the SYNC signal may be 1 PPS and the SYSCLK frequency may be 1 GHz. Note that when dividing the SyncE PLL VCO output signal to achieve the desired number of SYSCLK signals, the divider value may not be an integer and thus the control logic supplies the interpolative divider a divide control signal N.f that includes a fractional portion if needed. Given the right N.f divider value, the interpolative divider divides the VCO output signal and generates an output clock signal with the desired SYSCLK frequency. The generated SYSCLK will then be phase and frequency aligned based on the period and time stamps of received SYNC signals. In addition, the control logic determines the IO delay in  907  and adjusts the interpolative divider in  909  to remove the IO delay from the SYNC output signal supplied to the ToD counter. That can be accomplished by adjusting the fractional portion of the interpolative divider control signal. In an embodiment, the control logic responds to every active edge of the received SYNC signal to ensure the SYNC and SYSCLK signal stay locked to the desired timing of SYNC. The IO delay can be determined every active edge of SYNC or less often if desired as IO delay is typically changing slowly. The IO delay adjustment in  909  may be combined with the adjustment in  905  rather than being done separately. 
       FIG. 10  illustrates a slave line card  1001  that generates the SYSCLK signal utilizing an interpolative divider as described above.  FIG. 10  also shows the master timing card  1003  and backup timing card  1005  and the existence of the PTP loop between the slave line card and the master timing card. 
       FIG. 11  illustrates an embodiment of a network box  1100  showing that the master timing card  501  distributes the SYNC signal and the SyncE clock signal to the slave line card  1001  and to master line cards  1103  (only one of which is shown), thereby reducing the signal routing requirements for backplane  509 . 
     In an embodiment as shown in  FIG. 12 , the master timing card  1200  also uses one PLL  1201  to generate the SyncE signal  1203  and an interpolative divider  1205  to generate SYSCLK and uses divider  1207  to generate the SYNC signal  1209  from SYSCLK. The PLL  1201  supplies a timing signal  1208  from the SyncE PLL VCO. The master timing card  1200  supplies the SYNC signal  1209  and the SyncE clock signal  1203  to the backplane  1215  to distribute to the line cards. The PTP loop with the slave line card (not shown in  FIG. 12 ) interacts with the control logic  1217  to adjust the interpolative divider to ensure the SYNC signal  1209  is properly aligned with the timing of the upstream PHY coupled to the slave line card. 
     Thus, a network box has been described that eliminates the need to distribute SYSCLK and instead regenerates SYSCLK on each line card. The description of the invention set forth herein is illustrative and is not intended to limit the scope of the invention as set forth in the following claims. Variations and modifications of the embodiments disclosed herein may be made based on the description set forth herein, without departing from the scope of the invention as set forth in the following claims.