Abstract:
A method of precisely synchronizing clocks held in separate nodes on a communication network is provided that adjusts clock frequency based on a measure of relative clock rates and absolute clock offsets. In one embodiment, clock convergence is obtained with one synchronization session.

Description:
CROSS-REFERENCE TO RELATED APPLICATIONS  
       STATEMENT REGARDING FEDERALLY SPONSORED RESEARCH OR DEVELOPMENT  
       BACKGROUND OF THE INVENTION  
         [0001]    The present invention relates generally to industrial controllers for the control of machines and processes and in particular, for an industrial controller whose components communicate over a network and have precisely synchronized local clocks.  
           [0002]    Industrial controllers are special purpose computers and circuitry for the control of processes and machinery, for example, in a factory or the like. The industrial controller reads inputs from sensors associated with the controlled process and, executing one or more stored programs, provides outputs to actuators associated with the controlled process according to the stored control programs and inputs received by the control programs.  
           [0003]    The process may encompass a large area and thus it is known to distribute the components of the industrial controller so that each component may be close to the particular portion of the process to which it is related. These separated components communicate with each other on a high-speed network.  
           [0004]    For many applications, the speed of communication on the high-speed network is not sufficiently fast to properly synchronize operations at the various components of the industrial controller. For this reason, it is desirable to have precisely coordinated local clocks that can be used to synchronize events or data gathering at the spatially separated components of the industrial controller.  
           [0005]    Such local clocks will tend to drift apart over time either due to minute differences in the natural frequencies of the crystal oscillators in the clocks which causes a steady drift in relative time between the clocks, or because of differences in operating conditions such as temperature, mechanical shock, and aging which causes a variable drift in relative time between the clocks.  
           [0006]    For these reasons, a correction process is needed to maintain individual clocks in synchrony. The IEEE 1588 “Standard for a Precision Clock Synchronization Protocol for Networked Measurement and Control Systems” is directed to maintaining the synchrony between clocks on a network. The IEEE standard defines messages that can be used to exchange timing information, although it does not define a method of implementing time synchronization using these messages.  
           [0007]    Particularly for nondeterministic networks, such as Ethernet, efforts to synchronize clocks using network communications are hampered by the unknown delays that will be introduced by the networks. Tight synchronization, necessary to avoid large corrective jumps in the relative time of local clocks, may require frequent synchronization signals that may adversely affect the available network bandwidth.  
         BRIEF SUMMARY OF THE INVENTION  
         [0008]    The present invention provides a method of precisely synchronizing clocks on a network using a simple calculation that corrects relative errors in time and frequency in as little as one cycle of the synchronization process. The correction operates on clock frequency and thus eliminates jumps in time values.  
           [0009]    Specifically, the present invention provides a method of synchronizing clocks in separate nodes on a communication network starting with the step of determining a count rate of a master clock in a master node and a count rate of a slave clock in a slave node. The difference between the time values of the master clock and slave clock are then determined at a given point in time. The rate of the slave clock is adjusted based on the count rates of the master and slave clocks and on the difference in their clock values to cause the count rates and the counts of the master clock and slave clock to converge.  
           [0010]    Thus, it is one object of the invention to use a measurement of relative count rates, not just a measurement of time value difference, to provide improved control over the convergence of two clocks.  
           [0011]    These steps may be performed at regular intervals and the adjustment of the slave clock rate may be such as to promote convergence of the master and slave clock by the first interval after the adjustment.  
           [0012]    Thus, it is another object of the invention to provide a rapid convergence of the clocks on a network. This rapid convergence provides a predictable locking of the frequencies and times of the clock regardless of their initial conditions and provides close tracking between the clocks limited only by the length of the intervals.  
           [0013]    When the count rate of the master clock is MCC and the count rate of the slave clock is SCC and the difference between the count of the master clock and the count of the slave clock is CCD, the adjustment of the speed of the slave clock may multiply the speed of the slave clock by  
           MCC   +   CCD     SCC     .                         
 
           [0014]    Thus, it is another object of the invention to provide a simple calculation for synchronizing clocks that may be implemented in firmware or software at high speed.  
           [0015]    Determining the count rates of the two clocks may measure, at a first and second corresponding time n-1 and n, clock values MCT n-1  and MCT n  at the master clock and clock values SCT n-1  and SCT n  at the slave clock. The values of MCT n-1  and MCT n  may be subtracted to get the count rate of the master clock and the values of SCT n-1  and SCT n  may be subtracted to get the count rate of the slave clock.  
           [0016]    The values MCT n-1  and SCT n-1  and MCT n  and SCT n  may be provided to the slave node where adjustment of the slave clock is performed by transmitting, at a first and second time n-1 and n, from the master node to the slave node, a first and second synchronization message. The time of the transmission of the first and second synchronization messages is recorded at the master node as MCT n-1  and MCT n  and the time of the receipt of the first and second synchronization messages at the slave node is recorded as SCT n-1  and SCT n . MCT n-1  and MCT n  is then transmitted from the master node to the slave node.  
           [0017]    Thus, it is another object of the invention to communicate necessary information about the clock rate of the master node to the slave node in a way that is relatively indifferent to constant network delay.  
           [0018]    The count difference between the slave clock and master clock may be determined by transmitting from the master node to the slave node a synchronization message. The time of the transmission synchronization messages is recorded at the master node as MMT n  and the time of the receipt of the synchronization messages is recorded at the slave node as SCT n . MMT n  is then transmitted from the master node to the slave node and an estimate network delay is added to MMT n  to determine a master clock time MCT n . The difference between MCT n  and SCT is then determined.  
           [0019]    Thus, it is another object of the invention to provide a method of deducing at the slave node the difference between the master clock value and the slave clock value using the connecting network and standard IEEE synchronization messages.  
           [0020]    The network delay may be estimated by taking the average difference between transmission times and arrival times of delay assessing messages passing from the master node to the slave node and from the slave node to the master node, respectively.  
           [0021]    Thus, it is another object of the invention to provide a method that corrects for unknown or slowly varying network delay such as may be found on nondeterministic type communication networks such as Ethernet.  
           [0022]    These particular objects and advantages may apply to only some embodiments falling within the claims and thus do not define the scope of the invention. 
       
    
    
     BRIEF DESCRIPTION OF THE DRAWINGS  
       [0023]    [0023]FIG. 1 is a perspective view of a simple industrial control system having a master and slave node communicating on a network and individually communicating with components of a controlled process;  
         [0024]    [0024]FIG. 2 is a graph showing a transmission sequence of IEEE synchronization messages that may be used in the present invention;  
         [0025]    [0025]FIG. 3 is a simplified schematic of the circuitry of a node of FIG. 1 such as may implement the present invention in software using standard industrial controller components;  
         [0026]    [0026]FIG. 4 is a view similar to that of FIG. 3 showing implementation of the present invention in dedicated circuitry communicating with a standard network interface;  
         [0027]    [0027]FIG. 5 is a detailed view of the circuitry of FIG. 4 showing components of a variable speed clock and time stamping hardware;  
         [0028]    [0028]FIG. 6 is a flowchart showing the steps of synchronization implemented by the present invention; and  
         [0029]    [0029]FIG. 7 is a graph plotting clock values of a slave and master clock showing values measured and used in the calculations performed in the present invention. 
     
    
     DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT  
       [0030]    Referring now to FIG. 1, an industrial controller  10  may include a communication network  11 , such as Ethernet, connecting a first node  12   a  and a second node  12   b , both being components of an industrial controller. Each node  12  may also provide at least one input or output line  14  communicating with a sensor or actuator  16  in turn connected to the industrial process  18  as is understood in the art.  
         [0031]    Each of the nodes  12  may include a clock (as will be described below) providing for a local time. One clock is designated a master clock and other slave clocks are synchronized to the master clock over the network  11 .  
         [0032]    Referring now to FIG. 2, each of the nodes  12  may send and receive time synchronization messages defined by IEEE standard 1588 hereby incorporated by reference. Following this protocol, a master node  12   a  may send a synchronization message  20  to the slave node  12   b  followed by a follow-up message  22  sending a value of the local time at the master node  12  at which the synchronization message  20  was sent. Similarly, slave node  12   b  may send a delay request message  24  to the master node  12   a , which may reply, with the delay response message  26  sending a value of the local time at the master node  12  at which the delay request message  24  was received. As will be described below, the present invention uses these messages for clock synchronization.  
         [0033]    Referring to FIG. 3, each of the nodes  12  may include an internal processor  30  communicating by an internal bus  32  with a memory  34  holding a stored program executed by the processor  30 . The processor  30  also communicates via bus  32  with a network interface circuit  38  which in turn communicates with the network  11 . The network interface circuit provides the ability to send an interrupt signal via interrupt line  40  to the processor indicating when a message has been received or transmitted.  
         [0034]    In the first embodiment of the present invention, a local clock may be implemented by a general-purpose counter  36  receiving a clock signal from an oscillator  39  and also communicating with the bus  32 . The output of the counter  36  may be read by the processor  30  over the bus  32 , but it is not assumed that the speed of the counter  36  or oscillator may be directly controlled.  
         [0035]    All of these components are typically found on standard industrial controller nodes  12  and may be used to allow the processor  30  to time stamp messages being transmitted or received by network interface circuitry  38  on the network  11 . Generally, at the time of the interrupt over interrupt line  40 , the processor  30  reads the value of the counter  36  and stores that value in memory  34 . The counter  36 , though fixed in frequency and offset may have its value manipulated by a program (dividing or multiplying the counter value and/or adding or subtracting a value to the counter value) to create a virtual frequency and offset controlled virtual local clock that may be synchronized with other clocks as will be described below.  
         [0036]    Referring now to FIG. 4, in an alternative embodiment, the functions of the counter  36  and oscillator  39  may be performed by a time stamp circuit  50  also communicating on the internal bus  32  with processor  30  and memory  34 . The time stamp circuit  50  eavesdrops, via line  54 , on signals passing between the media access circuit  52  physical interface  56  forming parts of the network interface circuit  38 .  
         [0037]    The time stamp circuit  50  provides time stamping of time synchronization messages wholly apart from programs executed by the processor  30  and a true variable speed clock. Referring now to FIG. 5, the time stamp circuit  50  implements a variable speed clock  60  receiving an input from a fixed frequency oscillator  62 , which may be internal to the time stamp circuit  50  (as shown) or from an external source including the clock used generally by the node  12  (not shown).  
         [0038]    The oscillator  62  provides a clock signal to an accumulator  64  causing the accumulator  64  to add a value received from an addend register  66  to the current contents of the accumulator  64 . Repeated addition operations triggered by the oscillator  62  cause a regular overflow of the accumulator at an overflow output of the accumulator  64 . The signal from the overflow output strobes a count input of a clock counter  68  which provides a local clock for the node  12 . The value in the addend register  66  may be loaded by the processor  30  via bus  32  so that processor  30  may effectively control the frequency of the clock counter  68 .  
         [0039]    Initially, the addend register  66  will be loaded by the processor  30  with a value equal to 2 w  divided by FDC, where w is the width of accumulator  64  and FDC is a constant representing the ratio of frequency of oscillator  62  to desired nominal frequency of local clock counter  68 .  
         [0040]    For the purpose of time stamping, the output value of the clock counter  68  is relayed to both a “delay request message” time stamp register  70  and in parallel to a “synchronization message” time stamp register  72 . These registers also communicate with bus  32  allowing processor  30  to read the values of time stamps.  
         [0041]    The time stamping occurs under the control of time message detector circuit  74  detecting the “delay request message” and “synchronization message” via line  54  described above and routing a time stamp signal to the appropriate register through strobing on lines  76  and  78  according to methods understood in the art.  
         [0042]    The present invention provides synchronization in frequency and value of local clocks formed either from counter  36  or from clock counter  68  on a master and slave node  12   a  and  12   b.    
         [0043]    Referring now to FIG. 6, at a first process block  80  in a program executed by processors  30  of master node  12   a  and slave node  12   b , a rough synchronization of the clocks of the master node  12   a  and slave node  12   b  is performed. This rough synchronization may be performed whenever there is a disruption of the network  11  or a new node is connected.  
         [0044]    Referring also to FIG. 2, this rough synchronization performs the following steps: First, a synchronization message  20  is sent from the master node  12   a  to the slave node  12   b  with the time of transmission, MMT, (determined by the local clock of the master node  12   a ) being stored by the master node  12   a  and the time of reception SMT (determined by the local clock of the slave node  12   b ) being stored by the slave node  12   b . This storage may be accomplished either by operation of the processor reading the counter  36  upon strobing of the processor  30  by the network interface circuit  38  or by detection of the synchronization signal by time stamp circuit  50  shown in FIG. 4.  
         [0045]    Second, at a later time, the stored value MMT at the master node  12   a  is transmitted in a follow-up message  22  to the slave node  12   b . Third, upon receipt of the follow-up message  22 , the slave node  12   b  reads its local clock to obtain a slave clock time SCT.  
         [0046]    Third, the SMT value is subtracted from the SCT value and the MMT value is added to the difference (together with a constant time estimating the time necessary to perform these calculations by the processor  30  of the slave node  12   b ), and this value is written to the local clock ( 36  or  68 ) as a new time value.  
         [0047]    As will be understood, the above calculation effectively assumes that the delay on the network MSD in FIG. 2 is zero, and while this is not necessarily correct for most networks, it provides a first rough alignment of the clocks.  
         [0048]    Referring again to FIG. 6 at a succeeding processor block  82 , once the rough adjustment is complete, a fine adjustment process is implemented to keep the local clocks of nodes  12   a  and  12   b  in alignment without the jump that occurred in clock values that occurred with respect to process block  80 .  
         [0049]    Referring now to FIG. 7, generally each of the local slave and master clocks will have different time values and different rates as indicated by sloped lines  90  for the master clock and  92  for the slave clock. The graph of FIG. 7 provides a horizontal axis of absolute time superimposed on a horizontal axis of correction interval. The correction interval for each of the local clocks will generally not align in absolute time but are shown aligned for clarity. The vertical axis of FIG. 7 is local time of the master and slave clocks.  
         [0050]    At a first interval n-1, a measurement is made of the master clock time, MCT n-1 , and the slave clock time SCT n-1 . These values may be triggered by the transmission of a synchronization message  20  as shown in FIG. 2 where MCT n-1  is MMT and SCT n-1  is SMT as shown in FIG. 2. At a somewhat later time, this process may be repeated to collect values MCT n  and SCT n . Desirably, an equal time will have elapsed between the acquisition of MCT values on the master clock  12   a  and SCT values on the slave clock  12   b.    
         [0051]    A master clock count MCC may be deduced by subtracting MCT n-1  from MCT n  and likewise a slave clock count SCC may be determined by subtracting SCT n-1  from SCT n . This subtraction process may be performed either at the master node  12   a  or slave node  12   b  after the MCT values are transferred via follow-up messages  22 . The accuracy of these values is limited only by the consistency of the master to slave delay, MSD, value caused by transmission delays on the network  11 .  
         [0052]    At interval n, a clock count difference CCD is calculated being, for example, the value of SCT n  subtracted from the value of MCT n  and the MSD value added as may be best estimated. Initially, MSD is estimated to be zero but as will be understood from the following description eventually this is refined.  
         [0053]    A correction ratio is then formed having a numerator being the sum of MCC n  and CCD and a denominator of SCC n . That is:  
           MCC   +   CCD     SCC     .                         
 
         [0054]    This correction ratio may be applied to the clock of the local slave node  12   b  by adjusting the addend register  66  of FIG. 5 and FIG. 4 to properly adjust the frequency of the overflow of the accumulator  64  according to this ratio.  
         [0055]    Alternatively in the embodiment of FIG. 3, the above correction ratio is applied by the processor  30  to the value contained in a virtual addend register. Initially, the virtual addend register will be loaded by the processor  30  with a value equal to 2 w  divided by FDC, where w is the width of a virtual accumulator and FDC is a constant representing the ratio of frequency of oscillator  39  to desired nominal frequency of virtual local clock counter. Let HC represent the change in count value of the counter  36  since the interval of the last correction. Then a scaled change in count value is computed by multiplying HC with value of virtual addend register and dividing the result by 2 w , where w is the width of virtual accumulator. This scaled change in count value is added to the last corrected count value of virtual clock counter to produce a current corrected count value which may be used for current timing needs such as time stamping and as the basis for the next correction.  
         [0056]    It will be understood that the clock rate of the slave (the slope of line  92 ) is SCC and thus when this clock rate is multiplied by the correction ratio will cause the new clock rate to equal MCC+CCD. Thus after one additional interval, the local times of the master and slave clocks will converge as indicated by dotted line  100  at which time a new correction process will occur causing the slave and master clock both to have slopes of MCC. Thus, all error is eliminated after a signal correction interval provided the underlying drift between the clocks is constant and the network delay is properly estimated.  
         [0057]    It will be understood from this description, therefore, that the accuracy of this correction depends in part on the ability to accurately estimate the master to slave delay MSD. This estimate, which is initially zero, is refined as indicated in FIG. 6 at process block  102  and in FIG. 2, by repeated delay request, delay response messages. In this cycle, a message is initiated by the slave node  12   b  which obtains a time stamp of the time of transmission TS1 using its local clock. The receipt of this delay request message  24  is time stamped by the master node  12   a  as value TS2. This TS2 value is returned via the delay response message  26 . The current estimate of MSD is subtracted from the difference between time stamps TS2 minus TS1 and the resulting value is divided by 2 to produce next estimate of master to slave delay MSD.  
         [0058]    This process may be repeated over many cycles and the average obtained over a number of such cycles to improve the accuracy of this estimate. Further, if the values obtained that deviate by more than a predetermined amount, they may be discarded so as to provide the best possible estimate of MSD values.  
         [0059]    It is specifically intended that the present invention not be limited to the embodiments and illustrations contained herein, but include modified forms of those embodiments including portions of the embodiments and combinations of elements of different embodiments as come within the scope of the following claims.