Abstract:
A central time-of-day reference is integrated into a switch which interconnects I/O devices and host processors in a computer complex via fiber-optic links. A time reference oscillator in the switch serves to generate incrementing reference signals for the central reference and also provides a clock signal for the switch transmissions. Units to which the switch is attached obtain clock signals from their regenerated clocks which are synchronized to the time reference oscillator, and use the regenerated clock signals to control their local time-of-day references. Each unit periodically transmits a signal to the switch requesting a dynamic connection to the switch-based central time-of-day reference in order to receive a time of day message.

Description:
This is a Continuation of prior complete U.S. application Ser. No. 08/039,544, filed Mar. 29, 1993, now abandoned, which is a Continuation of prior complete U.S. application Ser. No. 07/650,315, filed Feb. 4, 1991, now abandoned. 
    
    
     BACKGROUND OF THE INVENTION 
     1. Field of the Invention 
     The present invention relates generally to the provision of a central reference for local time-of-day clocks in a computer complex, and more particularly to an improved method and apparatus for providing such a central reference for a complex of processors interconnected through a central switch via fiber optic links. 
     2. Description of the Related Art 
     In a computer system employing multiple processors and shared resources, a mechanism for synchronizing the local time-of-day (TOD) clocks and keeping them within a specified tolerance is essential. Reliable operation of such a computer system requires maintaining a log record of all accesses to shared resources in the correct order so that recovery can be effected in the event of a failure. The transactions are ordered in the log by means of a time stamp, and all the TOD clocks in the system must be synchronized closely enough so as to ensure consistency of the log. Lack of synchronization can also cause loss of integrity of shared databases and deadlocks. 
     Processors in a multiple processor complex have their own sources for the TOD clock. Although these clocks operate at the same nominal frequency, they may drift apart resulting in a significant accumulated error over time, unless they are synchronized periodically. A typical drift rate is 50 parts per million (ppm). The clocks of two computers in the complex may then deviate with respect to each other at a maximum rate of 100 ppm. At this rate, it takes 10 seconds for the clocks to be 1 millisecond apart. How often the clock needs to be synchronized depends on how much deviation can be tolerated between the individual computer systems in the complex. 
     Two distinct approaches to synchronization of TOD clocks have been proposed in the past. If synchronization is required only at a coarse granularity, for example once every 10 seconds, a distributed algorithm can be used. With one such distributed approach, the processors exchange messages containing the local TOD information at periodic intervals and update their clocks to correspond to the fastest clock in the system as closely as possible. The synchronization messages are sent over the same network that is used for data communication. Hence, the queuing delays in each node should be accounted for in calculating the frequency of updates as well as the correction factor to be applied. Since the queuing delays are difficult to estimate, the algorithm is useful only in a conflict-free network where every node is guaranteed access to the network over a time period. Further, because of the shared use of the links, a synchronization message may have to wait for a long period while a large block of data is being transferred. This large variance in communication time causes the granularity of clock synchronization to be large. 
     An alternate past proposal is to use dedicated links for passing the TOD information between systems allowing deterministic transmission delays. In one example of this approach, timing information is generated external to the processor complex by a reliable and accurate reference clock and distributed to individual processors in the complex over dedicated fiber optic links. The reference clock is duplicated for reliability. Synchronization of individual TOD clocks is achieved at two levels. At a low level, phase-locked loops (PLL&#39;s) at the computers are kept synchronized at the bit level to the time reference by transmitting a continuous signal on the links. At a higher level, messages containing TOD information are encoded into the signal stream; the messages are decoded by the individual processors and used to start or verify clock synchronization. Exemplary systems of this type are shown in the commonly assigned applications of T. B. Smith, U.S. Ser. No. 07/262,416, filed Oct. 25, 1988; now U.S. Pat. No. 5,146,585, issued Sep. 8, 1992 L. H. Appelbaum et al., U.S. Ser. No. 07/392,812, filed Aug. 11, 1989 now U.S. Pat. No. 5,249,206, issued Sep. 28, 1993; and W. A. Moorman et al., U.S. Ser. No. 07/537,389, filed Jun. 12, 1990 now U.S. Pat. No. 5,041,798, issued Aug. 20, 1991, the specifications of which are incorporated herein by reference. 
     This approach using dedicated links provides clock synchronization at a very fine granularity, but it is achieved at a high cost. It is necessary to install and maintain separate boxes for the timing reference as well as a dedicated network of fiber optic links for distribution of the clock information. However, it may not be necessary to verify synchronization frequently if a phase lock loop is used to continuously maintain low level synchronization. Without a requirement for frequent synchronization verification a dedicated link is not necessary. 
     SUMMARY OF THE INVENTION 
     It is therefore an object of the present invention to provide an improved system for providing TOD synchronizing information to update local TOD clocks in a complex of processors interconnected through a central switch via fiber optic links; a system which does not require a dedicated link and is compatible with existing CPU instruction protocols. 
     Briefly, this invention contemplates the provision of a TOD reference source integrated into the switch which interconnects I/O devices and host processors of a system via fiber optic serial links. A time reference oscillator in the switch serves to generate reference signals for the TOD reference and also provides a clock signal for the switch transmissions. Units to which the switch is attached obtain clock signals from their regenerated clocks which are synchronized to the time reference oscillator by means, for example, of a phase lock loop, and use the regenerated clock signals to control their local TOD counters. Each unit periodically transmits a signal to the switch requesting a dynamic connection to the switch-based TOD reference in order to receive a time-of-day message from a time-of-day counter which is part of the TOD reference in the switch. The TOD message comprises a frame of characters giving the time-of-day information. A special character called an on-time-event (OTE) marks the time the time-of-day information is current. At the time of receipt of the OTE character the time-of-day message received from the switch is compared with the contents of the local time-of-day counter to determine whether the unit is maintaining time-of-day synchronization. In addition, these same techniques can be used to initially set a local time-of-day counter and to start it running. 
    
    
     BRIEF DESCRIPTION OF THE DRAWINGS 
     The foregoing and other objects, aspects and advantages will be better understood from the following detailed description of a preferred embodiment of the invention with reference to the drawings, in which: 
     FIG. 1 is a block diagram of a computer complex providing a TOD time reference to units in the complex in accordance with the teachings of this invention. 
     FIG. 2 is a block diagram for synchronizing a local time-of-day clock in accordance with the teachings of this invention. 
     FIG. 3 is a flow diagram of the operation of the system shown in FIG. 1 in transmitting a time-of-day reference information. 
     FIG. 4 is a block diagram of a system similar to FIG. 1 showing a backup switch. 
    
    
     DESCRIPTION OF THE PREFERRED EMBODIMENTS 
     Referring now to FIG. 1, a computer complex has a number of units 10 connected by serial fiber optic links 12 to serial ports 14 of a switch 16. The switch 16 serves to interconnect the units 10. A unit 10 may be a channel of a central processing unit (CPU), a control unit, or any other unit that may include a local time-of-day clock function generator. The switch 16 may comprise a suitable serial port to fiber optic transmission link for interconnecting the units 10 such as that disclosed in commonly assigned application Ser. No. 07/429,267, filed Oct. 30, 1987 now U.S. Pat. No. 5,107,489, entitled &#34;SWITCH AND ITS PROTOCOL FOR MAKING DYNAMIC CONNECTIONS&#34;, granted Apr. 21, 1992 to Paul J. Brown et al; the specification of which is incorporated herein by reference. This patent illustrates a switch which is a dynamic switch having a protocol for establishing dynamic connections in a link by the use of frames, each frame having an identification of the source of the frame, an indication of the destination of the frame for the requested connection, and link controls to maintain, initiate or terminate a connection between the source and destination. The frames are bounded by a start frame delimiter and an end of frame delimiter which may also act as a connect link control and a disconnect link control, respectively, and the connections are made through the dynamic switch having dynamic-switch ports. The state of a dynamic-switch port is changed dependent upon its present state, the dynamic connection requested, and the direction and type of frames passing through the dynamic-switch port. 
     The switch indicated by the general reference numeral 16, in accordance with the teachings of this invention, includes a time-of-day reference oscillator or clock 18 and a central time-of-day reference 20 which includes a counter, control logic and buffer storage. Inputs from oscillator 18 increment the central time-of-day reference 20 which accumulates and outputs via switch logic 22 a time-of-day message, such as year, month, day, hour, minute, second, and an on-time-event character exactly at the time specified in the time-of-day message. The central time-of-day reference 20 may be of a suitable design known in the art. An external standard 21 for time-of-day information such as a radio signal (i.e. a national time standard that is broadcast by radio signal under the call letters WWV) or a highly accurate standalone reference may be used to ensure the long-term accuracy of central time-of-day reference 20. The central time-of-day reference 20 is coupled via switching logic 22 to serial switch ports 14. The reference oscillator 18 is directly coupled to switch ports 14 and serves as a transmit clock for all transmissions on the links 12, including data as well as TOD messages. This clock information is recovered at the units 10 and is used to maintain synchronization of their local time-of-day references. 
     Referring now to FIG. 2, it shows one example of how the local time-of-day reference of each unit 10 can be synchronized to the central time-of-day reference 20 through the serial switch 16 over the non-dedicated serial fiber optic links 12. The step or &#34;tick&#34; to the local time-of-day reference is derived from the basic data transmission on the link 12. A link logic module 30 formats data received on link 12 for utilization by the unit 10 with which it is associated and similarly formats data from the unit for transmission on the link. Link logic 30 generates a step pulse or &#34;tick&#34; synchronized by the incoming data stream at a predetermined interval; for example, every character time (50 nanoseconds) or alternatively every bit time. This step pulse is coupled to a modulo 20 counter 32 in the case of character time steps, which in turn provides a step pulse once every microsecond to increment the 1-microsecond position in a local time-of-day reference counter 34, which may be of a conventional design well known and widely used in the art. 
     At some preselected rate, a time-of-day controller 36 sends a request to an input/output processor (IOP) 38 to verify that the value in the local time-of-day reference 34 is synchronized with the value in the central time-of-day reference 20. The unit 10 makes a contact request to the switch 16 using standard protocols and, when connected, the IOP 38 transmits the verification request via the link 12 and the switching logic 22 to the central time-of-day reference 20. As will be explained in more detail in connection with FIG. 3, the central time-of-day reference 20 transmits back a time-of-day message of the general format &#34;at the tone the time will be HH:MM:SEC:XX:YY&#34;, a format compatible with the format kept by the local time-of-day reference 34. The time-of-day message provided via the switch 16 to the unit 10 is stored by IOP 38 in a synchronization register 40. The central time-of-day reference 20 also subsequentially transmits an on-time-event character to the unit 10 and then breaks the connection through the switch 16. The on-time-event character is delayed at the switch 16 so that all on-time-event characters arrive at the respective ends of the links 12 at the correct time, allowing for transmission time, plus or minus some small tolerance. 
     The logic module 30 forwards the on-time-event character to a time-of-day synchronization controller 52 which causes a comparison to be made in comparator 50 between the time-of-day value in register 40 and the time-of-day value in the local reference 34 at the instant established by the on-time-event character. 
     Referring now to FIG. 3, a time-of-day reference request from a unit 10 may be transmitted in a standard data format with the central time-of-day reference 20 address as the destination and the unit&#39;s address as the source. This request is indicated in function box 60. At switch 16, the current time reference data, including the time at the next on-time-event, parameters used for time conversion, and time reference status information will normally be formatted and buffer stored and the time-of-day reference logic 20 will notify the unit 10 from which the command is received. As indicated in function box 62, if the central time-of-day reference 20 has previously received a READ TIME reference command from a different unit 10 and not yet transmitted an on-time-event character or the command is received too close to the next on-time-event to assure that the TOD data can be used by the unit, then a Busy is returned to the address of the unit 10 which is the source of the request. 
     The transmission of a TOD reference message to the unit 10 consists of several frame transmissions. The first, as indicated by function box 64, is a control frame sent before or with the time-of-day message to establish a connection via the switch logic 22. Following the establishment of a connection (block 64), a time-of-day message will be transmitted to the unit 10 (block 66). The TOD message may be in a standard data format with a fixed unique message in the data field. When the TOD message has been transmitted the connection is terminated (block 68). A control frame is next transmitted significantly before the OTE character to insure establishment of a connection to the unit for the OTE character (block 70). The OTE character is then transmitted (block 72) at a point in time which causes the character to be received by the unit at the exact time that the on-time-event occurs in the central time-of-day reference 20. The time at which the OTE character is sent by the central time-of-day reference 20 is based on a transmission time calculation for the particular unit to which the OTE character is being sent. The transmission time is calculated when the unit initially comes on line, by determining the elapsed interval between transmission of a control frame and receipt of an acknowledgement by the unit divided by two, and the value is stored. Finally, the OTE character path is terminated (block 74). 
     FIG. 4 illustrates an embodiment of the invention in which redundant switches 16 and 16&#39; are used to interconnect the units 10. Each of the switches. 16 and 16&#39; includes a clock 18 and counter and control logic 20 for generating and transmitting TOD information. One of the switches (for example the left-hand switch 16 in FIG. 4) serves as the primary TOD reference, and the other serves as a backup in case of failure of the primary. A link 75 extending between the switching logic 22 of switch unit 16 and the switching logic 22 of switch unit 16&#39; provides a communication path between the two central TOD references 20 to maintain synchronization between the two references. Communication protocols on link 75 may be similar to those on links 12 to the units 10. Relative synchronization of the two central TOD references 20 may be achieved in any suitable manner, such as exchanging subdivided clock signals and phase-locking each reference oscillator 18 to the subdivided clock signal of the other unit 16 or 16&#39; as described in the copending applications referred to above. 
     While the invention has been described in terms of two preferred embodiments, those skilled in the art will recognize that the invention can be practiced with modification within the spirit and scope of the appended claims.