Patent Publication Number: US-5841604-A

Title: Masterless spindle synchronization in a multiple disk drive storage system

Description:
This application is a continuation of application Ser. No. 08/432,449 filed May 1, 1995, now abandoned, which is a continuation-in-part of application Ser. No. 08/167,778, filed Dec. 15, 1993, now abandoned. 
    
    
     FIELD OF INVENTION 
     The present invention relates to the field of computer data storage systems, and particularly, to a storage system that uses multiple disk drives having synchronized spindle rotation. 
     BACKGROUND OF THE INVENTION 
     Magnetic and optical disk drives are often used as a Direct Access Storage Device (DASD) for computer systems. Inside the disk drive a spindle rotates a medium, such as a magnetic or optical disk, underneath a transducing head. When writing data to the disk, digital data serves to modulate operation of the transducing head in order to record the data onto the disk in a series of concentric tracks. For example, in magnetic disk drives the digital data modulates the current in a coil of a magnetic write head in order to write a corresponding series of magnetic flux transitions onto the surface of a magnetic disk. To read this recorded data, the transducing head again passes over the medium and generates a read signal processed by read channel circuitry which detects and decodes the digital data. 
     With the dramatic decrease in the manufacturing cost of disk drives, a single storage system commonly incorporates multiple disk drives, thereby increasing the data processing reliability and efficiency. For example, storing the same data block on M disk drives at locations 360/M degrees apart decreases the access time to the data block. The largest spindle angle between the start of the data block and the read head is 360/M degrees as opposed to 360 degrees in a single drive system. U.S. Pat. No. 5,237,466 entitled, &#34;Method and Apparatus for Programmably Controlling Spindle Synchronization and Phase Among Disk Drives in a Storage Subsystem&#34; issued to Glaser et al., discloses a method and apparatus for controlling rotation of the M disk drives to achieve the 360/M degree phase relationship between the spinning disk drives. 
     Another advantage to a multiple disk drive storage system is that multiple copies of the same file can be stored on more than one drive. If one of the disk drive crashes, the file can still be retrieved from one of the operable disk drives. That is, multiple disk drive storage systems provide an automatic and concurrent backup capability, a vast improvement over the more cumbersome tape backup systems. 
     Yet another advantage is to reduce the overall latency of the storage system by interleaving the data blocks of a file over the multiple disks. In this manner, multiple data blocks can be retrieved in parallel, thereby reducing the overall access time to read the file. Interleaving the data blocks also allows very large files, such as large image files, to be stored on the multiple disk drives where one disk drive would not provide enough memory. 
     To achieve the above advantages, it is necessary that the storage disks of the multiple disk drives be rotated at the same frequency and at a predetermined phase relationship. Techniques for controlling the frequency and phase relationship of the rotating disks are well known. Most commonly, the disks are first frequency locked to a target frequency and then phase locked to a selected one of the rotating disks. 
     To frequency lock each disk: a phase angle is measured between an index mark recorded on the disk and the read head; a time period for the phase angle to change by 360 degrees is compared to a target --  time --  period; and the time difference operates to control the spindle rotation speed. After the disks are locked to the target rotation frequency (1/target --  time --  period), they are phase locked to a predetermine one of the disk drives known as the master disk drive. This is achieved by measuring the difference between the phase angle of the master disk drive and the phase angle of the other disk drives. The phase difference operates to control the spindle rotation speed. 
     An example prior art method for synchronizing rotation of a multiple disk drive storage system is described in U.S. Pat. No. 5,237,466 entitled, &#34;Method and Apparatus for Programmably Controlling Spindle Synchronization and Phase Among Disk Drives in a Storage Subsystem&#34; issued to Glaser et al., the disclosure of which is hereby incorporated by reference. 
     A problem with prior art techniques that synchronize spindle rotation of multiple disk drives to a predetermined master disk drive is that if the master disk drive crashes, the remaining disk drives will lose synchronization and the storage system will fail. The storage system must be shut down and the master disk drive replaced. Taking a storage system off-line in order to replace the master disk drive is highly undesirable. 
     Yet another problem with synchronizing to a master disk drive is that the master drive cannot be &#34;hot-swapped&#34; (replaced on-the-fly without shutting down the storage system.) Therefore, &#34;juke-box&#34; storage systems that interchange the disk drives on-the-fly are less reliable if implemented using the prior art master synchronization method. 
     What is needed is a masterless method for synchronizing the spindles of multiple disk drives in a storage system where synchronization is maintained if any one of the multiple disk drives crashes or if any one of the multiple disk drives is &#34;hot-swapped&#34;. 
     SUMMARY OF THE INVENTION 
     In a storage system using multiple disk drives, the disk drive spindles are rotated at the same frequency and at a predetermined phase relationship. A phase angle, measured as the difference between an index mark recorded on the disk and the disk&#39;s read head, is used to control the spindle rotation speed. Each of the disk drives are first independently frequency locked to a target frequency, and once frequency locked, each disk drive is phase locked in relation to one another. In the phase locking process, each disk generates a synchronizing signal for synchronizing the rotation of the disks to a predetermined phase relationship. If any one of the disk drives becomes inoperable (due to a failure or &#34;hot-swapping &#34;), the other disk drives remain synchronized since the synchronizing signal is not lost. 
     Frequency locking the multiple disk drives comprises the steps of: spinning up each disk drive to a predetermined rotation frequency; loading a counter with a target --  time --  period; decrementing the counter at a predetermined clock rate; and using the content of the counter to adjust the speed of the corresponding spindle when the phase angle of the index mark rotates through 360 degrees. When the disk drive has achieved frequency lock, the content of the counter is zero when the phase angle reaches 360 degrees (zero degrees.) 
     Once all of the disk drives are locked to the target frequency (1/target --  time --  period), the system switches into a phase lock mode but does not disturb the current operation of the counters. A terminal count output from each counter are ORed together to generate a synchronizing signal. Initially, each counter contains a different value because the disk drives are not yet phase locked. The first counter to reach terminal count (i.e., the first phase angle to reach zero degrees) activates the ORed synchronizing signal which resets all of the counters with the target --  time --  period. The counters are decremented at the predetermine clock rate and when the index mark is detected, the content of the corresponding counter represents the phase difference between the spindle&#39;s phase angle and the synchronizing signal. The counter value operates to adjust the spindle rotation speed in the direction that decreases the phase difference. When all of the counters simultaneously reach terminal count, their ORed terminal count outputs generate the synchronizing signal which again re-loads all of the counters with the target --  time --  period. This process continues until the disk drives are phase locked to one another. 
     Because every disk drive generates the phase synchronizing signal (terminal count), the phase relationship is maintained once the disk drives are phase locked even if any one of the disk drive crashes or if any one of the disk drives is replaced on-the-fly (i.e., &#34;hot-swapped&#34;). In other words, continuing operation of the storage system is not dependent on the continuing operation of a predetermined one of the disk drives (i.e., a master disk drive.) 
    
    
     BRIEF DESCRIPTION OF THE DRAWINGS 
     FIG. 1 shows a prior art implementation for frequency and phase locking multiple disk drives to a master disk drive. 
     FIG. 2 shows the implementation of the present invention for masterless synchronization of multiple disk drives. 
     FIG. 3 shows a storage disk having an index mark recorded thereon and a read head positioned there above. 
     FIGS. 4A-G shows the disk drive phase angles during frequency lock and the corresponding control signals. 
     FIGS. 5A-G shows the disk drive phase angles during phase lock and the corresponding control signals. 
     FIG. 6 shows a circuit diagram of the speed calculating circuitry for controlling the spindle rotation speed. 
    
    
     DETAILED DESCRIPTION OF THE DRAWINGS 
     Conventional Spindle Synchronization 
     FIG. 1 shows the conventional method for frequency and phase locking the multiple disk drives of a multiple disk drive storage system. One of the disk drives 86, designated as a master, generates a synchronizing signal 120 for synchronizing the other disk drives according to a predetermined phase relationship. A phase angle is measured as the distance from an index mark (110,210) recorded on the disk (96,196) and a corresponding read head (114,214). This phase angle is illustrated in greater detail in FIG. 3. The synchronizing signal 120 is activated when the index mark 110 of the master disk drive is detected by its corresponding read head 114 (i.e., when the phase angle of the master disk drive is zero). 
     Before phase locking the disk drives using the synchronizing signal 120 from the master disk drive 86, all of the disk drives are independently frequency locked to a predetermined reference frequency. A FREQ/PHASE control signal 90 disables the synchronizing signal 120 through a multiplexor 252 in each of the slave disk drives (only one slave disk drive 295 is shown) and selects a frequency lock signal 220 as the load signal LD 250 of a counter 232. The master disk drive 86 uses the synchronizing signal 120 as its frequency lock signal to load its counter 132. The frequency lock signals (120,220) are activated when the corresponding index marks (110,210) are detected by the corresponding read heads (114,214). When the index mark (110,210) is detected, the corresponding counter (132,232) is loaded with a reference value 146 and decremented at a predetermined clock rate 136. 
     The reference value 146 corresponds to the desired spindle rotation frequency and the counter will decrement to zero simultaneous with the corresponding index mark (110,210) being detected if the disk is frequency locked. If the disk (96,196) is rotating to fast, then the counter will not have reached zero when the index mark is detected, and if the disk (96,196) is rotating too slow, then the counter will have under flowed. The frequency lock signal (120,220) latches (166,266) the value (154,254) in the counter (132,232) when the index signal (110,210) is detected by the read head (114,214). The latched signal (170,270) is applied to speed calculating circuitry (174,274) which generates a motor control signal (296,295) and applies it to a spindle motor (106,206). The spindle motor (106,206) rotates the disk at a varying speed according to the motor control signal until the disk is substantially locked to the reference frequency 146. 
     When all of the disk drives are locked to the reference frequency 146, the FREQ/PHASE control signal 90 selects, via the multiplexor 252, the synchronizing signal 120 from the master disk drive 86. The synchronizing signal 120 loads all of the counters 232 in the slave disk drives 295 when the index signal 110 of the master disk drive is detected by its read head 114. The value in the slave disk drive counter 232 is latched 266 by the corresponding index detect signal 220 and used to vary the frequency (and thereby the phase) of the corresponding spindle 202. Eventually the slave disk drive 295 will be phase locked with the master disk drive 86 such that their index marks (110,210) are detected at the same time. 
     The problem with the conventional method for synchronizing the spindles of multiple disk drives is that the slave disk drives will lose the synchronization signal 120 if the master disk drive becomes inoperable. 
     Masterless Spindle Synchronization 
     The present invention overcomes the problem associated with losing the synchronization signal from a master disk drive by having every disk drive generate a synchronization signal. In this manner, the disk drives will remain synchronized if any one of the disk drives becomes inoperable. 
     An overview of the masterless synchronization method in a multiple disk drive storage system is shown in FIG. 2. Four disk drives 292A-292D (disk drives 292B-292C are represented as dots) incorporate the same circuitry and no one is a master. The circuitry in each disk drive operates similar to the conventional spindle synchronization circuit of FIG. 1. The difference between FIG. 1 and FIG. 2 is that the synchronization signal 200 for phase locking the drives is generated by wire ORing a terminal count output 258A-258D from each disk drive rather than using the index detect signal 120 from a master disk drive 86. A multiplexor 252A-252D selects between the frequency and phase lock modes similar to the multiplexor 252 in the slave disk drives of FIG. 1. 
     A detailed description of the frequency and phase lock modes in relation to the circuit diagram of FIG. 2 is provided in the following sections. 
     Frequency Lock 
     FIG. 4 illustrates the operation of the masterless synchronization method of the present invention during the frequency lock mode. The disks 296A-296D of FIG. 2 are depicted as rotating in a counter-clockwise direction with their index marks 210A-210D being detected by a corresponding read head 214A-214D. Shown below each disk 296A-296D is the value stored in the corresponding counter 232A-232D and latch circuit 266A-266D. Index values n A  -n B  shown to the right of the disks 296A-296D represent the number of times the corresponding index mark 210A-210D has been detected by the read head 214A-214D. 
     The storage system 80 enters frequency lock mode by setting the FREQ/PHASE control signal 90 low in order to connect the index detect signal 220A-220D, through the multiplexor 252A-252D, to the load signal 250A-250B of the counters 232A-232D for loading the target --  time --  period 146. 
     Before locking the disk drives 292A-292B to the target frequency (1/target --  time --  period), speed calculating circuitry 274A-274D ignores the latch signal 270A-270D and generates an initial speed control signal 295A-295D for a predetermined amount of time in order to &#34;spin-up&#34; the disks 296A-296D. Once the disks 296A-296D are spinning, the speed calculating circuitry 274A-274D generates the speed control signal 295A-295D in response to the latch signal 270A-270D in order to frequency lock the disks as shown in lines A-G of FIG. 4. 
     Starting with line A, the values in the latches 266A-266D reflect the current spindle control command. For the first disk 296A, the latch 266A stores the hex value 0AB1 which indicates the disk 296A is spinning too fast, and the speed control signal 295A operates to decrease the spindle 202A speed. The latch 266B of the second disk 296B stores the hex value FF9A which indicates the disk 296B is spinning to slow, and the speed control signal 295B operates to increase the spindle 202B speed. Similarly, the latches 266C-266D of disks 296B-296D store values indicating their off-speed status. The index values n A  -n B  are in an undefined state X since an index mark 210A-210D has yet not been detected in the frequency lock mode. 
     Continuing to line B, when the index mark 210B of the second disk drive 292B is detected by the read head 214B, the hex value 0010 stored in its counter 232B indicates that the disk 296B is spinning closer to the target frequency (1/target --  time --  period), but now too fast. The counter value 254B is latched into latch circuit 266B and used by the speed control circuitry 274B to generate a speed control signal 295B in order to decrease the spindle 202B speed. The counter 232B is loaded by the index detect signal 220B with the target --  time --  period 146. The index value n B  is set to 0 indicating that the index mark 210B has been detected for the first time during the frequency lock mode. 
     Operation of the disk drives 292A-292B continues in this manner until they are all independently frequency locked to the target frequency (1/target --  time --  period) as indicated by a counter value 254A-254D of zero when the index mark 210A-210D is detected. For example, when the index mark 210B of the second disk drive 296B is detected in line F, the counter value 254B is zero indicating that the disk 296B is frequency locked. 
     Once the disk drives 292A-292B are frequency locked, the storage system 80 transition into the phase lock mode in order to synchronize rotation of the spindles 202A-202D to a predetermined phase relationship. 
     Phase Lock 
     Operation of the phase lock mode is understood with reference to FIG. 5. The disks 296A-296D of FIG. 2 are depicted as rotating in a counter-clockwise direction with their index marks 210A-210D being detected by a corresponding read head 214A-214D. Shown below each disk 296A-296D is the value stored in the corresponding counter 232A-232D and latch circuit 266A-266D. Index values n A  -n B  shown to the right of the disks 296A-296D represent the number of times the corresponding index mark 210A-210D has been detected by the read head 214A-214D. 
     The storage system 80 enters phase lock mode by setting the FREQ/PHASE control signal 90 high in order to connect the synchronizing signal 200, through the multiplexor 252A-252D, to the load signal 250A-250B of the counters 232A-232D for loading the target --  time --  period 146. 
     Referring to line A of FIG. 5, since the disk drives are frequency locked, the latch circuits 266A-266D are initially zero. The counter 232B of the second disk drive 292B is the first to generate a terminal count synchronizing signal 258B during the phase lock process (i.e., the drive having its phase angle reach zero first.) In this manner, the other disk drives phase lock to the phase angle of the second disk 296B. The terminal count synchronizing signal 258B causes all of the counters 232A-232D to re-load the target --  time --  period 146 (hex value FFFF.) The index value n B  is set to 0 indicating that the index mark 210B has been detected for the first time during the phase lock mode. 
     Continuing to line B, when the index mark 210A of the first disk drive 292A is detected by the read head 214A, the hex value D47B stored in its counter 232A indicates that the difference between the phase angle of its disk 296A and the synchronizing disk 296B is less than 180 degrees. The index detected signal 220A latches 266A the counter value 254A, and the speed calculating circuitry 274A generates a speed control signal 295A that increases the rotation speed of the spindle 202A in order to decrease the phase difference. 
     When the index mark 210C of the third disk drive 292C is detected by the read head 214C as shown in line D, the hex value 56B1 indicates that the difference between the phase angle of its disk 296C and the synchronizing disk 296B is greater than 180 degrees. The index detected signal 220C latches 266C the counter value 254C, and the speed calculating circuitry 274C generates a speed control signal 295C that decreases the rotation speed of the spindle 202A in order to decrease the phase difference. 
     Continuing to line E, the phase angle of the synchronizing disk again reaches zero degrees meaning that all of the counters 232A-2132D have simultaneously reached terminal count, causing each disk drive to generate a synchronizing terminal count signal 258A-258D. If the initial synchronizing disk drive 292B becomes inoperable or is &#34;hot-swapped&#34;, the other disk drives will still phase lock and remain phase locked since the synchronizing signal 200, generated by each disk drive, is not lost. 
     Referring now to line F, when the index mark 210A of the first disk drive 292A is detected by the read head 214A, the phase angle difference between its disk 296A and the synchronizing disk 292B is smaller than in line B but still not zero. The hex value F07B stored in the counter 232A is a smaller negative number than the hex value D47B of line B which decreases the magnitude of the speed control signal 295A. Eventually the phase angle difference between the disks 296A-296D settles to zero and the disk drives 292A-292D are in phase lock. The phase lock relationship will persist even if the original synchronizing disk 292B is removed because the synchronizing signal 200 is generated by the terminal count synchronizing signal 258A-258D from each disk drive (i.e., there is no &#34;master&#34; disk drive.) 
     Speed Calculating Circuitry 
     The speed calculating circuitry 274A for disk drive 292A (and duplicated for disk drives 292B-292C) is shown in FIG. 6. 
     When the disk drive is initially turned on, a reset signal 100 selects a constant C 102 as the speed control output signal 295A in order to &#34;spin up&#34; the disk 296A. After a predetermined amount of time, the reset signal 100 selects a calculated control signal 300A as the speed control output signal 295A in order to begin the frequency lock process. 
     During frequency lock, the FREQ/PHASE control signal 90 is held low such that multiplexors 302A and 304A output a zero. The calculated control signal 300A is generated in response to the latched counter value 270A as: 
     
         U(n.sub.A)=U(n-1)·B.sub.0 +(CNTR.sub.A (n.sub.A)-CNTR.sub.A (n.sub.A -1))·A.sub.0 +(CNTR.sub.A (n.sub.A -1)-CNTR.sub.A (n.sub.A -2))·A.sub.1 ; 
    
     where: 
     U(n A )=the calculated control signal 300A; 
     n A  =the index value indicating the number of times the index mark 210A has been detected; 
     CNTR A  (n A )=the latched counter value 270A; and 
     A 0 , A 1  and B 0  =predetermined coefficients. 
     When CNTR A  (n A )-CNTR A  (n A  -1), the speed error, becomes less than a pre-determined value, the storage system 80 switches into phase lock mode. 
     During phase lock, the FREQ/PHASE control signal 90 is held high such that multiplexors 302A and 304A output the latched counter value 270A. The calculated control signal 300A becomes: 
     
         U(n.sub.A)=U(n-1)·B.sub.0 +(CNTR.sub.A (n.sub.A)-CNTR.sub.A (n.sub.A -1)+CNTR.sub.A (n.sub.A)/G)·A.sub.0 +(CNTR.sub.A (n.sub.A -1)-CNTR.sub.A (n.sub.A -2)+CNTR.sub.A (n.sub.A -1)/G)·A.sub.1 ; 
    
     where: 
     U(n A )=the calculated control signal 300A; 
     n A  =the index value indicating the number of times the index mark 210A has been detected; 
     CNTR A  (n A )=the latched counter value 270A; and 
     A 0 , A 1 , B 0  and G=predetermined coefficients. 
     Many changes in form and detail could be made without departing from the spirit and scope of the present invention; the particular embodiments disclosed are not intended to be limiting. For instance, the phase angles of the disks 292A-292D could be phase locked in an offset relationship by delaying the counter load signals 250A-250D. A software implementation could also replace the circuitry for implementing the spindle synchronization. These and other modifications derived from the disclosed embodiment are within the intended scope of the present invention as properly construed from the following claims.