Abstract:
Data storage system software is installed from nonvolatile memory. A storage processor is booted, transferring information stored in a nonvolatile memory module to a disk drive system, thereby enabling the system processor to boot directly from the disk drive system in subsequent boots. After the information is transferred the storage processor reboots using the information transferred to the disk drive system.

Description:
BACKGROUND OF THE INVENTION 
     1. Field of the Invention 
     The present invention relates to installing data storage system software on disk drive systems. 
     2. Brief Description of Related Prior Art 
     As is known in the art, large mainframe computer systems and data servers sometimes require large capacity data storage systems. One type of data storage system is a magnetic disk storage system. Here a bank of disk drives and the computer systems and data servers are coupled together through an interface. The interface includes storage processors that operate in such a way that they are transparent to the computer. That is, data is stored in, and retrieved from, the bank of disk drives in such a way that the mainframe computer system or data server functions as if it is operating with one mainframe memory. One type of data storage system is a Redundant Array of Inexpensive Disks (RAID) data storage system. A RAID data storage system includes two or more disk drives in combination for fault tolerance and performance. A RAID data storage system is typically made up of a front-end data processor (DPE) and mated with back-end storage disk array enclosure (DAE). Typically the DPE boots up using information which is pre-installed on disks of either the DPE or the DAE. The pre-installed information conventionally needs to be customized to the exact type, configuration and nature of the DPE and the exact type, configuration and nature of the disk drives chosen to form the RAID data storage system. Consequently the disk drives which are pre-installed with information for a specific DPE and disk drive configuration are different from similar disk drives which are pre-installed with information for a different DPE and disk drive configuration, due to the information difference install on the drives. 
     SUMMARY OF THE INVENTION 
     Data storage system software is installed from nonvolatile memory. A storage processor is booted, transferring information stored in a nonvolatile memory module to a disk drive system, thereby enabling the system processor to boot directly from the disk drive system in subsequent boots. After the information is transferred the storage processor reboots using the information transferred to the disk drive system. 
     One or more implementations of the invention may provide one or more of the following advantages. 
     Manufacturers and integrator of data storage system are enabled to ship a data storage systems to customers configured using a common set DPE combined with common set of empty disk drives. The use of empty disk drive simplifies inventory management and increase flexibility in the manufacturing process. 
     Other advantages and features will become apparent from the following description, including the drawings, and from the claims. 
     The details of one or more embodiments of the invention are set forth in the accompanying drawings and the description below. Other features, objects, and advantages of the invention are apparent from the description and drawings, and from the claims. 
    
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         FIGS. 1-3  are block diagrams of a RAID data storage system with serial SCSI (SAS) expansion; 
         FIGS. 4-4A  are block diagrams of interconnections of enclosures in a RAID data storage system with SAS expansion; 
         FIG. 5  is a block diagram of a partition structure of the disks on the data storage system; 
         FIG. 6  is a flow chart of a storage processor (SP) booting process. 
     
    
    
     DETAILED DESCRIPTION 
     Conventionally manufacturers and integrators including original equipment manufacturers (OEM), value added resellers (VAR), and distributors that resell storage products need to assemble arrays to match the configuration of each customer order. This typically requires selecting the desired type of DPE and installing the desired numbers and types of drives requested by the customer. To simplify site installation the boot drives each array are preinstalled with the array&#39;s operating system software. The preinstalled software is matched to the type of DPE used in any particular RAID data storage system. RAID data storage system vendors often support many types of drives in the array (SAS, Serial ATA), different capacities), this means the manufacturer and integrator must stock a different array part number for each different drive type and DPE type combination. When adding other preinstalled options on the DPE such as a Fibre or Internet SCSI (iSCSI) front-end, the number of different part numbers multiplies, increasing inventory costs and storage space. 
     The manufacturer and integrator also need to anticipate the quantity of different types of arrays to stock and many numbers of different types of drives with particular preinstalled software to stock. 
     Eliminating the requirement for pre-installation of software on disk drives substantially overcomes such shortcomings of conventional approaches. 
     Referring now to  FIG. 1 , a data storage system  10  is shown coupled to a pair of host computer/servers  12   a ,  12   b , as shown. The data storage system  10  includes a plurality of, here for example, two chassis or enclosures  14 ,  16 , as shown. Enclosure  14  is sometimes referred to herein as a DPE and enclosure  16  is sometimes referred to herein as a DAE. The DPE  14  and DAE  16  are described in more detail in connection with  FIGS. 2 and 3 , respectively. Suffice it to say here that DPE  14  includes a pair of front end controllers  18   a ,  18   b , each having a pair of ports coupled to the pair of host computer/servers  12   a ,  12   b , as shown. The DPE  14  also includes a pair of storage processors  20   a ,  20   b  coupled to each other with storage processor  20   a  being connected to front end controller  18   a  and storage processor  20   b  being connected to front end controller  18   b , as shown. The storage processors  20   a  and  20   b  are connected to a bank of disk drives  22   a - 22   n  though a plurality of multiplexers  24   a - 24   n , as shown. 
     The storage processors  20   a ,  20   b  of DPE  14  are connected to the DAE  16  though a pair of cables  130   a ,  130   b , respectively, as shown. As is described in more detail in connection with  FIG. 3 , the DAE  16  includes additional disk drives  22 ′ a - 22 ′ n , here for example, twelve disk drives, and is used to increase the storage capacity of the data storage system  10 . Thus, in this example, the number of disk drives  22   a - 22   n  in DPE  14  is twelve and the user has chosen to expand the storage capacity to twenty four disk drives by connecting the DAE  16  which in this example includes twelve disk drives  22 ′ a - 22 ′ n.    
     The storage processors  20   a ,  20   b  of DPE  14  are connected to nonvolatile memory modules  90   a ,  90   b.    
     Referring now to  FIG. 2 , the DPE  14  is shown to include the pair of storage processors  20   a ,  20   b , each disposed on a corresponding one of a pair of printed circuit boards STORAGE PROCESSOR BOARD A and STORAGE PROCESSOR BOARD B [fix since SP was already defined], respectively, as indicated. Each one of the printed circuit boards has disposed thereon: (a) a processor  30 ; (b) a translator  32  controlled by the processor  30 ; (c) a SAS expander  34   a  on STORAGE PROCESSOR BOARD A and SAS expander  34   b  on STORAGE PROCESSOR BOARD B each having a bidirectional front end port  36  and a plurality of bidirectional backend ports  38   a - 38   n , and an expansion port  40   a  for STORAGE PROCESSOR BOARD A and  40   b  STORAGE PROCESSOR BOARD B; and (d) a SAS controller  42  coupled between the translator  32  and the expander controller  34 ; as shown. The DPE  14  also includes an interposer printed circuit board  44  having thereon the plurality of, here twelve, multiplexers  24   a - 24   n . While the preferred implementation uses an interposer printed circuit board to connect the STORAGE PROCESSOR BOARD to the disk drives, other implementations are possible. 
     Each one of the multiplexers  24   a - 24   n  has: (a) a pair of bidirectional front end ports  48   a ,  48   b ; and (b) a pair of bidirectional back end ports  50   a ,  50   b . For each one of the plurality of multiplexers  24   a - 24   n , a first one of the pair of bidirectional front end ports for example port  48   a  is connected to a corresponding backend port  38   a  of the SAS expander  34   a  disposed on a first one of the pair of storage processor printed circuit boards, here STORAGE PROCESSOR BOARD A; and a second one of the pair of bidirectional front end ports  48   b  is connected to a corresponding backend port  38   n  of the SAS expander  34   b  disposed on a second one of the pair of storage processor printed circuit boards here STORAGE PROCESSOR BOARD B. 
     As noted above, the DPE  14  includes a plurality of disk drives  22   a - 22   n . Each one of the disk drives is coupled to at least one backend port  50   a ,  50   b  of a corresponding one of the plurality of multiplexers  22   a - 22   n . More particularly, in the disk drive  22   a - 22   n  is a SAS disk drive having a pair of ports, as shown in  FIG. 2 , the pair of ports is connected to the pair of backend ports of the multiplexer; on the other hand, if the disk drive is a Serial ATA (SATA) disk drive having a single port the signal port is connected to only one of the pair of backend ports of the multiplexer. The multiplexers are here active multiplexers described in the above referenced pending patent application the subject matter thereof being incorporated herein by reference. 
     The DPE  14  also includes a pair of management controllers  60 , each one being disposed on a corresponding one of the pair of storage processor printed circuit boards here STORAGE PROCESSOR BOARD A and here STORAGE PROCESSOR BOARD B, as shown. A first of the pair of management controllers  60 , here the controller  60  disposed on STORAGE PROCESSOR BOARD A includes an additional front end port  36   a  of the SAS expander  34  disposed on such storage processor printed circuit boards and the second one of the pair of management controllers  60  disposed on the STORAGE PROCESSOR BOARD B is coupled to an additional front end port  36   b  of the SAS expander  34 , as shown. 
     Monitors  62   a ,  62   b ,  62   c  herein sometimes referred to as a Vital Product Data (VPD), are disposed on the STORAGE PROCESSOR BOARD A, STORAGE PROCESSOR BOARD B and interposer board  44 , respectively, as shown. The monitors  62   a ,  62   b , and  62   c  are coupled to the pair of management controllers  60  on the STORAGE PROCESSOR BOARDS A and B, as shown. Vital Product Data includes information programmed by the factory into a “resume” EEPROM on some Field Replaceable Units (FRUs), generally containing some unique information on each part such as a World Wide Number and serial number. The term “VPD” is often used to refer to the EEPROM itself. Here, there is a VPD EEPROM on each STORAGE PROCESSOR BOARD A, STORAGE PROCESSOR BOARD B and interposer board  44 . 
     Referring now to  FIG. 3 , DAE  16  is shown to include a pair of SAS expander printed circuit boards  64   a ,  64   b , a pair of SAS expanders  66   a ,  66   b , each one being disposed on a corresponding one of the pair of SAS expander printed circuit boards  64   a ,  64   b , each one of the pair of SAS expanders  66   a ,  66   b  has a bidirectional front end expansion port  68   a ,  68   b , respectively, and a bidirectional backend expansion port  70   a ,  70   b , respectively. 
     Also included in DAE  16  is an interposer printed circuit  72  board. A plurality of, here twelve, multiplexers  74   a - 74   n  is disposed on the interposer printed circuit board  72 , each one of the plurality of multiplexers  74   a - 74   n  includes (a) a pair of bidirectional front end ports  76   a ,  76   b ; (b) a pair of bidirectional back end ports  78   a ,  78   b . For each one of the multiplexers  74   a - 74   n , a first one of the pair of bidirectional front end ports here port  76   a , for example, is connected to a corresponding one of backend ports  80   a - 80   n  of the SAS expander  66   a  and a second one of the pair of bidirectional front end ports, here  76   b , for example, is connected to a corresponding backend port of the SAS expander  66   b  as shown. The DAE  16  includes, as noted above, the plurality of disk drives  22 ′ a - 22 ′ n , each one being coupled to at least one backend port  78   a ,  78   b  of a corresponding one of the plurality of multiplexers  74   a - 74   n . More particularly, in the disk drive  22 ′ a - 22 ′ n  is a SAS disk drive having a pair of ports, as shown in  FIG. 3 , the pair of ports is connected to the pair of backend ports of the multiplexer; on the other hand, if the disk drive is a SATA disk drive having a single port the signal port is connected to only one of the pair of backend ports of the multiplexer. The multiplexers are here active multiplexers described in the above referenced pending patent application the subject matter thereof being incorporated herein by reference. 
     Referring again also to  FIGS. 1 and 2 , the expansion ports  40   a ,  40   b  of SAS expanders  34   a ,  34   b  are connected to the bidirectional front end expansion ports  68   a ,  68   b , respectively, as shown. Thus, SAS expander  34   a  is connected to SAS expander  64   a  through cable  130   a  and SAS expander  34   b  is connected to SAS expander  64   b  through cable  130   b . Thus, referring to  FIG. 1 , data can pass between any one of the host computer/servers  12   a ,  12   b  and any one of the here twenty four disk drives  22   a - 22   n  and  22 ′ a - 22 ′ n.    
     Referring again to FIG.,  3 , as with DPE  14  ( FIG. 2 ) the DAE  16  includes a pair of management controllers, each one being disposed on a corresponding one of the pair of expander printed circuit boards, a first of the pair of expansion board management controllers being coupled to an additional front end port of the SAS expander disposed on the first one of the pair of expander printed circuit boards and a second one the pair of expansion management controllers being coupled to an additional front end port of the SAS expander disposed on the second one of the pair of expander printed circuit boards. 
     Further, as with the DPE  14 , the DAE  16  includes monitors  62 ′ a ,  62 ′ b ,  62 ′ c  having Vital Product Data (VPD) as well as enclosure numerical displays. 
     Thus, the data storage system  10  ( FIG. 1 ) may be further expanded as shown in  FIG. 4  in a cabinet here having four DAEs  16  and a DPE  12 . As noted above, here a DPE has up to 12 disk drives, and each one of the four DAEs, has 12 disk drives to provide, in this example, a data storage system having up to 60 disk drives. Enclosures can be wired up in various ways, two of which are shown in  FIG. 4  and another being shown in  FIG. 4A . The connections between enclosures consist of standard SAS signals and cables. 
     Each one of the cables includes four SAS lanes so that at any one instant in time, at most 4 messages can be going to 4 different drives, but successive messages can be sent to different drives using the same SAS lane. Those 4 lanes are also used to send traffic to drives on downstream expanders, so a message can be sent on one of the input lanes, out one of the 4 output lanes to an input lane on the next box. 
     Here, in the DPE there are eight lanes between the translator and the SAS controller; four SAS lanes between the pair of SAS controllers; one SAS lane between each multiplexer and a backend SAS port; and four lanes at each of the expansion ports  40   a ,  40   b . For each DAE there are four SAS lanes between each one of the ports  70   a ,  70   b  and the connected one of the pair of SAS expanders  64   a ,  64   b , respectively, and one SAS lane between each multiplexer and a backend SAS port. 
     The conventional manufacturing process preloads a boot image directly on the disks,  20   a - 20   n , as part of the manufacturing process using an Image Copy Application (ICA) process. The ICA provides a mechanism to load “virgin” copies of operating system (OS) images to the appropriate regions on the array&#39;s drives with a minimum of support hardware required. 
     ICA images specific to the DPE and selected disk drive combination are downloaded to the drives which are be configured into the DPE. ICA images are compressed to save space and download time. This process creates disk drives which are now customized parts mated to a particular DPE technology. This avoids the need for distributors to stock a multitude of preconfigured RAID data storage system combinations representing all the unique DPE and disk drive type configurations. It is preferable to allow for separate DPE, and non customized disks shipments. This allows the RAID data storage system to initialize its first four boot drives without using conventional LAN-Based ICA manufacturing tools. One SP in each DPE includes a bootable flash memory module containing an initializer program as well as compressed ICA images to be written to the OS disks. The initialization of the disk drives to include bootable information occurs upon the first boot up typically at the customer&#39;s site. This eliminates the preloading of the disk as part of the manufacturing process. 
     A customer can purchase a DPE chassis and standard drives of various interfaces such as the 3.0 Gbit/s SATA (SATA2) or SAS and of different storage capacities. Upon initial installation the customer must populate the DPE with disks, assuring that a minimum of first four drives are inserted properly in the appropriate slots and are of the same type (SAS or SATA2) and of the same storage capacity. Initially, Extended Power On Self Test (POST) executing on the DPE checks the drives for a bootable image. If there is no bootable image, Extended POST boots the SP from the nonvolatile memory module. In at least one implementation a flash memory module is used as the nonvolatile memory module. 
     The preferred implementation uses M-Systems&#39; uDiskOnChip™ (uDOC™) as the flash memory module. The uDOC™ is a flash memory storage device that uses a Universal Serial Bus (USB) interface. Referring to  FIG. 5 , the information  500  on flash memory module  90  includes partition  1   510 , and partition  2   540 . In an embedded system the DiskOnChip acts as a boot device, filling the same role as an IDE hard drive. Microsoft XP Embedded (XPe)  520  does not have native support to boot from USB device, but M-Systems uDiskOnChip is delivered with XP Embedded components that enable it to boot. These components are included in the XPe OS data  520 . The preferred environment uses Microsoft XP Embedded (XPe); however, other implementations are possible. The uDOC part was chosen because it provides the industry&#39;s fastest OS boot and application load time. It applies error detection and on-the-fly error correction, as well as automatic bad block handling to map out bad blocks and ensure that no data is lost. 
     An uDOCPart utility provided by M-Systems is used to partition and format the uDiskOnChip. uDiskOnChip  500 , can be divided into up to four partitions, where the first one can be designated as a bootable drive  510 . A proposed partition table is shown in  FIG. 5 . The first bootable partition  510  is formatted with NTFS. It holds Windows XP embedded OS  520 , with the initializer program  530  in its Startup directory. The bootable XP software  520  contains minimal XP components to save space in flash memory module and to speedup the booting process. Second partition  540  is also formatted with a Microsoft Windows NT® file system and holds two images: Flare (data storage system operating system) Partition  550  and Utility Partition  560 . The Flare Partition is configured to be the bootable partition. Both the FLARE Partition  550  and Utility Partition  560  image are stored in a compressed ICA format. 
       FIG. 6  is a flow chart that describes the DPE power up sequence. This sequence allows for the unattended transfer information to and configuring of disks  22   a - 22   d  as bootable devices. In step  610  the SP A  20   a  powers up and runs Basic Input/Output System (BIOS) and POST. In step  620  SP A looks for the Flare signature on the appropriate drives  22   a - 22   n  depending on the SP, and boots from the selected drive. 
     If SP A cannot find a Flare signature on the appropriate disk drives  22   a - 22   n , the BIOS/POST code in SP A looks for a bootable image on flash memory module, step  630 . If a bootable image is found on the flash memory module, the SP proceeds to step  640  and booting flash memory module running an initializer program contained in the boot image. The initializer module runs required diagnostic checks on the system configuration and hardware  632 . Part of this diagnostic validates that the system has the required number of disks, and disk types sizes installed. In case of errors the imaging process will terminate with the error  698 . 
     If no bootable image is found on either the disk drives or the flash memory module, the SP BIOS/POST proceeds to step  698  which sets an error status. In step  640 , the initializer program creates signatures and Flare Partitions on disks  22   a - 22   d , wiping out any data already on the disks. The initializer program transfers encrypted/compressed partition images  550  and  560  from flash module into memory on SP A in step  644 . In step  650  initializer program then decrypts/decompresses the image in memory and writes it to the appropriate places on disks  22   a - 22   d . The SP copies the decompressed image from the Flare partition image  550  and the decompressed utility partition image  560  from system memory to the disks  20   a - 20   d , creating two partitions respectively on each drive. 
     The initializer reboots both SP A  20   a  and SP B  20   b , in step  650 . BIOS/POST runs on both SP A and SP B and proceeds to booting Flare code which was moved to the appropriate drives in steps  640 ,  644 ,  650 . In step  670  Flare continues with its normal initialization and checks to see whether there is a boot image on the flash memory module. If there is a boot image on the flash memory module the software proceeds to step  680  where the Flare code erases the boot image from the flash memory module. Erasing the image has many benefits, among which, but not limited to, are the ability to use this flash memory module for other data uses, and preventing the improper use of licensed software. If there is no image in the flash memory module, the boot process proceeds to step  690  where the normal boot sequence continues. 
     Upon the first boot of Flare Partition after the image copy is completed an image deletion process runs and erases the content of the flash memory module, however before erasing the content of the flash memory module, checks are made  680  to ensure that SP A and SP B are up and running and stable enough to carry on regular storage activity. The process may be run in reverse. Utilities which were moved from the decompressed flash memory module to the data storage system also provide the ability to move the Flare and Utility partitions from the bootable disk drives into SP memory, where they are compressed into an ICA image. This image can then be transferred from SP memory to the nonvolatile memory module  90   a  and any bootable data on the data storage system can be erased, allowing configured systems to be restored to pre-configured state, which is useful when the RAID data storage system has been used for demonstration and other purposes. While the preferred implementation uses a flash data module connected to only one SP, other implementations allow multiple SPs, with connected flash data modules, in a data storage system. Any number of methods, such as the SP&#39;s configuration in the data storage system as either SP A or SP B, can be used for determining which flash data module connected SP transfers information stored in its flash data module to the disk drive system. 
     In the preferred implementation, the user is given visual user feedback concerning the status of the information transfer process by light emitting diodes (LEDs). Various blink patterns and fault LEDs indicate transfer status and the type and nature of faults that might occur in the transfer process. 
     A number of embodiments of the invention have been described. Nevertheless, it is understood that various modifications may be made without departing from the spirit and scope of the invention. Accordingly, other embodiments are within the scope of the following claims.