Source: https://patents.google.com/patent/US7581060B2/en
Timestamp: 2018-05-20 16:15:02
Document Index: 175016868

Matched Legal Cases: ['art 500', 'art 500', 'art 500', 'art 130', 'art 500', 'art.\n6']

US7581060B2 - Storage device control apparatus and control method for the storage device control apparatus - Google Patents
Storage device control apparatus and control method for the storage device control apparatus Download PDF
US7581060B2
US7581060B2 US11984570 US98457007A US7581060B2 US 7581060 B2 US7581060 B2 US 7581060B2 US 11984570 US11984570 US 11984570 US 98457007 A US98457007 A US 98457007A US 7581060 B2 US7581060 B2 US 7581060B2
US11984570
US20080209101A1 (en )
The present application is a continuation of application Ser. No. 11/606,878, filed Dec. 1, 2006, now U.S. Pat. No. 7,320,051; which is a continuation of application Ser. No. 10/654,996, filed Sep. 5, 2003, now U.S. Pat. No. 7,231,490; which claims prior based on JP-A-2003-111405, filed Apr. 16, 2003, the contents of which are incorporated by reference herein.
A variety of storage systems used as data storage apparatuses in computer systems has been proposed. The storage systems range from small to large systems.
Communication between the disk control device 110 and host computers 200 can be carried out according to various communication protocols, for example, Fiber Channel, SCSI (Small Computer System Interface), FICON (Fibre Connection) (registered trademark), ESCON (Enterprise System Connection) (registered trademark), ACONARC (Advanced Connection Architecture) (registered trademark), FIBARC (Fibre Connection Architecture) (registered trademark), and TCP/IP (Transmission Control Protocol/Internet Protocol). These communication protocols may co-exist in the system. For example, the disk control device 110 may communicate with Host A 200 via Fiber Channel and Host B 200 via TCP/IP. When the host computer 200 is a main frame computer, FICON, ESCON, ACONARC, or FIBRARC may be used, while it is an open system computer, Fiber Channel, SCSI, or TCP/IP may be used. The data read/write requests from the host computers 200 may be made by the block, as a unit of data management in a storage volume, or by the file by specifying the name of a file. In the latter case, the disk control device 110 functions as a NAS (Network Attached Storage) that enables file-level access from the host computers 200.
The host IF controller 810 has an interface function with a host computer 200. The disk IF controller 860 has an interface function for I/O control to the storage volumes. The local cache 830 stores data exchanged between the host computer 200 and the storage volumes. The cache controller 820 controls the local cache 830. In the embodiment, the storage control units 800 form clusters so that even when a failure occurs to one of the storage control units 800 in the same cluster, the other storage control unit 800 in the same cluster can take over the processing of the failed storage control unit 800 to continue the processing. One cache controller 820 is connected to the other cache controller 820 of the other storage control unit 800 in a cluster through an inter-pair connection portion 850. Thus, the storage control units 800 store each other's data in the respective local caches 830 to duplicate the data. The internal IF controller 840 is connected with the global caches 600, the disk control units 400, the channel control units 300, and the other storage control units 800 through the internal connection part 500. It should be noted that each storage control unit 800 may be equipped with the host IF controller 810, the disk IF controller 860, and the internal IF controller 840 without the local cache 830 and the cache controller 820. In this case, the storage control units 800 may be such that the respective internal IF controllers 840 are connected to each other through the inter-pair connection portion 850. Further, data exchanged between the host computer 200 and the storage volumes may be stored in the global caches 600 to be described later, rather than in the local caches 830, or the is data may be exchanged therebetween without being stored in the local caches 830 or the global caches 600.
As mentioned above, the storage control unit 800 includes a package of the host IF controller 810 having the interface function with the host computer 200, the disk IF controller 860 with the interface function for I/O control to the storage volumes, and the local cache 830 for storing data exchanged between the host computer 200 and the storage volumes. This configuration allows for easy expansion of the system by just adding optional storage control units 800 to the system. The package means that multiple functions are modularized into a single part. Maintenance and management such as replacement of parts are carried out on a package basis.
Further, in the storage control unit 800, since the host IF controller 810, the disk IF controller 860, and the local cache 830 are included in the same package, the performance of data I/O between the host computer 200 and the storage volumes can be improved.
The disk control device 110 using such storage control units 800 is designed to have the best cost-efficiency at the time of initial introduction. Despite its cost-efficiency, since the disk control device 110 still maintain expandability, it is suitable for a wide range from small-and-medium systems to large systems. For example, as will be described later, the storage control unit 800 may be housed in a case together with the power source 112 and the fan 113 to form a module type controller 111. In this case, the module type controller 111 makes the initial introduction of the storage system 100 easy. The expansion of the system is also made easy by just adding module type controllers 111 sequentially to the system. This is effective in implementing a flexible system capable of changing the size of the system according to the needs of customers who may be going to start their businesses or who are facing the ever-changing business climate. Further, data read out by the disk IF controller 860 from a storage volume is stored in a data area 831 of the local cache 830 via the cache controller 820 without through the internal connection part 500 and the global cache 600, thereby achieving high-speed reading.
The cache area management table 833 holds information for specifying respective storage areas of the self-directed SAVOL 836A, the other-directed DAVOL 836B, and the communication buffer 837 provided in the data area 831. The information for specifying the storage areas is, for example, local cache address information. In the example of FIG. 13, the data area 831 is allocated to addresses in which the self-directed SAVOL area 836A ranges from addresses “00000000” to “AFFFFFFF,” the other-directed DAVOL area 836B ranges from addresses “B0000000” to “EFFFFFFF,” and the communication buffer 837 ranges from addresses “F0000000” to “FFFFFFFF.” The allocation of addresses for each area can be changed by making a change in the contents of the cache area management table 833. For example, if many of data I/O requests from the host computer 200 are directed to storage volumes connected to the storage control unit 800 that has received the data I/O requests, the allocation of addresses for the self-directed SAVOL area 836A can be increased. As a result, since it can be expected that the cache hit rate of the local cache 830 to the data I/O requests from the host computer 200 will increase, the performance of the storage system 100 can be improved. The change in the contents of the cache area management table 833 can be made, for example, on the management terminal 160 by an operator doing maintenance of the storage system 100.
The “CA No” column holds identification Numbers assigned to host IF controllers 810 of storage control units 800 or host IF controllers of channel is control units 300 mounted in the disk control device 110. In the example of FIG. 13, CA00 and CA01 are entered in the “CA No” column. As shown in FIG. 2, CA00 and CA01 represent the host IF controllers 810 of the storage control units 800 in a cluster.
The “DA No” field holds identification numbers assigned to disk IF controllers 860 of storage control units 800 or disk IF controllers 460 of disk control units 400 mounted in the disk control device 110. In the example of FIG. 13, DA00 and DA01, DA02 and DA03, and DA04 and DA05 are entered in the “DA No” column. DA00 and DA01 represent disk IF controllers 860 of the same storage control units 800 as those including the host IF controllers 810 identified with CA00 and CA01. DA02 and DA03 represent disk IF controllers 860 of storage control units 800 different from those including the host IF controllers 810 identified with CA00 and CA01. DA04 and DA05 represent disk IF controllers 460 of disk control units 400 different from the storage control units 800 including the host IF controllers 810 identified with CA00 and CA01. When receiving at CA00 and CA01 data I/O requests from the host computer 200, the storage control units 800 process all the I/O requests regardless of whether they are directed to storage volumes connected with their own disk IF controllers 860 or to storage volumes connected with disk IF controllers 860 or 460 of another pair of storage control units 800 or disk control units 400.
The CPU 161 controls the entire operation of the management terminal 160. The CPU 161 reads a management program 169 from the storage device 168 into the memory 162 as appropriate to execute the program so that various features for maintenance of the storage system 100 will be implemented. For example, it can set a logical volume on a disk drive group and install the program 813 executed in the host IF controller 810 of the storage control unit 800. The recording medium reader 164 reads programs and data recorded on a recording medium 167. The read programs and data are stored in the memory 162 or the storage device 168. Thus, for example, the management program 169 or the program 813 recorded on the recording medium 167 can be read out of the recording medium 167 with the recording medium reader 164 and stored in the memory 162 or the storage device 168. The recording medium 167 may be a flexible disk, CD-ROM, DVD-ROM, semiconductor memory, etc. The recording medium reader 164 may be incorporated in the management terminal 160, or added externally. The management program 169 is stored in the storage device 168. The storage device 168 is, for example, a hard disk drive or semiconductor storage device. The input device 165 is used for the operator to enter data into the management terminal 160. The input device 165 includes a keyboard and a mouse, for example. The output device 166 is to output information to the outside world, and includes a display and a printer, for example. The port 163 is to communicate with the disk control device 110. The port 163 can also be used to communicate with another computer, not shown. In this case, for example, the program 813 can be received from another computer through the port 163 and installed in the storage control unit 800.
FIG. 23 is a block diagram showing the structure of the storage system 100 using the initial controller 111 shown in FIGS. 21 and 22. As shown in FIG. 23, storage control units 800 are provided as the modular controller or initial controller 111. In this case, the storage system 100 does not need to include the expensive internal connection part 500 designed for the maximum size of is the storage system 100. In other words, the integrated controller 111 does not need introducing, thereby realizing the storage system 100 at low cost. Further, the global caches 600 and the disk control units 400 are all provided as an integrated controller to be inserted in the mounting part 130. In this case, the internal connection part 500 and the initial controller 111 are connected through a cable.
It is first determined whether the unit to perform I/O control to the newly configured storage volume is a storage control unit 800 (S2000). If it is the storage control unit 800, a lock is applied to the volume management table 835 of the local cache 830 (S2001). Then information relating to the newly added storage volume is written into the volume management table 835 (S2002). Upon completion of the writing process, the lock to the volume management table 835 of the local cache 830 is released (S2003). Subsequently, the same writing is performed on the volume management table 605 of the global cache 600, and the processing is ended (S2004 to S2006). When it is determined in step S2000 that the target unit is not the storage control unit 800, only the volume management table 605 of the global cache 600 is processed.
At first, the cache controller IF circuit 821 or 321 secures an area (communication buffer) for message access (S10000). In other words, it secures a vacant area in the communication buffer 837 of the local cache 830 or 330 of the unit to perform I/O control to the storage volume to which the data I/O request is directed.
a plurality of channel control units, each with a host interface controller formed therein for receiving data input/output (I/O) requests from a first host computer;
a plurality of disk control units, each with a disk interface controller formed therein for performing I/O control of data to first storage volumes storing data in response to the data I/O requests;
a plurality of storage control units, each with a host interface controller for receiving data I/O requests from a second host computer, a disk interface controller for performing I/O control of data to second storage volumes, and a memory formed therein for temporarily storing data being transferred between said host interface controller and said disk interface controller; and
an internal connection part which is able to connect to and/or disconnect from each of said channel control units, said disk control units, and said storage control units independently,
wherein, upon receipt of a data I/O request from the first host computer to a channel control unit, said host interface controller, included in said channel control unit, analyzes the data I/O request, and based on the analysis, determines whether the data I/O request is a read request or a write request, determines an address of data to be accessed, and sends a command according to whether the data I/O request is a read request or a write request to one of the disk control units, and
wherein, upon receipt of a data I/O request from the second host computer to a storage control unit, said host interface controller, included in said storage control unit, analyzes the data I/O request, and based on the analysis, determines whether the data I/O request is a read request or a write request, determines an address of data to be accessed, sends a command according to whether the data I/O request is a read request or a write request to said memory if the data to be accessed is stored in said memory, and sends a command according to whether the data I/O request is a read request or a write request to the disk interface controller included in said storage control unit if the data to be accessed is not stored in said memory.
a first storage area for storing data to be stored in storage volumes to which said storage control unit performs I/O control, and
5. A storage device control apparatus according to claim 2, further comprising:
a mounting part which removably mounts each of said channel control units, said disk control units, and said storage control units in said storage device control section in a manner to be independent of one another to permit independent connection or disconnection of each of said units to said internal connection part.
6. A control method for a storage device control apparatus including a plurality of channel control units, each with a host interface controller formed therein for receiving data input/output (I/O) requests from a first host computer, a plurality of disk control units, each with a disk interface controller formed therein for performing I/O control of data to first storage volumes storing data in response to the data I/O requests, a plurality of storage control units, each with a host interface controller for receiving data I/O requests from a second host computer, a disk interface controller for performing I/O control of data to second storage volumes, and a memory formed therein for temporarily storing data being transferred between said host interface controller and said disk interface controller, and an internal connection part which is able to connect to/or disconnect from each of said channel control units, the disk control units, and the storage control units independently, wherein a plurality of the storage control units each holds in the memory therein at least information for identifying a unit to perform I/O control to a storage volume to which a data I/O request, said control method comprising the steps of:
receiving data I/O request at one of said storage control units from the second host computer;
referring to information by the storage control unit to identify a storage control unit to perform I/O control to a storage volume to which the data I/O request is directed; and
performing the I/O control by the storage control unit when the unit to perform the I/O control is the one storage control unit, or letting another storage control unit perform the I/O control when the unit to perform the I/O control is not the one storage control unit,
wherein said internal connection part allows the channel control units, the disk control units, and the storage control units to each be independently connected or disconnected to said storage device control apparatus,
wherein, upon receipt of the data I/O request from the second host computer and performing the I/O control by the one or the another storage control unit, said host interface controller, included in the one or the another storage control unit, analyzes the data I/O request, and based on the analysis, determines whether the data I/O request is a read request or a write request, determines an address of data to be accessed, sends a command according to whether the data I/O request is a read request or a write request to said memory if the data to be accessed is stored in said memory, and sends a command according to whether the data I/O request is a read request or a write request to the disk interface controller included in said storage control unit if the data to be accessed is not stored in said memory.
7. A control method according to claim 6, wherein when the data I/O request received by one storage control unit is a data read request, and the unit to perform the I/O control is not the one storage control unit, said step of carrying out the I/O control by said another storage control unit comprises the steps of:
said step of receiving the data from said another storage control unit by the one storage control unit includes the step of:
said step of receiving data indicating that the I/O control of the write data has been completed from said another storage control unit by the one storage control unit, includes the step of:
15. A control method according to claim 13, wherein when at least one storage control unit, and one disk control unit are mounted in the mounting part, the memory of the at least one storage control unit includes:
a first storage area for storing data to be stored in storage volumes to which the at least one storage control unit performs I/O control, and
16. A control method according to claim 13, wherein when at least one storage control unit, and one disk control unit are mounted in the mounting part, the memory of the storage control unit includes:
performing the I/O control to the second storage area by the at least one storage control unit when the unit to perform the I/O control is the one disk control unit
17. A control method according to claim 13, wherein when a disk control unit is mounted in the mounting part, said control method further comprising the steps of:
18. A control method according to claim 13, wherein at least one storage control unit and one disk control unit are mounted in the mounting part, said method further comprising the steps of:
19. A control method according to claim 18, wherein when the data I/O request received by the at least one storage control unit is a data read request, and the unit to perform the I/O control is not the at least one storage control unit, said step of performing the I/O control by the disk control unit comprises the steps of:
20. A control method according to claim 18, wherein when the data I/O request received by the at least one storage control unit is a data write request, and the unit to perform the I/O control is not the at least one storage control unit, said step of performing the I/O control by the disk control unit comprises the steps of:
21. A control method according to claim 18, wherein the memory of the at least one storage control unit includes:
US11984570 2003-04-16 2007-11-20 Storage device control apparatus and control method for the storage device control apparatus Expired - Fee Related US7581060B2 (en)
JP2003-111405 2003-04-16
JP2003111405A JP4413518B2 (en) 2003-04-16 2003-04-16 Control method of a storage device controller, and a storage device control unit
US10654996 US7231490B2 (en) 2003-04-16 2003-09-05 Storage device control apparatus and control method for the storage device control apparatus
US11606878 US7320051B2 (en) 2003-04-16 2006-12-01 Storage device control apparatus and control method for the storage device control apparatus
US11984570 US7581060B2 (en) 2003-04-16 2007-11-20 Storage device control apparatus and control method for the storage device control apparatus
US20080209101A1 true US20080209101A1 (en) 2008-08-28
US7581060B2 true US7581060B2 (en) 2009-08-25
ID=32906034
US10654996 Expired - Fee Related US7231490B2 (en) 2003-04-16 2003-09-05 Storage device control apparatus and control method for the storage device control apparatus
US11606878 Expired - Fee Related US7320051B2 (en) 2003-04-16 2006-12-01 Storage device control apparatus and control method for the storage device control apparatus
US11984570 Expired - Fee Related US7581060B2 (en) 2003-04-16 2007-11-20 Storage device control apparatus and control method for the storage device control apparatus
US (3) US7231490B2 (en)
EP (1) EP1469380B1 (en)
JP (1) JP4413518B2 (en)
CN (3) CN1855027A (en)
US7047388B2 (en) * 2002-12-06 2006-05-16 Hitachi, Ltd. Control method for storage device controller system, and storage device controller system
JP4412981B2 (en) 2003-11-26 2010-02-10 株式会社日立製作所 Data caching method in the storage system 及同 system
US8099634B2 (en) 2008-09-09 2012-01-17 International Business Machines Corporation Autonomic component service state management for a multiple function component
JP5621246B2 (en) * 2009-11-26 2014-11-12 日本電気株式会社 Method of controlling a disk enclosure and storage system
KR20170140443A (en) * 2014-09-15 2017-12-20 후아웨이 테크놀러지 컴퍼니 리미티드 Write data request processing method and storage array
JP2000267815A (en) 1999-03-17 2000-09-29 Hitachi Ltd Disk array controller
EP1132805A2 (en) 2000-03-10 2001-09-12 Hitachi, Ltd. Disk array controller and method for adapting the number of control units in a disk array controller
US20010047462A1 (en) 1999-04-09 2001-11-29 Dimitroff John E. Cascaded removable media data storage system
US20020046321A1 (en) 2000-10-17 2002-04-18 Hiroki Kanai Disk controller and method of controlling the cache
US20020083270A1 (en) 2000-12-27 2002-06-27 Chilton Kendell A. Data storage systems and methods which utilize an on-board cache
JP2002333956A (en) 2001-05-09 2002-11-22 Hitachi Ltd Computer system using disk controller and its operation service
WO2003030006A1 (en) 2001-09-28 2003-04-10 Chaparral Network Storage Inc. Controller data sharing using a modular dma architecture
JP2003241905A (en) 2002-02-15 2003-08-29 Hitachi Ltd Storage device, data storing method and control device
JP2003263279A (en) 2003-03-27 2003-09-19 Hitachi Ltd Disk array control apparatus
JP2003323261A (en) 2002-04-26 2003-11-14 Hitachi Ltd Disk control system, disk control apparatus, disk system and control method thereof
US20030224824A1 (en) 2002-05-31 2003-12-04 Hanson George E. Radio configuration and control of computer subsystems
US20050050268A1 (en) 2001-07-04 2005-03-03 Hitachi, Ltd. Disk array control apparatus and control data transfer method using the same
US7231490B2 (en) 2003-04-16 2007-06-12 Hitachi, Ltd. Storage device control apparatus and control method for the storage device control apparatus
US830270A (en) * 1905-05-25 1906-09-04 Reuben Willetts Gas-engine.
US4600321A (en) * 1985-02-06 1986-07-15 Okun Kwan Matrix print head
US6480905B1 (en) 1999-12-11 2002-11-12 International Business Machines Corporation Data storage library with efficient cartridge insert
US7130961B2 (en) 2000-10-17 2006-10-31 Hitachi, Ltd. Disk controller and method of controlling the cache
US20040158656A1 (en) 2001-05-09 2004-08-12 Hitachi, Ltd. Computer system using disk controller and operating service thereof
US6961788B2 (en) 2002-04-26 2005-11-01 Hitachi, Ltd. Disk control device and control method therefor
CN1538282A (en) 2004-10-20 application
US7231490B2 (en) 2007-06-12 grant
US20040210713A1 (en) 2004-10-21 application
JP4413518B2 (en) 2010-02-10 grant
EP1469380A3 (en) 2008-02-27 application
CN1265274C (en) 2006-07-19 grant
CN1881168A (en) 2006-12-20 application
CN1855027A (en) 2006-11-01 application
US7320051B2 (en) 2008-01-15 grant
US20080209101A1 (en) 2008-08-28 application
EP1469380A2 (en) 2004-10-20 application
EP1469380B1 (en) 2013-02-27 grant
JP2004318484A (en) 2004-11-11 application
US20070079066A1 (en) 2007-04-05 application
CN100517210C (en) 2009-07-22 grant
US20020161971A1 (en) 2002-10-31 Library of hard disk drives with transparent emulating interface
US6973551B1 (en) 2005-12-06 Data storage system having atomic memory operation