Source: http://www.google.com/patents/US20030221062?dq=%223do%22+dna
Timestamp: 2017-07-25 07:42:56
Document Index: 473970020

Matched Legal Cases: ['art 51', 'art 52', 'art 51', 'art 52', 'art 51', 'art 52', 'art 52', 'art 502', 'art 51', 'art 52', 'art 52', 'art 51', 'art 51', 'art 52']

Patent US20030221062 - Disk array system and cache control method - Google PatentsSearch Images Maps Play YouTube News Gmail Drive More »Sign inPatentsDisclosed is a disk array system that can be expanded effectively in scale by increasing the number of input/output channels and disk adapters and improved in such performance as the number of input/output operations per second, data transfer rate. The disk array system is provided with input/output...http://www.google.com/patents/US20030221062?utm_source=gb-gplus-sharePatent US20030221062 - Disk array system and cache control methodAdvanced Patent SearchTry the new Google Patents, with machine-classified Google Scholar results, and Japanese and South Korean patents.Publication numberUS20030221062 A1Publication typeApplicationApplication numberUS 10/219,768Publication dateNov 27, 2003Filing dateAug 16, 2002Priority dateMay 24, 2002Also published asUS7054997, US7392341, US20060174061Publication number10219768, 219768, US 2003/0221062 A1, US 2003/221062 A1, US 20030221062 A1, US 20030221062A1, US 2003221062 A1, US 2003221062A1, US-A1-20030221062, US-A1-2003221062, US2003/0221062A1, US2003/221062A1, US20030221062 A1, US20030221062A1, US2003221062 A1, US2003221062A1InventorsKentaro ShimadaOriginal AssigneeHitachi, Ltd.Export CitationBiBTeX, EndNote, RefManPatent Citations (19), Referenced by (24), Classifications (21), Legal Events (3) External Links: USPTO, USPTO Assignment, EspacenetDisk array system and cache control method
US 20030221062 A1Abstract
Disclosed is a disk array system that can be expanded effectively in scale by increasing the number of input/output channels and disk adapters and improved in such performance as the number of input/output operations per second, data transfer rate. The disk array system is provided with input/output channels to be coupled to a host computer, cache memories coupled to each of input/output channels respectively and used to store input/output data temporarily, disk drives, disk adapters coupled to the disk drives, and network switches used to couple the input/output channels to the disk adapters. Images(9) Claims(10)
BRIEF DESCRIPTION OF THE DRAWINGS [0017] [0017]FIG. 1 is an embodiment of a disk array system of the invention; [0018] [0018]FIG. 2 is an embodiment of an input/output channel used for the disk array system of the invention; [0019] [0019]FIG. 3 is an embodiment of a disk adapter used for the disk array system of the invention; [0020] [0020]FIG. 4 is an embodiment of a cache memory used for the disk array system of the invention; [0021] [0021]FIG. 5 is another embodiment of the cache memory used for the disk array system of the invention; [0022] [0022]FIG. 6 is an example of a data table format that determines a disk adapter number according to a data block number in an input/output channel; [0023] [0023]FIG. 7 is an example of a data table format that determines a disk drive according to a data block number in a disk adapter; [0024] [0024]FIG. 8 is an example of an invalidate command format used for a cache memory, sent to an input/output channel from a disk adapter; [0025] [0025]FIG. 9 is an example of an update command format used for a cache memory, sent to an input/output channel from a disk adapter; [0026] [0026]FIG. 10 is a first embodiment of a method of controlling a cache memory in the disk array system of the invention; [0027] [0027]FIG. 11 is an example of timing chart of the operation of the cache memory in the embodiment shown in FIG. 10; [0028] [0028]FIG. 12 is another example of timing chart of the operation of the cache memory shown in the embodiment shown in FIG. 10; [0029] [0029]FIG. 13 is a second embodiment of method of controlling a cache memory in the disk array system of the invention; and [0030] [0030]FIG. 14 is a third embodiment of method of controlling a cache memory in the disk array system of the invention.
DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS [0031] Hereunder, the preferred embodiment of the invention will be described with reference to the accompanying drawings. [0032] [0032]FIG. 1 is a block diagram of a disk array system in a preferred embodiment of the invention. [0033] In FIG. 1, the disk array system 1 has, four input/output channels 10 a to 10 d coupled to external appliances (host computers) to send/receive data to/from the host computer, four cache memories 11 a to 11 d coupled to the input/output channels 10 a to 10 d each to store input/output data temporarily so as to speed up data transfer, four disk drives 14 a to 14 d, four disk adapters 13 a to 13 d coupled to the disk drives 14 a to 14 d to read/write data from/to the disk drives, and two network switches 12 a and 12 b to couple the input/output channels 10 a to 10 d to the disk adapters 13 a to 13 d. [0034] The disk array system 1 shown in FIG. 1 has two network switches 12 a and 12 b in order to improve the data transfer rate between each of the input/output channels 10 a to 10 d and each of the disk adapters 13 a to 13 d and in order to improve reliability. There is some redundancy in the path between the input/output channel 10 a to 10 d and the disk adapter 13 a to 13 d. [0035] In FIG. 1, any of the four input/output channels 10 a to 10 d can receive data requests (read request) from the host computer. [0036] While it is assumed here that the input/output channel 10 a receives the read requests, other input/output channels 10 a to 10 d also operate similarly. [0037] Receiving a data request, at first the input/output channel 10 a checks its coupled cache memory 11 a for the presence of the requested data. [0038] Predetermined data block numbers are used between the disk array system and the host computers so as to identify each requested data. [0039] When the requested data is found in the cache memory 11 a, the input/output channel 10 a reads the data from the cache memory 11 a and sends the data to the host computer. [0040] The input/output channel 10 a needs only to check the cache memory 11 a coupled to itself; the input/output channel 10 a does not need to make any other data communication or wait for data. Consequently, the data transfer is speeded up significantly. [0041] According to any of the previously described conventional techniques disclosed in the official gazettes of JP-A No. 203201/1999 and JP-A No. 290790/2001, it takes much time to read data from a cache memory, since the reading is done through a switch provided between the cache memory and the subject input/output channel. In addition, when one cache memory is checked for data presence from plural input/output channels and/or when data is read from the cache memory by those channels, a conflict often occurs among the accesses from those channels, thereby the accesses would be suspended. [0042] On the other hand, the presented invention can speed up the processing about three to five times faster than any of the conventional ones, since the invention includes no such reasons that might cause the performance degradation. [0043] For example, in the above-described conventional examples, at least four times of communications are required for an input/output channel to read data from a cache memory, since the read request is sent from an input/output channel to a switch, then from the switch to a cache memory, and furthermore, the data read from the cache memory is sent to a switch, then from the switch to the input/output channel. When a conflict occurs in the course, the data reading will further be delayed. [0044] In the disk array system in the embodiment of the invention shown in FIG. 1, by contrast, the input/output channel 10 a reads data directly from the cache memory 11 a coupled to itself, thereby the data communication is made just twice. In addition, the reading path is short and no conflict occurs in the path. [0045] When the requested data is not found in the cache memory 11 a, the input/output channel 10 a determines which one of the disk adapters 13 a to 13 d is coupled to the disk drive that stores the requested data according to the requested data block number, and sends the data request to the disk adapter. [0046] In this example, it is assumed that the disk drive coupled to the disk adapter 13 a stores the requested data. [0047] At this time, the input/output channel 10 a uses the network switch 12 a or 12 b to send the data request to the disk adapter 13 a. [0048] Receiving the data request, the disk adapter 13 a determines which of the disk drives 14 a and 14 b stores the data according to the data block number, then reads the data therefrom. [0049] At the same time, the disk adaptor 13 a checks if the requested data has been sent to any of the other input/output channels 10 b to 10 d. If the check result is YES, the disk adaptor 13 a sends an invalidate command to the channels 10 b to 10 d to delete the data from the cache memories 11 b to 11 d coupled to each of them. [0050] In the above case, when the requested data found in the cache memories 11 b to 11 d is modified and the modified data has not yet been written back in the disk drive, the disk adapter 13 a receives the modified data from the cache memories 11 b to 11 d through the input/output channels 10 b to 10 d, then overwrites the modified data on the data read newly from a disk drive. [0051] After this, the disk adapter 13 a sends the data to the input/output channel 10 a. [0052] The disk adapter 13 a then records that the data has been sent to the input/output channel 10 a so as to reflect the fact in the subsequent processing. [0053] When the overwriting is done on the data read from a disk drive, the disk adapter 13 a writes back the overwritten data to the disk drive. [0054] The input/output channel 10 a, when receiving the data from the disk adapter 13 a, sends the data to the host computer and also stores the data in the cache memory 11 a coupled to itself. [0055] The input/output channel 10 a, when receiving a write request from the host computer, checks its coupled cache memory 11 a for the presence of the data having the received data block number similarly to the response to a read request. When the data is found in the cache memory 11 a, the input/output channel 10 a updates the data in the cache memory 11 a. [0056] The updated data is sent to the disk adapter 13 a, then written back to the necessary disk drives, for example, after a preset time is up, when the cache memory 11 a becomes full, or when the input/output channel 10 a receives an invalid command from the disk adapter 13 a because of a data request issued by one of the other input/output channels 10 b to 10 d to the data of the same data block number. [0057] When the requested data is not found in the cache memory 11 a, the disk adapter 13 a reads the data from a disk, then sends the data to the input/output channel 10 a similarly to the above processing for a read request. [0058] The input/output channel 10 a then stores the data received from the disk adapter 13 a in the cache memory 11 a, then writes the write data received from the host computer in the required data blocks, thereby the data therein is updated. [0059] The processing of the input/output channel 10 a after the data updating is similar to those executed when the original data is found in the cache memory 11 a. [0060] [0060]FIG. 2 is a block diagram of each of the input/output channels 10 a to 10 d usable in the embodiment of the invention shown in FIG. 1. [0061] In FIG. 2, the input/output channels 10 a to 10 d are respectively provided with a processor 201 used as a control unit for controlling its processing. [0062] Each of the input/output channels 10 a to 10 d is also provided with a local memory 202 for storing a control program to be executed by the processor 201, as well as information (data) required for its processing. [0063] The disk array system is further provided with a host interface circuit 204 for coupling itself to the host computer and network couple ion circuits 203 a and 203 b for coupling itself to network switches 12 a and 12 b. [0064] Those devices are coupled to the processor 201 through a processor bus 205, which is a fast signal line bus. Further, the cache memories 11 a to 11 d are also coupled to the processor bus 205. [0065] Consequently, the processor 201 can read/write data from/to the cache memory 11 a to 11 d very fast. [0066] In the embodiment shown in FIG. 2, it is expected that the processor 201 executes the control program stored in the local memory 202 to execute the functions of the input/output channels. [0067] On the other hand, some or all the functions of the control program can be implemented as a hardware item instead. [0068] [0068]FIG. 3 is a block diagram of each of the disk adapters 13 a to 13 d usable in the embodiment of the invention shown in FIG. 1. [0069] In FIG. 3, the disk adapter is provided with a processor 301 used as a control unit for controlling its processing. [0070] The disk adapter is also provided with a local memory 302 for storing a control program to be executed by the processor 301, as well as information (data) required for its processing. [0071] The disk adapter is further provided with a disk drive interface circuit 304 for coupling itself to the disk drives 14 a to 14 d, as well as network couple ion circuits 303 a and 303 b for coupling itself to network switches 12 a and 12 b. [0072] Those devices are coupled to the processor 301 through a processor bus 305, which is a fast signal bus. [0073] In the embodiment shown in FIG. 3, it is expected that the processor 301 executes the control program stored in the local memory 302 to execute the functions of the disk adapters. Similarly to the input/output channels, some or all the functions of the control program can be implemented as a hardware item. [0074] [0074]FIG. 4 is a block diagram of each of cache memories 11 a to 11 d usable in the embodiment of the invention shown in FIG. 1. [0075] The cache memory shown in FIG. 4 is small in capacity. The cache memory is composed of a search control unit 40 and a memory 41. The memory 41 stores various data items in the fields of valid bit V 401, rewrite bit D 402, data block number 403, and data 404. [0076] The input/output channels 10 a notifies the search control unit 40 of a data block number of the data requested by the host computer through an address bus 411 and instructs the search control unit 40 to check the memory 41 for the presence of the data having both of the requested data block number 403 and the valid bit V set at ‘1’ (valid). [0077] When the requested data is found in the memory 41, the search control unit 40 sends a hit signal to the input/output channel 10 a through a response signal bus 412 to notify the data existence. [0078] The input/output channel 10 a reads the data 404 through the data bus 413. [0079] When the request is a write one, the input/output channel 10 a notifies the search control unit 40 of the write command through the control command bus 414 and writes the data transferred through the data bus 413 in the data field 404, thereby the dirty bit D 402 is set (ex., at ‘1’). Thus, the updating is recorded. [0080] When the requested data is not found, the search control unit 40 sends a miss signal to the input/output channel 10 a through the response signal bus 412. The signal denotes the data is not present. [0081] When any of the disk adapters 13 a to 13 d sends an invalidate command, the input/output channels 10 a notifies the search control unit 40 of the data block number through the address bus 411 and instructs the search control unit 40 to check the memory 41 for the presence of the data having both of the matching data block number 403 and the valid bit V 401 set at ‘1’ (valid). [0082] When the data is found, the input/output channel 10 a also checks the state of the dirty bit D 402 for set or not set. [0083] When the dirty bit D 402 is set, the search control unit 40 sends a dirty hit signal to the input/output channel 10 a through the response signal bus 412. [0084] Receiving the dirty hit signal, the input/output channels 10 a reads the data 404 through the data bus 413 and sends back the data to the one of the disk adapters 13 a to 13 d that has sent the invalidate signal. [0085] When the dirty bit D 402 is not set, the search control unit 40 sends a clean hit signal to the input/output channel 10 a through the response signal bus 412. [0086] In those two cases, any of the input/output channels 10 a to 10 d notifies the search control unit 40 of the data block number through the address bus 411 and the invalidate command through the control command bus 414 again, then resets the valid bit V 401 (ex., to ‘0’) to invalidate the data set. [0087] When the search fails, the search control unit 40 sends a miss signal to the input/output channels 10 a through the response signal bus 412 to notify the search failure. In any of the above cases, the input/output channels 10 a finally returns an invalidation complete signal to the one of the disk adapter 13 a to 13 d that has sent the invalidate command to notify the completion of the invalidation. [0088] When the dirty bit D 402 is set and updating of the data read from a disk drive is recorded, besides as described above, the updated data is sent to the disk adapters 13 a to 13 d when a preset time is up or when the cache memory becomes full as described with reference to FIG. 1, then the data is written back to the necessary disk drives, thereby the dirty bit D 402 is reset (ex., ‘0’) to delete the record of the update. [0089] In FIG. 4, while the search control unit 40 actually controls searching in the memory 41, the searching can also be done by the processor 201 of the input/output channels 10 a to 10 d, which is provided only with the memory 41 and not provided with the search control unit specially. [0090] Furthermore, while the address bus 411, the response signal bus 412, the data bus 413, and the control command bus 414 are coupled integrally to each of the input/output channels 10 a to 10 d in FIG. 4, the cache memory can also provided only with a single signal bus, which is used to send commands equivalent to those transferred through the address bus 411, the response signal bus 412, the data bus 413, and the control command bus 414. [0091] In the example shown in FIG. 4, it is required that the single memory 41 stores every set of data written in the fields of valid bit V 401, rewrite bit D 402, data block number 403, and data 404. Thus, data comes to be searched in the whole memory 41. [0092] Consequently, when the memory 41 is expanded in capacity, it is expected that searching takes much time. To avoid the problem, the example shown in FIG. 5 comes to be focused on. [0093] [0093]FIG. 5 is another block diagram of each of the cache memories 11 a to 11 d usable in the embodiment of the invention shown in FIG. 1. [0094] In FIG. 5, the cache memory is configured by a search control unit 50, a tag part 51, and a data part 52. The tag part 51 stores each set of data written in the fields of valid bit V 501, dirty bit D 502, data block number 503, and data location 504. [0095] The data part 52 stores the data written in the field 505. [0096] The fields of valid bit V 501, dirty bit D 502, and data block number 503 in the tag part 51 are identical to those in the example shown in FIG. 4. [0097] The data location field 504 stores a location of each data item written in the data part field 52. [0098] When data is to be read from the data part 52, a data location signal 515 notifies the data part 52 of the data location within the data part 502 according to the data location 504 corresponding to the data block number 503, thereby the data is read onto the data bus 513. [0099] Similarly to the read operation described above, write data transferred through the data bus 513 is written in the location specified by the data location signal 515. [0100] Signals to be transferred through the address bus 511, the control command bus 514, and the response signal bus 512 are identical to those in the example shown in FIG. 4. [0101] With the configuration, the cache memory shown in FIG. 5 can search target data only in the tag part 51. This is why the searching time is not affected so much by the capacity of the data part 52. [0102] In other words, in the example shown in FIG. 4, when the number of data blocks increases, more semiconductor chips are required, so that data comes to be searched in those semiconductor chips, resulting in an increase of the searching time. This is because the memory 41 also stores the data written in the data part 52. On the other hand, in the example shown in FIG. 5, the semiconductor chip that stores data written in the data part is provided separately from the semiconductor chip that stores data written in the tag part. Consequently, the number of semiconductor chips for storing the tag part can be reduced, thereby data is searched in the reduced number of chips. The searching time thus becomes short. [0103] While the search control unit 50 actually controls searching in the tag part 51 in FIG. 5, the cache memories 11 a to 11 d can also be searched by the processor 201 in each of the input/output channels 10 a to 10 d. In this case, each of the cache memories 11 a to 11 d is provided only with the tag part 51 and the data part 52 and not provided with the search control unit specially. [0104] Furthermore, while the address bus 511, the response signal bus 512, the data bus 513, and the command bus 514 are integrally coupled to each of the input/output channels 10 a to 10 d in FIG. 5, each of the cache memories 11 a to 11 d can also be provided only with a single signal bus just like the example shown in FIG. 4 and commands equivalent to the signals that pass in the above buses 511, 512, 513, and 514 are transmitted in the single bus. [0105] [0105]FIG. 6 is a format of a data table provided in each of the input/output channels 10 a to 10 d. The table stores data block numbers and disk adapter numbers used to identify which of the disk adapters 13 a to 13 d should be requested for object data when data requested from the host computer is not found in the coupled cache memories 11 a to 11 d. [0106] In the data table, a data block number and a disk adapter number are paired and the data block numbers are set according to the amount predetermined between the host computer and each of the input/output channels 10 a to 10 d. [0107] Any number of consecutive data block numbers (ex., 16) to be stored in the disk drives coupled to the same disk adapter can be grouped so as to create pairs of the start disk drive number and a disk adapter number. [0108] As a result, the data table can be reduced only to, for example, {fraction (1/16)} in size. [0109] Such a data table is stored, for example, in the local memory 202 shown in FIG. 2 and the data in the table is read/written by the processor 201 as needed. [0110] [0110]FIG. 7 is a format of another data table used to store data denoting a location of each data item requested from each input/output channel 10 a to 10 d. The data table is provided in each of the disk adapters 13 a to 13 d. [0111] Each of the data tables stores data sets, each being configured by a data block number, a disk drive number, a data location, and an input/output channel number that denotes a destination input/output channel. [0112] When data of the data block number has not been sent to any input/output channel before, the input/output channel number field is stored as a predetermined invalid input/output channel number. [0113] When the cache memory of an input/output channel becomes full, thereby the sent data is rejected/deleted from the cache memory, the input/output channel number field can be set as an invalid input/output channel number. [0114] The size of these data tables can be determined by the amount of data block numbers stored in any of the subject disk drives 14 a to 14 d coupled to each of the disk adapters 13 a to 13 d. [0115] These data tables are actually stored in the local memory 302 shown, for example, in FIG. 3 and data in the tables is read/written by the processor 301 as needed. [0116] [0116]FIG. 8 is a format of the invalidate command. This command is sent from the disk adapters 13 a to 13 d to the input/output channels 10 a to 10 d so as to invalidate some data sent to the cache memory of the input/output channels 10 a to 10 d when the sent data is requested again from another input/output channel. [0117] The format is composed of an invalidate command code that represents an invalidate command and a data block number of the data to be invalidated. [0118] In addition to the above method for invalidating the data sent to the cache memory of one of the input/output channels 10 a to 10 d with an invalidate command when the sent data is requested again from another input/output channel as described above, there is also another method used by any of the disk adapters 13 a to 13 d to implement the same function without doing the invalidation. Concretely, when some same data is updated among plural input/output channels, the system writes back the updated data to the necessary disk drives through the disk adapters 13 a to 13 d and updates the data in the cache memories of all of other input/output channels instead of the invalidation. [0119] [0119]FIG. 9 is a format of the update command to be sent to the input/output channels 10 a to 10 d from the disk adapters 13 a to 13 d instead of the invalidate command shown in FIG. 8 in such a case. [0120] The update command is composed of a command code that denotes the update command, a subject data block number, and update data used to update the subject data. [0121] How to control the cache memories 11 a to 11 d in response to the update command shown in FIG. 9 will be described in detail later. [0122] [0122]FIG. 10 is a flowchart for controlling each of the cache memories 11 a to 11 d in response to the invalidate command shown in FIG. 8. [0123] In FIG. 10, it is assumed that data requested from the host computer is not found in the cache memory. [0124] When the requested data is found in the cache memory as a result of searching in response to a read request, the system is just required to read the data from the cache memory and send the data to the host computer as described in the example of the cache memory shown in FIGS. 4 and 5. When data is to be written, the system is just required to update the data in the cache memory with the write data received from the host computer. [0125] In FIG. 10, a data request is received in step 601 and the requested data is not found in the cache memory of the subject input/output channel. [0126] In step 602, the system determines a disk drive that stores requested data. [0127] In detail, the input/output channel determines a disk adapter coupled to the disk drive that stores the requested data with reference to the data table as shown in FIG. 6, then sends the data request to the disk adapter. [0128] The disk adapter, upon receiving the data request, determines the target disk drive and the target data location in the drive with reference to the data table as shown in FIG. 7. [0129] In step 603, the disk adaptor starts to read the data from the disk drive according to the result determined in step 602. [0130] Actually, the disk adaptor need to complete the reading of the data from the disk drive only before step 608 or 612 as will be described later. There is no need for the disk adaptor to wait until the data reading is completed in step 602. [0131] In step 604, the disk adaptor checks if the same data has been sent to the cache memory of other input/output channel before. [0132] In detail, the disk adaptor checks if any valid input/output channel number is registered as a destination input/output channel number in the data table as shown in FIG. 7. [0133] In step 605, the disk adaptor determines if the requested data has been sent to the cache memory of other input/output channels before. [0134] When the determination result is NO (not sent), the system goes to step 611. [0135] When the determination result is YES, the system sends an invalidate command as shown in FIG. 8 to the input/output channel in step 606 to delete the data from the cache memory of the input/output channel. [0136] At this time, the system determines in step 607 if the data has been modified in the cache memory of the input/output channel and the modified data has not yet been written back to the disk drive. [0137] In detail, the system makes this determination by checking the dirty bit D 402 or 502 shown in FIG. 4 or 5. [0138] When the determination result is NO (not written back), the system reads the modified data from the input/output channels cache memory and returns the data to the disk adapter in step 608. [0139] The disk adapter waits for completion of reading of the data from the disk drive, which has started in step 603, then overwrites the modified data returned from the input/output channel on the data read from the disk drive. [0140] In step 609, the disk adaptor waits for an invalidation complete signal to be received from the input/output channel to confirm the completion of the invalidation. [0141] In step 610, the disk adaptor sets a predetermined invalid channel number as described in FIG. 7 as the destination input/output channel number to delete the destination record from the data table shown in FIG. 7. [0142] After this, the input/output channel number that is to send another data in step 611 (to be described later) is recorded as this destination input/output channel number. This step can thus be omitted. [0143] In step 611, the disk adaptor determines to send data to the cache memory of the input/output channel, which is determined as a new request source in step 601, then records the input/output channel number. [0144] In detail, the system records a valid channel number as the destination input/output channel number in the data table as shown in FIG. 7. [0145] If the reading that has started in step 603 is not completed yet at this time, the system waits for completion of reading from the disk drive in step 612. The system then sends the read data to the input/output channel so that the data is returned to the host computer and stored in the cache memory coupled to the input/output channel. [0146] In step 613, the system determines if any data is overwritten on the data read from the disk drive after the processing in step 608. [0147] When some data is overwritten, the system goes to step 614 to write back the overwritten data to the disk drive. [0148] In the embodiment of the controlling method present invention, employed for the cache memory shown in FIG. 10, it is possible to complete reading from the disk drive, which begins in step 603 in or before step 608 or 612, to overlap the processing with the invalidation of the cache memory of another input input/output channel that has read the data, thereby eliminating the overhead to occur with respect to the invalidation of cache memory. [0149] [0149]FIG. 11 is a timing chart that denotes the elimination of the overhead. [0150] In FIG. 11, time t1 corresponds to step 601 shown in FIG. 10, in which the system receives a data request at the time t1. [0151] After the processing in step 602, the system goes to step 603 shown in FIG. 10 at time t2 to start data reading from the disk drive. [0152] After this, the system executes the processing in steps 604 and 605, then goes to step 606 at time t3 to issue an invalidate command to the input/output channel. [0153] [0153]FIG. 11 shows a case in which it is found in an invalidation processing that there is no data that has not yet been written back. [0154] In FIG. 11, the system goes to step 609 at time t4 after completing the processing in step 607, then confirms the completion of the invalidation in the same step (609). [0155] The system then executes the processing in steps 610 and 611 and completes the reading from the target disk drive at time t5. [0156] After this, the system goes to step 612 at time t6 to send the data to the target input/output channel. [0157] After the processing in step 613, the processing in this flowchart is ended. [0158] As shown in FIG. 11, reading from a disk drive between t2 (step 603) and time t5 overlaps with the invalidation between t3 (step 606) and time t4 (step 609), so that no overhead occurs in the invalidation. [0159] [0159]FIG. 12 is a timing chart for invalidation processing in which there is some data that has not yet been written back to any disk drive in the cache memory to be invalidated. [0160] In FIG. 12, time t1 corresponds to step 601 shown in FIG. 10. At time t1, the system receives a data request. [0161] After the processing in step 602, the system goes to step 603 shown in FIG. 10 at time t2 and starts to read data from the target disk drive. [0162] After this, the system executes the processing in steps 604 and 605, then goes to step 606 at time t3 to issue an invalidate command to the target input/output channel. [0163] In FIG. 12, it is assumed here that some data that has not written back to the disk drive is found in an input/output channel during the invalidation. [0164] The system then goes to step 607 at time t4 to report the presence of not-yet-written-back data to the disk adapter through the input/output channel. [0165] Consequently, the system, after completing the reading of data from the disk drive at time t5, goes to step 608 at time t6 and instructs the input/output channel to send the not-written-back data so as to overwrite the data on the data read from the disk drive in the disk adapter. [0166] The system then completes both sending of the not-written-data through this input/output channel and overwriting of the data on the data read from the disk drive at time t7, then goes to step 609 to confirm the completion of the invalidation in the same step (609). [0167] The system then executes the processing in steps 610 and 611 and goes to step 612 at time t8 to send the data to the target input/output channel. [0168] Then, the system executes the processing in step 613 and step 614 at time t9 to write back the overwritten data to the necessary disk drives. [0169] This writing-back is completed at time t10. [0170] In FIG. 12, some overhead occurs due to the overwriting between time t6 (step 608) and time t7 (step 609) and the writing back to the disk drives between time t9 (step 614) and time t10. [0171] The overhead occurrence frequency in this processing seems to be low, since the occurrence is limited only upon a data request issued from a host computer coupled to another input/output channel while the object data is stored in the cache memory of an input/output channel and the data has not yet written back to the disk drives after the data is modified by a host computer. [0172] In the embodiment of the cache memory control method of the invention shown in FIG. 10, reading from a disk drive and invalidation of another cache memory overlap as shown in FIGS. 11 and 12. On the other hand, there is a more simple method, which invalidates another cache memory first. Then, if there is any data that is not written back to the disk drive in the cache memory to be invalidated, the data in the cache memory is written back to the necessary disk drives, then the data is read from any of the disk drives. This can not utilize the overlapping to occur between invalidation of another cache memory and reading from the disk drive, but the controlling becomes simple. [0173] There is also another method that when overlapping of invalidation of another cache memory with reading from a disk drive can not be utilized while there is any data that is not written back to the disk drive in the cache memory to be invalidated, the data is written back to the disk drives, then the written-back data is not read from the disk drives. Instead, the written-back data is used as it is. Consequently, the system can reduce the time of reading from a disk drive though it can not utilize the overlapping invalidation of another cache memory with reading from the disk drive. [0174] As described above, FIG. 10 shows an example in which the system, when receiving a request for the same data from another input/output channel after having sent the data to an input/output channel, invalidates the data sent to the cache memory of the former input/output channel. [0175] On the other hand, some applications that run in a host computer often cope with the occurrence of an overhead during a write operation more easily than the occurrence of an overhead during a read operation. [0176] [0176]FIG. 13 is a flowchart for controlling the cache memory of the invention with respect to such a case. [0177] In FIG. 13, the system receives a data request in step 701. In this example, it is assumed that the requested data is not found in the cache memory of the input/output channel just like the example shown in FIG. 10. [0178] In step 702, the system determines which disk drive stores the requested data. [0179] In step 703, the system begins to read the data from the disk drive according to the determination in step 702. [0180] The system is just expected to complete the reading the data from the disk drive in or before the step 706. [0181] Next, in step 704, the system determines to send the data to the cache memory of the input/output channel that is the determined as the request source in step 701, then adds this input/output channel to the record of the destination input/output channels in step 705. [0182] In this embodiment, one data item can come to be sent to plural input/output channels. [0183] In detail, this record is written in the input/output channel number field in the data table as shown in FIG. 7. In the embodiment shown in FIG. 13, it is required that all the numbers of the input/output channels provided for the disk array system need to be recorded in this input/output channel number field with respect to one data block number set. [0184] When the reading from the disk drive, started in step 703, is not completed at this time, the system waits for the completion, then sends the read data to the target input/output channel so that the data is returned to the host computer and stored in the cache memory coupled to the input/output channel in step 706. [0185] It is also assumed here that the input/output channel has received a new write request from the host computer in step 707. [0186] In this embodiment, the input/output channel does not write the data in the cache memory immediately, but notifies the disk adapter of the received write request. [0187] The disk adapter checks in step 708 if there are cache memories of any other input/output channels, which have received the same data. [0188] When the check result is NO in step 709, the system goes to step 713. [0189] When the check result is YES, the system goes to step 710 to issue an invalidation request to the cache memories of all those input/output channels, which have received the same data. [0190] The invalidate command shown, for example, in FIG. 8 can be used as this invalidation request. [0191] After confirming the invalidation in step 711, the system deletes the invalidated cache memory's input/output channel number from the record of the input/output channel numbers in step 712. [0192] In step 713, the system permits writing for the input/output channel that has received the write request, so that the input/output channel receives write data from the host computer and writes the data in its cache memory. [0193] The system then sends the data written in the cache memory to the disk adapter from the input/output channel in step 714 and instructs the disk adapter to write back the data to the necessary disk drives. [0194] Consequently, inconsistency of modifying the same data by another input/output channel after that can be avoided. [0195] In the example shown in FIG. 13, while the first data request processing is more simplified than that in the example shown in FIG. 10, overhead occurs when in writing, since the cache memories of other input/output channels will be invalidated. [0196] Furthermore, when in writing, it is required that data is written in the cache memory of the input/output channel, as well as the written data is transferred to the target disk adapter from the input/output channel, then written back to the necessary disk drives. [0197] Furthermore, because the cache memories are invalidated as such, when the same data is requested from another input/output channel at this time, the data must be read again from the disk drive through the disk adapter. [0198] This last problem can be avoided, however, by employing the following example as shown in FIG. 14. [0199] [0199]FIG. 14 is a flowchart for controlling the cache memory of another embodiment of the invention. [0200] In FIG. 14, the processing in steps 801 to 806 are identical to those in steps 701 to 706 shown in FIG. 13. [0201] In other words, the system receives a data request in step 801. Also in this case, it is assumed that the requested data is not found in the cache memory of the input/output channel, as in the examples in FIGS. 10 and 13. [0202] In step 802, the system determines a disk drive that stores the requested data. [0203] In step 803, the system begins to read the data from the disk drive according to the determination in step 802. [0204] In this embodiment, the system is just expected to complete the reading of the data from the disk drive in or before step 806. [0205] In step 804, the system determines to send the data to the cache memory of the input/output channel that is the request source in step 801. The system then adds this input/output channel to the record of the destination input/output channels in step 805. [0206] In this embodiment, just like the example shown in FIG. 13, one data item can come to correspond to plural destination input/output channels. [0207] If the reading from the disk drive, started in step 803, is not completed yet at this time, the system waits for the completion in step 806, then sends the read data to the input/output channel so that the data is returned to the host computer and stored in the cache memory coupled to the input/output channel. [0208] In step 807, it is also assumed that the input/output channel has received a write request from the host computer. [0209] In FIG. 14, the system permits writing to the cache of the input/output channel in step 808, so that the data received from the host computer is written in the cache memory. [0210] In step 809, the system instructs the input/output channel to transfer the data to the disk adapter and writes the data back to the original data stored disk drive. [0211] Next, the disk adapter checks if the same data has been sent to any other input/output channels cache memory in step 810. [0212] When the check result is YES in step 811, the system goes to step 812. [0213] In step 812, the system instructs the disk adapter to send the data written in the disk drive to all of other input/output channels cache memories that store the old data so as to update the data. [0214] The update command as shown in FIG. 9 can be used for this data updating. [0215] Consequently, inconsistency can be avoided just like in the example shown in FIG. 13 even when another input/output channel updates the data hereafter. [0216] Even in the example shown in FIG. 14, the first data request processing is simplified more than that in FIG. 10. When in writing, however, some overhead occurs in the processing due to the updating of the cache memories of other input/output channels. [0217] Furthermore, when in write processing, the data must also be written in the cache memory of the input/output channel and transferred from the input/output channel to the disk adapter, then written back to the necessary disk drives. [0218] However, because the data in the cache memories of other input/output channels is updated with the written data in the write operation, the data is just read from one of those cache memories upon receiving a request to the same data from another input/output channel even after some write operations. [0219] According to the invention, plural cache memories are distributed among input/output channels and there is no need to couple input/output channels and disk adapters to a single cache memory where the concentrated couple ion has been required conventionally. Consequently, the invention makes it effective to configure a large scale disk array system by increasing the number of input/output channels and the number of disk adapters. [0220] Furthermore, each network switch used to couple an input/output channel to a disk adapter is just required to have a delay time shorter than the disk drive reading time in the disk adapter. The disk array system can thus configure a multi-step network easily and the scale can further be expanded effectively. [0221] Furthermore, because a cache memory is coupled directly to each input/output channel, the performance of the system including the number of input/output operations per second can be improved significantly when the data is stored in the cache memory. [0222] And, because each input/output channel is coupled directly to a disk adapter through network switches, even when data is to be read from a disk drive, the reading path is short and the data can be transferred fast. [0223] The foregoing invention has been described in terms of preferred embodiments. However, those skilled, in the art will recognize that many variations of such embodiments exist. Such variations are intended to be within the scope of the present invention and the appended claims. Patent CitationsCited PatentFiling datePublication dateApplicantTitleUS5459856 *Dec 3, 1992Oct 17, 1995Hitachi, Ltd.System having independent access paths for permitting independent access from the host and storage device to respective cache memoriesUS5689729 *Jul 13, 1995Nov 18, 1997Hitachi, Ltd.Storage subsystem having plurality of access paths permitting independent access to cache memory from host and independent access to the cache from rotating storage deviceUS5951655 *Jul 29, 1997Sep 14, 1999Hitachi, Ltd.External storage subsystem having independent access paths for permitting independent access from a host and a storage device to respective cache memoriesUS5987477 *Jul 11, 1997Nov 16, 1999International Business Machines CorporationParallel file system and method for parallel write sharingUS6185658 *Dec 17, 1997Feb 6, 2001International Business Machines CorporationCache with enhanced victim selection using the coherency states of cache linesUS6243795 *Aug 4, 1998Jun 5, 2001The Board Of Governors For Higher Education, State Of Rhode Island And Providence PlantationsRedundant, asymmetrically parallel disk cache for a data storage systemUS6351791 *Jun 25, 1998Feb 26, 2002International Business Machines CorporationCircuit arrangement and method of maintaining cache coherence utilizing snoop response collection logic that disregards extraneous retry responsesUS6378039 *Aug 31, 2000Apr 23, 2002Hitachi, Ltd.Storage subsystem which balances loads across a plurality of disk controllersUS6446237 *Aug 4, 1998Sep 3, 2002International Business Machines CorporationUpdating and reading data and parity blocks in a shared disk systemUS6574709 *Sep 30, 1999Jun 3, 2003International Business Machine CorporationSystem, apparatus, and method providing cache data mirroring to a data storage systemUS6587919 *Mar 19, 2002Jul 1, 2003Emc CorporationSystem and method for disk mapping and data retrievalUS6647461 *Jan 3, 2003Nov 11, 2003Hitachi, Ltd.Disk array controller, its disk array control unit, and increase method of the unitUS6647474 *Aug 20, 2002Nov 11, 2003Emc CorporationRemote data mirroring system using local and remote write pending indicatorsUS6671782 *Mar 29, 2000Dec 30, 2003International Business Machines CorporationMethod and apparatus for processing read requests in a shared disk systemUS6766414 *May 3, 2001Jul 20, 2004International Business Machines CorporationMethods, apparatus and system for caching dataUS6775739 *Apr 10, 1998Aug 10, 2004Emc CorporationMirrored drive performance using matching algorithmsUS6816891 *Sep 26, 2000Nov 9, 2004Emc CorporationNetwork file server sharing local caches of file access information in data processors assigned to respective file systemUS6832291 *Sep 16, 1997Dec 14, 2004Samsung Electronics Co., Ltd.Memory system for improving data input/output performance and method of caching data recovery informationUS6883064 *Feb 19, 2002Apr 19, 2005Hitachi, Ltd.Disk array controller comprising a plurality of disk array controlling units* Cited by examinerReferenced byCiting PatentFiling datePublication dateApplicantTitleUS7133976Jan 13, 2006Nov 7, 2006Hitachi, Ltd.Disk array device, method for controlling the disk array device and storage systemUS7139880Jul 28, 2003Nov 21, 2006Hitachi, Ltd.Disk array device, method for controlling the disk array device and storage systemUS7263580 *May 20, 2004Aug 28, 2007Hitachi, Ltd.Cache flush based on checkpoint timerUS7287125Oct 11, 2006Oct 23, 2007Hitachi, Ltd.Disk array device, method for controlling the disk array device and storage systemUS7383395 *Oct 27, 2004Jun 3, 2008Hitachi, Ltd.Storage deviceUS7487298Sep 5, 2007Feb 3, 2009Hitachi, Ltd.Disk array device, method for controlling the disk array device and storage systemUS7543123 *Nov 7, 2005Jun 2, 2009International Business Machines CorporationMultistage virtual memory paging systemUS7694081 *Jul 26, 2006Apr 6, 2010Hitachi, Ltd.Storage system and method for controlling storage systemUS7814270 *Jun 15, 2007Oct 12, 2010Hitachi, Ltd.Storage systems and methods of controlling cache memory of storage systemsUS7827364Feb 13, 2009Nov 2, 2010International Business Machines CorporationMultistage virtual memory paging systemUS8996959 *Dec 17, 2012Mar 31, 2015Samsung Electronics Co., Ltd.Adaptive copy-back method and storage device using sameUS9015272Feb 23, 2012Apr 21, 2015Denso CorporationMicrocomputerUS20040078517 *Jul 28, 2003Apr 22, 2004Hitachi, Ltd.Disk array device, method for controlling the disk array device and storage systemUS20050246488 *May 20, 2004Nov 3, 2005Hiroyasu KibaMethod and system for data processing for controlling a cache memoryUS20060047900 *Oct 27, 2004Mar 2, 2006Katsuya TanakaStorage deviceUS20070033342 *Oct 11, 2006Feb 8, 2007Seiji KanekoDisk array device, method for controlling the disk array device and storage systemUS20070106853 *Nov 7, 2005May 10, 2007International Business Machines CorporationMultistage virtual memory paging systemUS20070245089 *Jun 15, 2007Oct 18, 2007Kentaro ShimadaStorage systems and methods of controlling cache memory of storage systemsUS20070250665 *Jun 15, 2007Oct 25, 2007Kentaro ShimadaStorage systems and methods of controlling cache memory of storage systemsUS20070255899 *Jul 26, 2006Nov 1, 2007Hitachi, Ltd.Storage system and method for controlling storage systemUS20080016285 *Sep 5, 2007Jan 17, 2008Seiji KanekoDisk array device, method for controlling the disk array device and storage systemUS20090150625 *Feb 13, 2009Jun 11, 2009International Business Machines CorporationMultistage Virtual Memory Paging SystemUS20130159815 *Dec 17, 2012Jun 20, 2013Samsung Electronics Co., Ltd.Adaptive copy-back method and storage device using sameUS20170147207 *Nov 20, 2015May 25, 2017Arm Ltd.Non-volatile buffer for memory operations* Cited by examinerClassifications U.S. Classification711/114, 711/113, 711/E12.022, 711/E12.019International ClassificationG06F3/06, G06F11/20, G06F12/08Cooperative ClassificationG06F3/0613, G06F11/2007, G06F11/201, G06F12/0891, G06F12/0866, G06F11/2005, G06F3/0656, G06F3/0689, G06F11/2092European ClassificationG06F3/06A2P4, G06F3/06A6L4R, G06F3/06A4T2, G06F12/08B20, G06F12/08B12Legal EventsDateCodeEventDescriptionAug 16, 2002ASAssignmentOwner name: HITACHI, LTD., JAPANFree format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:SHIMADA, KENTARO;REEL/FRAME:013204/0963Effective date: 20020709Oct 23, 2009FPAYFee paymentYear of fee payment: 4Oct 30, 2013FPAYFee paymentYear of fee payment: 8RotateOriginal ImageGoogle Home - Sitemap - USPTO Bulk Downloads - Privacy Policy - Terms of Service - About Google Patents - Send FeedbackData provided by IFI CLAIMS Patent Services