Source: https://patents.google.com/patent/US20080005490A1/en
Timestamp: 2020-02-28 19:35:58
Document Index: 283250268

Matched Legal Cases: ['Application No. 2006', 'art 1', 'art 1', 'art 1', 'art 200', 'art 101']

US20080005490A1 - Storage control apparatus and method for controlling number of commands executed in storage control apparatus - Google Patents
Storage control apparatus and method for controlling number of commands executed in storage control apparatus Download PDF
US20080005490A1
US20080005490A1 US11/495,755 US49575506A US2008005490A1 US 20080005490 A1 US20080005490 A1 US 20080005490A1 US 49575506 A US49575506 A US 49575506A US 2008005490 A1 US2008005490 A1 US 2008005490A1
US11/495,755
US7685342B2 (en
Koji Iwamitsu
Bunitsu Ando
2006-05-31 Priority to JP2006152675A priority Critical patent/JP4901310B2/en
2006-05-31 Priority to JP2006-152675 priority
2006-07-31 Assigned to HITACHI, LTD. reassignment HITACHI, LTD. ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: ANDO, BUNITSU, AOYAMA, HIDEKAZU, IWAMITSU, KOJI, SHIRAKI, SHINJIRO
2008-01-03 Publication of US20080005490A1 publication Critical patent/US20080005490A1/en
2010-03-23 Publication of US7685342B2 publication Critical patent/US7685342B2/en
A storage control apparatus of the present invention controls the number of multiple commands issued from a host machine without shutting down the host machine. A communication port of the storage control apparatus carries out communications with the hosts in accordance with the iSCSI protocol. Command processing resources are managed for each communication port. A resource allocation control part calculates the number of commands capable of being received on the basis of the remaining amount of command processing resources inside shared port resources, a change in the number of commands received from a host, communication delay time, and the state of execution of a command issued from a host or the like. A MaxCmdSN is calculated by adding the results of command processing by a command execution part and the receivable number calculated by the resource allocation control part to the value of the latest CmdSN received from a host. The storage control apparatus adds the MaxCmdSN to a transmission frame and transmits it to the host.
This application relates to and claims priority from Japanese Patent Application No. 2006-152675, filed on May 31, 2006, the entire disclosure of which is incorporated herein by reference.
Accordingly, an object of the present invention is to provide a storage control apparatus and a method for controlling the number of commands executed in a storage control apparatus constituted such that the number of multiple commands issued by a host machine can be properly controlled without stopping the host machine or changing the system configuration. Another object of the present invention is to provide a storage control apparatus and a method for controlling the number of commands executed in a storage control apparatus constituted such that situations in which a storage control apparatus notifies a host machine of a QueueFull state can be held in check, and performance degradation can be prevented by virtue of adjusting the receivable number of commands that a storage control apparatus notifies to a host machine corresponding to the communication delay time between the storage control apparatus and the host machine. Additional objects of the present invention should become clear from the disclosures of the embodiments to be described hereinbelow.
An embodiment of the present invention will be explained below based on the figures. In this embodiment, as will be explained hereinbelow, command processing resources of a storage control apparatus are respectively managed in communication port units, and a command processing resource allocated to each host is dynamically managed inside the storage control apparatus based on changes in the number of commands issued from a host and arrival times.
The notification part 1D notifies the respective hosts 2 via the communication port 1A of the results of command processing executed by the command execution part 1B, and the command receivable number calculated by the resource allocation control part 1E. The command receivable number is the number of commands capable of being received from this host 2, and a storage control apparatus 1 can receive and process a command receivable number's worth of commands from a host 2 (hereinafter, command receivable number may be abbreviates as “receivable number”).
The remaining amount determining function 1E3 compares the amount of command processing resources PR inside the shared port resources 1F against a prescribed threshold value, and outputs this determination result. As will be described in the embodiment to be explained hereinbelow, the prescribed threshold value can be adjusted based on the receivable number notified to the hosts 2, and the maximum number of processable commands for this communication port 1A. The maximum number of processable commands for this communication port 1A signifies the maximum amount of command processing resources PR capable of being managed by the shared port resources 1F of this communication port 1A. In the following explanation, there will be times when the maximum number of commands capable of being processed in each communication port 1A will be called “command processable number”.
FIG. 2 is a schematic diagram showing the overall constitution of a storage system comprising a storage control apparatus 10 according to this embodiment. This storage system, for example, can be constituted comprising at least one or more storage control apparatuses 10, a plurality of hosts 20, and at least one or more switches 30. Further, as will be explained below, the storage control apparatus 10 comprises a controller 100 having a communication port 101, and a storage part 200 having a storage device 210.
The control tables 150A, 150B, for example, can be provided in a memory device, such as a rewritable non-volatile memory. The control tables 150A, 150B correspond to the respective communication ports 101A, 101B. Since the control tables 150A, 150B have the same structure, hereinbelow they will be called “control table 150” except when it is necessary to distinguish between them.
FIG. 4 is a schematic diagram showing a by-port command processing management table T1. The by-port command processing management table T1 is a table for managing the command processing resources in each communication port 101A, 101B (called “ports 101” hereinbelow, except when it is necessary to distinguish between them). As shown in FIG. 4, the storage control apparatus 10 of this embodiment is capable of processing a maximum number n of commands in each part 101. The command processing information shown in FIG. 4 pertains to the command processing resources, and command processing information is set one at a time for each command received from a host 20.
FIG. 5 is a schematic diagram showing a by-port number of remaining command processing management resources table T2. Hereinafter, this table is referred to as the number remaining table T2 in some cases. The number remaining table T2 is for managing the amount of remaining command processing resources (command processing information) being managed by each port 101. The remaining amount is the unused command processing resources, which have not been allocated to any hosts 20. In the figures, the remaining number (remaining amount) of command processing resources in the respective ports 101 will at times be displayed as “RPR”.
FIG. 6 is a schematic diagram showing a by-port number of executed commands table T3. On occasion, this table T3 will be abbreviated as number of executed commands table T3 below. The number of executed commands table T3 is for managing the number of commands being executed in each port 101, that is, the number of commands in the process of being executed. In other words, the number of executed commands table T3 manages how many commands received via the associated communication ports 101 are currently being executed. The number of commands executed by-port will be abbreviated at times as “ENp” in the figures.
The steady state flag is information showing the status of the number of executed commands (ENc) relative to a host 20. That is, the steady state flag is information showing the command issuing status of a host 20. When it is determined that a host 20 is in a steady state, “1” is set in the steady state flag. When it is determined that a host 20 is not in a steady state, “0” is set in the steady state flag. A steady state indicates a state wherein the value of the number of executed commands (ENc) executed for a host 20 is stable. As a state other than a steady state, there is a decreasing state. A decreasing state indicates a state wherein the value of the number of executed commands (ENc) executed for a host 20 has decreased from the steady state. The steady state flag is used in the embodiments to be explained hereinbelow together with the number of executed commands history T41.
Next, the operation of a storage control apparatus 10 according to this embodiment will be explained while referring to a flowchart. The flowcharts show an overview of the processing, and there will be instances when this processing will differ with an actual program. Furthermore, the term step will be abbreviated as “S”.
A second embodiment will be explained based on FIG.'S 13 through 15. The following embodiments, to include this embodiment, correspond to variations of the above-described first embodiment. In this embodiment, the end phase of command issuing is predicted based on the number of executed commands ENc, which were issued from a host 20, and the command processing resources allocated to this host 20 are returned to the resource pool (RPR) at the beginning phase.
FIG. 14 is a diagram schematically showing the change in the number of executed commands ENc. When a first command (#1) is received from a host 20, the number of received commands of the storage control apparatus 10 is “1”. Since the processing of this first command is not complete, the number of responded commands is “0”. Therefore, at the point in time at which the storage control apparatus 10 received the first command, the number of executed commands ENc constituted 1 (ENc=number of received commands−number of responded commands).
If it is supposed that the storage control apparatus 10 received the first through the fourth (#4) commands during the period up until the processing results of the first command were notified to the host 20, the number of executed commands ENc will steadily increase from “1”→“2”→“3”→“4”. This state can also be called the increasing state.
When the processing of the initially received commands (#1 and so forth) is complete, and the time for making a notification to the host 20 arrives, the value of the number of executed commands ENc is stable at “4” in the example shown in the figure. When the value of the number of commands being executed ENc continues more than a prescribed number of times, showing the same value (“4” in this example), the storage control apparatus 10 can determine that processing has transitioned from the increasing state to the steady state.
A third embodiment will be explained based on FIG'S. 16 through 18. In this embodiment, when the arrival of a command from a host 20 is delayed, the command processing resources allocated to this host 20 are maintained without returning them to the resource pool, until either the preceding executed-number-0 time period T0, or a prescribed time RT has elapsed.
FIG. 17 is a flowchart showing the process for continuing to secure command processing resources when there is communication delay time between a host 20 and the storage control apparatus 10. First of all, the storage control apparatus 10 determines whether or not the number of executed commands ENc changed from “1” to “0” (S71).
When the number of executed commands ENc changed from “1” to “0” (S71: YES), the storage control apparatus 10 stores the time at which ENc=0 as the start time TS of the executed-number-0 time period T0 (S72). Next, the storage control apparatus 10 compares the time period T0, which was measured the preceding time in S76 to be explained hereinbelow, against a prescribed time RT (S73). That is, the storage control apparatus 10 determines whether or not the preceding time period TO is within a prescribed time RT (S73). When the preceding time period TO during which the number of executed commands ENc was 0 is less than the prescribed time RT (S73: YES), the storage control apparatus 10 terminates processing without doing anything. By contrast, when the preceding time period To has attained the prescribed time RT (S73: NO), the storage control apparatus 10 returns all the command processing resources allocated to the host 20 for which ENc=0 to the resource pool (RPR) (S74).
When the number of executed commands ENc has not changed from “1” to “0” (S71: NO), the storage control apparatus 10 determines whether or not the number of executed commands ENc has changed from “0” to “1” (S75). The changing of the ENc from “0” to “1” occurs when the executed-number-0 time period To has ended. Accordingly, the storage control apparatus 10 updates the value of the time period TO in table T4 (S76). That is, the storage control apparatus 10 calculates the latest time period TO by subtracting the start time TS from the current time, and registers it in table T4.
A fourth embodiment will be explained on the basis of FIG'S. 19 and 20. In this embodiment, an order of priority is set in advance for the hosts 20, and command processing resources, which are managed by ports 101, are distributed in accordance with this order of priority.
said control part:
(2) respectively notifying said host machines of a receivable number, which indicates the number of commands capable of being received from said host machines, based on this amount of allocated said command processing resources; and
(3) respectively controlling the amount of said command processing resources allocated to said host machines on the basis of a communication delay time between said host machines and said communication ports.
said host machines issue said commands by making sequence numbers, for identifying the issuing order of the commands, correspond to said commands, and
said control part calculates said receivable number by adding to said sequence number the amount of said command processing resources allocated to said host machine, when notifying said host machine of the processing results of said command received from said host machine.
a plurality of communication ports, which are respectively connected via the iSCSI protocol to a plurality of host machines, each of which issues commands;
a control part, which is respectively connected to said superordinate communication part, said subordinate communication part, said cache memory, and said control memory, respectively processes said commands received by said superordinate communication part by way of said communication port from said host machines, and notifies said host machines of results of the processing from said superordinate communication part via said communication port, wherein
(2) respectively calculates a receivable number, which indicates the number of commands capable of being received from said prescribed host machines, by adding a sequence number, for showing the command issuing order notified from said prescribed host machine, to the amount of said command processing resources allocated to said prescribed host machines, and respectively notifies said prescribed host machines of this calculated receivable number; and
(3) returns to said shared port resources said command processing resources to be allocated to said prescribed host machines, in accordance with the decrease in the number of said commands issued from said prescribed host machines.
receiving sequence numbers indicating commands from said host machine, and the order of the commands;
storing said sequence numbers;
a control part, which respectively processes commands received from said host machines via said communication ports, and which respectively transmits the results of processing to said host machines, wherein said control part:
(2) respectively notifies said host machines of a receivable number, which indicates the number of commands capable of being received from said host machines based on the amount of said allocated command processing resources.
US11/495,755 2006-05-31 2006-07-31 Storage control apparatus and method for controlling number of commands executed in storage control apparatus Active 2028-07-04 US7685342B2 (en)
JP2006152675A JP4901310B2 (en) 2006-05-31 2006-05-31 Storage control device and command execution number control method for storage control device
JP2006-152675 2006-05-31
US20080005490A1 true US20080005490A1 (en) 2008-01-03
US7685342B2 US7685342B2 (en) 2010-03-23
ID=38457564
US11/495,755 Active 2028-07-04 US7685342B2 (en) 2006-05-31 2006-07-31 Storage control apparatus and method for controlling number of commands executed in storage control apparatus
US (1) US7685342B2 (en)
EP (1) EP1862893A3 (en)
JP (1) JP4901310B2 (en)
US20090228610A1 (en) * 2008-03-10 2009-09-10 Fujitsu Limited Storage system, storage apparatus, and control method for storage system
US20130166797A1 (en) * 2011-12-27 2013-06-27 Hitachi Ltd. Storage apparatus and method for controlling same
CN105511882A (en) * 2015-12-11 2016-04-20 曙光信息产业(北京)有限公司 Realizing method and device for tasks of CPMD (car-parrinello molecular dynamics)
US20170083248A1 (en) * 2015-09-22 2017-03-23 Kabushiki Kaisha Toshiba Memory system that selects data to be transmitted from a data buffer through a port
JP4654273B2 (en) 2008-07-14 2011-03-16 富士通株式会社 Storage device and control device
FR2954539B1 (en) * 2009-12-23 2014-08-29 Thales Sa Method and device for detecting error transfers for microcontroller or microprocessor to guarantee partitioning.
US8838871B2 (en) 2012-04-09 2014-09-16 Dell Products L.P. Methods and systems for virtualization of storage services in an integrated chassis
CN103530131A (en) * 2013-10-28 2014-01-22 曙光信息产业(北京)有限公司 Method and device for implementing Gaussian calculating task
JPH11212900A (en) * 1998-01-22 1999-08-06 Fujitsu Ltd System controller
2006-05-31 JP JP2006152675A patent/JP4901310B2/en active Active
2006-07-31 US US11/495,755 patent/US7685342B2/en active Active
2006-11-10 EP EP06255784A patent/EP1862893A3/en not_active Withdrawn
US9843532B1 (en) * 2010-09-22 2017-12-12 Amdocs Development Limited System, method, and computer program for managing distribution of a finite amount of resources
US8799534B2 (en) * 2011-12-27 2014-08-05 Hitachi, Ltd. Storage apparatus and method for controlling same
TWI511157B (en) * 2012-08-23 2015-12-01 Apple Inc Efficient enforcement of command execution order in solid state drives
US7685342B2 (en) 2010-03-23
EP1862893A3 (en) 2009-11-18
EP1862893A2 (en) 2007-12-05
JP2007323356A (en) 2007-12-13
JP4901310B2 (en) 2012-03-21
US7120742B2 (en) 2006-10-10 Storage system having a plurality of interfaces
JP4933284B2 (en) 2012-05-16 Storage apparatus and load balancing method
Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:SHIRAKI, SHINJIRO;IWAMITSU, KOJI;AOYAMA, HIDEKAZU;AND OTHERS;REEL/FRAME:018114/0136