Patent ID: 12197784

DETAILED DESCRIPTION

Below, embodiments of the present disclosure will be described in detail and clearly to such an extent that one skilled in the art may implement these embodiments. In the following description, specific details such as detailed components and structures are merely provided to assist the overall understanding of embodiments of the present disclosure. Therefore, it should be apparent to those skilled in the art that various changes and modifications of the embodiments described herein may be made without departing from the scope and spirit of the disclosure. The terms described in the specification are terms defined in consideration of the functions in the present disclosure and are not limited to a specific function.

Components that are described in the detailed description with reference to the terms “circuit”, “block”, etc. may be implemented with software, hardware, or a combination thereof. For example, the software may be a machine code, firmware, an embedded code, and application software. For example, the hardware may include an electrical circuit, an electronic circuit, a processor, a computer, integrated circuit cores, a pressure sensor, an inertial sensor, a micro electro mechanical system (MEMS), a passive element, or a combination thereof.

FIG.1is a block diagram illustrating a storage system according to an embodiment of the present disclosure. Referring toFIG.1, a storage system SS may include a hypervisor HV, a plurality of hosts11to1n, and a storage device100. In an embodiment, the storage system SS may be included in at least one of various information processing devices such as a personal computer (PC), a laptop computer, a server, a workstation, a smartphone, and a tablet PC.

Each of the hypervisor HV and the plurality of hosts11to1nmay access the storage device100. In an embodiment, the hypervisor HV and the plurality of hosts11to1nmay be single or multi-core processors respectively included in different computing nodes. For example, the computing nodes could be located on a same computer or on different computers that communicate with one another using a computer network. Alternatively, at least some of the hypervisor HV and the plurality of hosts11to1nmay be different processors included in the same computing node. However, the present disclosure is not limited thereto. The hypervisor HV and the plurality of hosts11to1nmay be processors configured to process different applications or may be different virtual machines driven in a computing system. The hypervisor HV may run one or more virtual machines. A virtual machine may be a virtual environment that functions as virtual computer system with its own central processing unit (CPU), memory, network interface, and storage. The hypervisor HV may provision resources of a system to one or more of the virtual machines.

The storage device100may include a storage controller110(e.g., a controller circuit) and a nonvolatile memory device120. The storage device100may be configured to support a multi-host (e.g., multiple hosts) or a multi-tenant (e.g., multiple tenants). That is, the storage device100may operate under control of a plurality of hosts. For example, based on commands issued from the hypervisor HV and the plurality of hosts11to1n, the storage controller110may store data in the nonvolatile memory device120or may read data from the nonvolatile memory device120.

In an embodiment, the storage device100may communicate with the hypervisor HV and the plurality of hosts11to1nbased on a PCI-express (Peripheral Component Interconnect express) interface or a PCI-express based NVMe (Nonvolatile Memory Express) interface.

The storage device100may allocate a resource in response to a request from each of the plurality of hosts11to1n. For example, the storage controller110may allocate resources of the same size or different sizes to the first to n-th hosts11to1n, respectively. In an embodiment, a resource may include at least one of a nonvolatile memory space of the storage device100, a volatile memory space of the storage device100, and an internal command queue capacity of the storage device100. For example, the resource may include one or more command queues of the storage device100or slots of a given one of the command queues. For brevity of description, below, the expression “allocation of a resource to a specific host” may be interpreted as a resource that is occupied by the specific host. For example, a resource that is allocated to the first host11may be referred to as a resource that the first host11occupies.

The storage controller110may include a performance manager111. The performance manager111may manage the allocation of resources to the first to n-th hosts11to1n. For example, the performance manager111may appropriately allocate a resource to each of the first to n-th hosts11to1nsuch that the quality of service required by each of the first to n-th hosts11to1nis satisfied. Each of the first to n-th hosts11to1nmay have different quality of service requirements. For example, if a first host has a higher quality of service (QoS) requirement than a second host, then the first host may need more of the resources than the second host to satisfy its higher QoS.

In an embodiment, the hypervisor HV controls the allocation of resources to the first to n-th hosts11to1n. The performance manager111may set a threshold value for the amount of resource to be allocated to each of the first to n-th hosts11to1nunder control of the hypervisor HV. That is, the performance manager111may set a resource occupancy threshold for each of the first to n-th hosts11to1n. For example, the performance manager111may set a threshold value of a resource occupancy ratio for each of the first to n-th hosts11to1n. Alternatively, the performance manager111may set a threshold value for the amount of resource that each of the first to n-th hosts11to1noccupies.

In an embodiment, the performance manager111may stop the allocation of a resource to a host, in which an allocated resource amount reaches a corresponding threshold value, from among the first to n-th hosts11to1n. For example, when the amount of resource allocated to the first host11reaches a first threshold value, the performance manager111may stop the additional allocation of a resource to the first host11. That is, the performance manager111may temporarily stop the allocation of a resource to the first host11until the amount of resource allocated to the first host11decreases to less than the first threshold value or equal to the first threshold value. For example, if the first threshold value is 5, the first host11is already receiving 5 resources, and the first host11just requested a sixth resource, the performance manager111will not allocate the sixth resource to the first host11.

In an embodiment, the hypervisor HV is configured request information about a resource occupancy threshold of each of the first to n-th hosts11to1n.

In an embodiment, the hypervisor HV is configured to request changing a resource occupancy threshold of each or one or more of the first to n-th hosts11to1n. How the storage device100operates in response to a request of the hypervisor HV will be described in detail with reference toFIGS.5A,5B, and7A to7C. In an embodiment, since a resource occupancy threshold is set for each of the first to n-th hosts11to1n, a resource may be prevented from being monopolized by one or more of the hosts. A configuration and an operation of the performance manager111will be described in detail with reference to the following drawings.

FIG.2is a block diagram illustrating a storage controller ofFIG.1in detail according to an example embodiment. Referring toFIGS.1and2, the storage controller110may include the performance manager111, a host interface layer112(e.g., an interface circuit), a nonvolatile memory device interface113, a volatile memory device114, and a processor115. The performance manager111, the host interface layer112, the nonvolatile memory device interface113, the volatile memory device114, and the processor115may be connected with each other through a bus.

The storage controller110may communicate with the hypervisor HV and the first to n-th hosts11to1nthrough the host interface layer112. In an embodiment, the host interface layer112may include at least one of various host interfaces such as a PCI-express (Peripheral Component Interconnect express) interface, an NVMe (nonvolatile memory express) interface, a SATA (Serial ATA) interface, a SAS (Serial Attached SCSI) interface, and a UFS (Universal Flash Storage) interface. For brevity of description, below, it is assumed that the host interface layer112communicates with the hypervisor HV and the first to n-th hosts11to1nbased on the PCI-express interface.

In an embodiment, the host interface layer112is configured to fetch a command from each of the hypervisor HV and the first to n-th hosts11to1n. In this embodiment, the fetched command may be stored in an internal command queue (i.e., may be enqueued). Below, for brevity of description, a command issued from the first to n-th hosts11to1nmay be referred to as a “host command”. For example, internal command queue may be located within the storage device100.

The storage controller110may communicate with the nonvolatile memory device120through the nonvolatile memory device interface113(e.g., an interface circuit). For example, the nonvolatile memory device interface113may be a NAND interface. In an embodiment, a memory space of the nonvolatile memory device120may be allocated to the first to n-th hosts11to1nunder control of the performance manager111. In this embodiment, each of the first to n-th hosts11to1nmay occupy a nonvolatile memory space allocated thereto.

The volatile memory device114may be used as a working memory, a buffer memory, or a cache memory of the storage controller110. For example, the volatile memory device114may be implemented with a static random access memory (SRAM) or a dynamic random access memory (DRAM). In an embodiment, a memory space of the volatile memory device114may be allocated to the first to n-th hosts11to1nunder control of the performance manager111. In this embodiment, each of the first to n-th hosts11to1nmay occupy a volatile memory space allocated thereto.

The processor115may control an overall operation of the storage controller110. For example, the processor115may execute various applications (e.g., a flash translation layer (FTL)) on the storage controller110.

The performance manager111may manage the allocation of resources to the first to n-th hosts11to1n. For example, the performance manager111may allocate at least one of the internal command queue, the volatile memory space, and the nonvolatile memory space. Below, an embodiment in which the performance manager111allocates the internal command queue (not illustrated) will be described. However, the present disclosure is not limited thereto. For example, the performance manager111may manage the allocation of at least one or more of various resources, such as the internal command queue, the volatile memory space, and the nonvolatile memory space, to each of the first to n-th hosts11to1n.

In an embodiment, the performance manager111may be implemented in the form of hardware, software, or a combination thereof. For example, at least a part of the performance manager111may be included in the storage controller110in the form of a separate circuit, device, or chip. Also, at least a part of the performance manager111may be implemented in the form of a software module that is loaded to the volatile memory device114by the processor115. That is, for brevity of description, the performance manager111is illustrated inFIG.2as a separate component, but all or some of the above functions of the performance manager111may be implemented (or included) in one or more of any other components.

FIG.3is a block diagram for describing an operation of a storage system according to an embodiment of the present disclosure. Referring toFIGS.1to3, the storage system SS may include the hypervisor HV, the first to n-th hosts11to1n, and the storage controller110. The storage controller110may include a command fetcher CF, an internal command queue ICQ, the performance manager111, and a command parser CP. For brevity of description, the hypervisor HV and the first to n-th hosts11to1nare described above, and thus, additional description will be omitted to avoid redundancy.

In an embodiment, the command fetcher CF and the internal command queue ICQ are included in the host interface layer112.

The command fetcher CF may fetch commands issued from the hypervisor HV and the first to n-th hosts11to1n. That is, the command fetcher CF may fetch a command from a submission queue of each of the hypervisor HV and the first to n-th hosts11to1n. The command fetcher CF may provide the internal command queue ICQ with the commands (i.e., host commands) fetched from the first to n-th hosts11to1n.

The internal command queue ICQ may store the fetched host commands. In an embodiment, among the host commands stored in the internal command queue ICQ, host commands fetched from the first host11may be referred to as “first host commands HC1” (marked by a horizontal stripe), host commands fetched from the second host12may be referred to as “second host commands HC2” (marked by a vertical stripe), and host commands fetched from the third host13may be referred to as “third host commands HC3” (marked by a cross stripe). As in the above description, host commands fetched from the n-th host1nmay be referred to as “n-th host commands HCn”.

In an embodiment, a capacity of the internal command queue ICQ may be restricted. That is, the number of host commands capable of being stored in the internal command queue ICQ may be restricted to a certain amount. Below, for descriptive convenience, it is assumed that a maximum of 20 host commands are capable of being stored in the internal command queue ICQ. However, the present disclosure is not limited thereto.

The command parser CP may parse host commands stored in the internal command queue ICQ. For example, the command parser CP may process the host commands stored in the internal command queue ICQ in a first-in first-out (FIFO) manner. However, the present disclosure is not limited thereto. For example, the command parser CP may process the stored host commands based on various manners determined in compliance with an internal policy of the storage device100.

The performance manager111may monitor a status of the internal command queue ICQ. For example, the performance manager111may monitor the total number of commands stored the internal command queue ICQ. Alternatively, the performance manager111may monitor the number of host commands corresponding to each of the first to n-th hosts11to1nfrom the internal command queue ICQ. For example, the performance manager111may determine which of the host commands currently stored in the internal command queue ICQ correspond to each of the first to n-th hosts11to1nand generate corresponding counts.

The performance manager111may control the command fetcher CF based on a monitoring result. For example, the performance manager111may generate the monitoring result from monitoring the number of the host commands. The command fetcher CF may fetch a host command from one of the first to n-th hosts11to1nunder control of the performance manager111.

In an embodiment, in the case where the internal command queue ICQ is full, that is,20host commands are stored in the internal command queue ICQ, the performance manager111may determine to stop fetching commands from all the hosts11to1n. For example, the performance manager111may determine to stop fetching commands until the number of host commands stored in the internal command queue ICQ decreases to less than20(i.e., until at least some of the stored commands are processed). The command fetcher CF may stop fetching host commands from the first to n-th hosts11to1nin response to the determination of the performance manager111.

The performance manager111may set a resource occupancy threshold for each of the first to n-th hosts11to1n. For example, the performance manager111may set an internal command queue occupancy threshold for each of the first to n-th hosts11to1n. That is, the performance manager111may set first to n-th threshold values to the numbers of first to n-th host commands HC1to HCn, respectively. Below, for brevity of description, it is assumed that the first threshold value is “8”, the second threshold value is “6”, and the third threshold value is “1”, but the present disclosure is not limited thereto. In an embodiment, the performance manager111set the internal command queue occupancy thresholds based on the individual and possibly different QoS requirements of each of the first to n-th hosts11to1n. For example, these QoS requirements could be initially stored in the storage device100or sent to the storage device100from each of the hosts.

For example, the first to n-th threshold values may refer to internal command queue occupancy thresholds respectively corresponding to the first to n-th hosts11to1n. That is, the first to n-th threshold values may refer to threshold values respectively associated with the numbers of first to n-th host commands HC1to HCn stored in the internal command queue ICQ. As in the above description, the first to n-th threshold values may refer to various resource occupancy thresholds respectively corresponding to the first to n-th hosts11to1n.

In an embodiment, the performance manager111may include a resource occupancy threshold table ROTT. The resource occupancy threshold table ROTT may store resource occupancy thresholds (e.g., internal command queue occupancy thresholds) of the first to n-th hosts11to1n. Below, the description will be provided assuming the resource occupancy threshold table ROTT stores internal command queue occupancy thresholds respectively corresponding to the first to n-th hosts11to1n. However, the present disclosure is not limited thereto. For example, the resource occupancy threshold table ROTT may store various resource occupancy thresholds such as a volatile memory space occupancy threshold, a nonvolatile memory space occupancy threshold, and an internal command queue occupancy threshold corresponding to each host.

For example, the resource occupancy threshold table ROTT may store threshold values respectively associated with the numbers of first to n-th host commands HC1to HCn stored in the internal command queue ICQ. That is, the resource occupancy threshold table ROTT may store the first threshold value (e.g., 8) for the first host11, the second threshold value (e.g., 6) for the second host12, and the third threshold value (e.g., 1) for the third host13.

The performance manager111may determine whether to fetch a command from each host based on the set threshold value. For example, when the number of first host commands HC1stored in the internal command queue ICQ is more than or equal to the first threshold value (e.g., 8) (i.e., in the case where the number of first host commands HC1reaches the first threshold value), the performance manager111may determine to stop fetching a command from the first host11. In this case, the command fetcher CF may stop fetching host commands from the first host11in response to the determination of the performance manager111. Operations that the performance manager111and the command fetcher CF perform based on the second to n-th threshold values are similar to those described above, and thus, additional description will be omitted to avoid redundancy. In an embodiment, each of the hosts has its own unique host identifier (ID) and the resource occupancy threshold table ROTT include an entry for each of the host IDs that indicates a threshold value for a corresponding host.

The internal command queue ICQ may be monopolized by a specific host. For example, 20 first host commands HC1may be stored in the internal command queue ICQ. In this case, because host commands are not fetched from the remaining hosts (i.e., the second to n-th hosts12to1n), the quality of service (QoS) perceived by the remaining hosts may decrease.

However, embodiments of the present disclosure may prevent the internal command queue ICQ from being monopolized. An upper limit of the amount of resource that each host occupies the internal command queue ICQ may be restricted by the performance manager111. For example, in the case where the number of first host commands HC1is restricted to a maximum of 8, the internal command queue ICQ may be prevented from being monopolized by the first host11.

In an embodiment, the hypervisor HV may issue various types of commands. Commands that the hypervisor HV issues will be described in detail with reference toFIGS.5A and5B.

In an embodiment, the performance manager111is configured to dynamically update the threshold values of the hosts. An embodiment in which threshold values are dynamically updated will be described in detail with reference toFIG.6.

FIG.4is a flowchart illustrating an operation of a storage device ofFIG.3according to an example embodiment. Referring toFIGS.3and4, in operation S100, the storage device100sets the first to n-th threshold values of the first to n-th hosts11to1n.

In an embodiment, the storage device100sets a threshold value for each of the first to n-th hosts11to1n, under control of the hypervisor HV. For example, the storage device100may set the first to n-th threshold values for the first to n-th hosts11to1n, respectively, based on a command issued from the hypervisor HV. For example, a command may be output from the hypervisor HV to the storage device100that includes a host ID of one of the hosts and a threshold value for the one host.

In operation S110, the storage device100sequentially fetches host commands from the first to n-th hosts11to1n. For example, the command fetcher CF may sequentially fetch host commands from the first to n-th hosts11to1n. For example, the storage device100could fetch a host command from the first host11and then fetch a host command from the second host12or vice versa.

In an embodiment, an order in which the command fetcher CF fetches host commands from the first to n-th hosts11to1nmay be determined based on a round robin or weighted round robin manner. For example, when round robin is used, a single command or a same number of commands could be fetched in turn sequentially from each of the first to n-th hosts11to1nand after a command is fetched from the last host1n, the next fetch would be with respect to the first host11. For example, when weighted round robin is used, more commands may be fetched from certain hosts such as those having a higher QoS requirement. For example, if the first host11has a higher QoS requirement than the other hosts, two commands could be fetched from the first host11and then one command could be next fetched in turn sequentially from each of hosts12to1n, and the process may be repeated by again fetching two commands from the first host11.

In operation S120, the storage device100stores the fetched host commands in the internal command queue ICQ. For example, the command fetcher CF may provide the internal command queue ICQ with the host commands fetched from the first to n-th hosts11to1n. The internal command queue ICQ may store the first to n-th host commands HC1to HCn.

In operation S130, the storage device100stops (or pauses) fetching a command from a host corresponding to host commands which are stored in the internal command queue ICQ with greater or equal number to a threshold value. That is, when the number of k-th host commands HCk (k being an integer of 1 or more and n or less) stored in the internal command queue ICQ is more than or equal to the k-th threshold value, the storage device100may stop fetching a command from the k-th host1k. Accordingly, the internal command queue ICQ may be prevented from being monopolized by the k-th host1k. For example, if the threshold value of the first host11is 5 and there are already 5 or more host commands in the internal command queue ICQ from the host11, even though the command fetcher CF is scheduled to next fetch another command from the host11, the command fetcher CF skips this next fetch.

In an embodiment, a threshold value for the number of host commands may be set or changed under control of the hypervisor HV.

In an embodiment, a threshold value for the number of host commands is dynamically updated by the storage device100. For example, the storage device100may appropriately change the set threshold values even though a separate command is not issued from the hypervisor HV. For example, the storage device100may change threshold values of one or more of the hosts, independently from the hypervisor HV. An operation in which a threshold value is dynamically updated by the storage device100will be described in detail with reference toFIG.6.

FIGS.5A and5Bare block diagrams illustrating how a storage controller operates in response to a request of a hypervisor ofFIG.3.

Referring toFIGS.3and5A, the hypervisor HV may request resource status information from the storage controller110. For example, the hypervisor HV may issue a command for requesting the resource status information of the storage controller110.

In an embodiment, in the case where the hypervisor HV and the storage controller110communicate based on the PCI-express interface, the request of the hypervisor HV may be implemented with a “get features” command.

The storage controller110may return the resource status information in response to the request of the hypervisor HV. For example, the storage controller110(in more detail, the command fetcher CF) may fetch the command issued by the hypervisor HV. The storage controller110may return the resource status information in response to the fetched command. For example, the hypervisor HV may issue a command for requesting the resource status information, the storage controller110may fetch the issued command, the storage controller110may execute the fetched command to determine which one of the hosts to provide the resource status information for, and the storage controller110may provide the corresponding resource status information to the hypervisor HV.

In an embodiment, the storage controller110may return, to the hypervisor HV, information about a total amount of the internal command queue ICQ and/or information about a threshold value allocated for each host. For example, the total amount may indicate how many total commands are stored in the internal command queue ICQ or the total amount of commands stored for each of the hosts in the internal command ICQ. However, the present disclosure is not limited thereto. For example, the storage controller110may further return information about occupancy thresholds and total amounts of various resources such as a nonvolatile memory space and a volatile memory space.

Referring toFIGS.3and5B, the hypervisor HV may request the storage controller110to update resource occupancy threshold information. That is, the hypervisor HV may issue a command for requesting the storage controller110to update the resource occupancy threshold information. For example, the hypervisor HV may request the storage controller110to update the internal command queue occupancy threshold (i.e., the first threshold value) of the first host11so as to be changed to “9” and to update the internal command queue occupancy threshold (i.e., the second threshold value) of the second host12so as to be changed to “5”. For example, the hypervisor HV may send a command to the storage controller110including the host ID of the first host11and a changed threshold value so that the storage controller110can update the corresponding host threshold value based on the received changed threshold value. For example, the hypervisor HV may send a command to the storage controller110including several tuples, where each tuple includes a host ID of a distinct one of the hosts and a changed threshold value so that the storage controller110can update the host threshold values of the corresponding hosts based on the received changed threshold values.

In an embodiment, in the case where the hypervisor HV and the storage controller110communicate based on the PCI-express interface, the request of the hypervisor HV may be implemented with a “set features” command.

The storage controller110may update the resource occupancy threshold in response to the request of the hypervisor HV. For example, the performance manager111may change the first threshold value to “9” and may change the second threshold value to “5”.

FIG.6is a diagram illustrating an operation of a storage device ofFIG.1. An operation in which the performance manager111dynamically changes set resource occupancy thresholds will be described with reference toFIGS.1,3, and6. For brevity of description, an operation in which internal command queue occupancy thresholds (i.e., the first to third threshold values) respectively corresponding to the first to third hosts11to13are dynamically changed will be described with reference toFIG.6, but the present disclosure is not limited to a resource kind or the number of hosts.

The performance manager111may monitor an internal command queue occupancy status and an internal command queue occupancy threshold corresponding to a specific host. The performance manager111may change the set internal command queue occupancy threshold based on the internal command queue occupancy status and the internal command queue occupancy threshold monitored for each host. For example, at second to fifth time points t2to t5, the performance manager111may monitor an amount by which each host occupies the internal command queue ICQ. However, the present disclosure is not limited to the number of time points when the performance manager111performs a monitoring operation.

In an embodiment, in the case where a specific host consistently occupies the internal command queue ICQ as much as the internal command queue occupancy threshold, the performance manager111may increase the set resource occupancy threshold. For example, if the host occupies the internal command queue ICQ as much as the internal command queue occupancy threshold by several consecutive time periods, the performance manager111may increase its resource occupancy threshold.

For example, at the first time point t1, the internal command queue occupancy threshold (e.g., the first threshold value) set for the first host11may be “8”. For example, a maximum of 8 first host commands HC1may be stored in the internal command queue ICQ.

During a time period from t2to t5, the first host11may occupy the internal command queue ICQ as much as the first threshold value. For example, up to 8 first host commands HC1(i.e., the maximum number of first host commands HC1) may be stored in the internal command queue ICQ.

In this case, at a sixth time point t6, the performance manager111increases a value of the internal command queue occupancy threshold (e.g., the first threshold value) for the first host11by “1”.

In an embodiment, in the case where a state in which a specific host occupies a resource whose amount is smaller than the set resource occupancy threshold is maintained, the performance manager111may decrease the set resource occupancy threshold.

For example, at the first time point t1, the internal command queue occupancy threshold (e.g., the first threshold value) set for the second host12may be “6”. That is, a maximum of 6 second host commands HC2may be stored in the internal command queue ICQ.

However, during a time period from t2to t5, the second host12may occupy the internal command queue ICQ to be less than the second threshold value. For example, the number of second host commands HC2stored in the internal command queue ICQ may be “5” at the second time point t2, may be “4” at the third time point t3, and may be “3” at the fourth and fifth time points t4and t5.

In this case, at the sixth time point t6, the performance manager111may decrease a value of the internal command queue occupancy threshold (e.g., the second threshold value) set for the second host12by “1”.

In an embodiment, the performance manager111may maintain a resource occupancy threshold set for a specific host.

For example, at the first time point t1, the internal command queue occupancy threshold (e.g., the third threshold value) set for the third host13may be “1”. That is, a maximum of one third host command HC3may be stored in the internal command queue ICQ.

At the second and fourth time points t2and t4, the third host13may occupy the internal command queue ICQ as much as the third threshold value. That is, at some time points among a plurality of monitoring time points, the third host13may occupy the internal command queue ICQ as much as the set threshold value.

At the third and fifth time points t3and t5, the third host13may occupy the internal command queue ICQ to be less than the third threshold value. That is, at some time points among the plurality of monitoring time points, the third host13may occupy the internal command queue ICQ to be less than the set threshold value.

In this case, at the sixth time point t6, the performance manager111does not change a value of the internal command queue occupancy threshold (e.g., the third threshold value) set for the third host13.

In an embodiment, the dynamic change described with reference toFIG.6may be performed without the intervention of the hypervisor HV. That is, the dynamic change may be automatically performed by the performance manager111.

In an embodiment, when the performance manager111determines that the occupancy of a given host is at its threshold value or one below its threshold value for two or more consecutive time points, the performance manager111increases the threshold value for the given host by one or more. In an embodiment, when the performance manager111determines that the occupancy of a given host is one or two below its threshold value for two or more consecutive time points, the performance manager111decreases the threshold value for the given host by one or more.

For brevity of description, an embodiment in which a threshold value increases when there is maintained a state where a specific host occupies the internal command queue ICQ as much as a corresponding threshold value is illustrated inFIG.6. However, the present disclosure is not limited thereto. For example, the performance manager111may change the resource occupancy threshold based on a ratio of a monitored resource occupancy threshold and a monitored resource occupancy amount. For example, the performance manager111may increase the resource occupancy threshold associated with a host in which a value obtained by dividing a resource occupancy amount by a resource occupancy threshold is consistently 0.9 or more. As in the above description, the performance manager111may decrease the resource occupancy threshold associated with a host in which a value obtained by dividing a resource occupancy amount by a resource occupancy threshold is consistently 0.3 or less.

Also, the operation in which the storage device100dynamically changes an occupancy threshold of the internal command queue ICQ is disclosed inFIG.6, but the present disclosure is not limited thereto. For example, as in the above manner described with reference toFIG.6, the storage device100may update various resource occupancy thresholds such as a volatile memory space occupancy threshold and a nonvolatile memory space occupancy threshold.

FIGS.7A to7Care flowcharts illustrating operations of a storage device changing a resource occupancy threshold.

Referring toFIGS.1,3to6, and7A, the storage device100may update a resource occupancy threshold under control of the hypervisor HV. For example, in operation S11, the hypervisor HV may monitor the QoS of a multi-host. That is, the hypervisor HV may monitor the QoS for each of the first to n-th hosts11to1n.

In operation S12, the hypervisor HV may request resource status information from the storage device100. For example, the hypervisor HV may issue a command for requesting the resource status information. The storage device100may fetch the command issued by the hypervisor HV.

In operation S13, the storage device100may return the resource status information to the hypervisor HV. For example, the storage device100may return a total amount of a resource and/or a resource occupancy threshold set for each host to the hypervisor HV.

In an embodiment, the request in operation S12may be the same as the request of the hypervisor HV described with reference toFIG.5A. Also, the return in operation S13may be the same as the return operation of the storage controller110described with reference toFIG.5A. That is, in operation S13, the storage device100may return a total amount of the internal command queue ICQ and an internal command queue occupancy threshold for each host.

In operation S14, the hypervisor HV may request the storage device100to update the resource occupancy threshold. For example, the hypervisor HV may request the storage device100to change the internal command queue occupancy threshold. The operations of the hypervisor HV and the storage device100in operation S14are similar to those described with reference toFIG.5B, and thus, additional description will be omitted to avoid redundancy.

In operation S15, the storage device100may update the resource occupancy threshold in response to the request in operation S14. For example, the storage device100may change the set internal command queue occupancy threshold.

Referring toFIGS.1,3to6, and7B, the storage device100may update the resource occupancy threshold under control of the hypervisor HV. For example, in operation S21, the storage device100may monitor the resource occupancy status. For example, the performance manager111of the storage device100may monitor the internal command queue occupancy status for each of the first to n-th hosts11to1n. That is, the performance manager111may monitor the numbers of first to n-th host commands HC1to HCn stored in the internal command queue ICQ.

In operation S22, the storage device100may notify the hypervisor HV that there is a need to update the resource occupancy threshold. For example, when it is determined that the internal command queue ICQ is monopolized by some hosts or when it is determined that occupancy thresholds are unnecessarily allocated to some hosts, the storage device100may request the hypervisor HV to change the resource occupancy threshold(s).

In operation S23to operation S26, the hypervisor HV may check the resource status of the storage device100and may request the storage device100to update the resource occupancy threshold. The storage device100may update the resource occupancy threshold in response to the request of the hypervisor HV. The operations of the hypervisor HV and the storage device100in operation S23to operation S26are similar to those described with reference toFIG.7A, and thus, additional description will be omitted to avoid redundancy.

Referring toFIGS.1,3to6, and7C, the storage device100may update a resource occupancy threshold under control of the hypervisor HV.

In operation S31, the storage device100may dynamically change the resource occupancy threshold. For example, the storage device100may automatically update internal command queue occupancy thresholds respectively set for the first to n-th hosts11to1n. An operation in which the storage device100dynamically changes internal command queue occupancy thresholds in operation S31is similar to that described with reference toFIG.6, and thus, additional description will be omitted to avoid redundancy.

In operation S32, the storage device100may notify the hypervisor HV that the resource occupancy threshold has been completely updated.

In operation S33to operation S36, the hypervisor HV may check the resource status of the storage device100and may request the storage device100to update the resource occupancy threshold. The storage device100may update the resource occupancy threshold in response to the request of the hypervisor HV. The operations of the hypervisor HV and the storage device100in operation S33to operation S36are similar to those in operation S12to operation S15described with reference toFIG.7A, and thus, additional description will be omitted to avoid redundancy.

According to an embodiment of the present disclosure, a resource occupancy threshold may be set for each of a plurality of hosts. In this case, a resource may be prevented from being monopolized by a specific host among the plurality of hosts. Accordingly, a storage device of improved performance may be provided.

While the present disclosure has been described with reference to embodiments thereof, it will be apparent to those of ordinary skill in the art that various changes and modifications may be made thereto without departing from the spirit and scope of the present disclosure as set forth in the following claims.