Quality of service control of logical devices for a memory sub-system

A processing device in a memory sub-system iteratively processes input/output (I/O) operations corresponding to a plurality of logical devices associated with a memory device. Tor each of the plurality of logical devices, the processing includes identifying a current logical device, determining one or more I/O operations in queue for the current logical device, and determining a number of operation credits associated with the current logical device. The number of credits is based at least in part on a set of quality of service (QoS) parameters for the current logical device. The processing further includes responsive to determining that the number of operation credits satisfies a threshold condition, performing the one or more I/O operations for the current logical device and identifying a subsequent logical device of the plurality of logical devices.

TECHNICAL FIELD

The present disclosure generally relates to a memory sub-system, and more specifically, relates to quality of service control of logical devices for memory sub-systems.

BACKGROUND

DETAILED DESCRIPTION

A memory sub-system can include high performance memory devices that can be partitioned across multiple clients or users. Conventional memory sub-systems that are configured in this way can partition one or more memory devices into logical devices that can be assigned to different users or utilized for different application purposes. In such instances, the total capacity, as well as the access bandwidth and input/output operations per second (IOPS) capability, of a physical device can be apportioned to the logical devices (sometimes referred to as a virtualized “physical function”), either evenly across the logical devices or in different capacity profiles or configurations based on particular client or customer implementations. For example, a 1 Terabyte (Tb) physical device can be partitioned into multiple logical devices with equivalent capacities (e.g., 10 partitions of 100 Gigabytes (Gb), 1000 partitions of 1 Gb, etc.), or differing capacities (e.g., 5 partitions of 100 Gb and 20 partitions of 25 Gb). Furthermore, the available access bandwidth and/or TOPS capability can be partitioned among the multiple logical devices, in addition to or instead of the storage capacity. For example, certain percentages of the available bandwidth and/or TOPS capability can be assigned to each logical device.

Conventional memory sub-systems that are configured in this manner typically implement quality of service (QoS) policies to provide performance agreements to the users of each logical device or partition. In such cases, a QoS policy typically provides latency agreements to ensure that read and/or write operations satisfy predetermined threshold values. In many conventional implementations, however, the QoS policies are applied evenly across the partitions of a memory device. In other words, the latency capabilities of the physical device are distributed evenly across the logical devices/partitions. This can significantly limit implementation flexibility for the logical devices since it prevents configuring high performance partitions and lower performance partitions on the same physical drive. Additionally, many conventional memory sub-systems implement QoS policies in terms of latency rather than bandwidth measurements or TOPS metrics. Moreover, conventional implementations are configured such that the QoS policy of one logical device does not typically operate with any awareness of other logical devices for the same physical device. In such instances, overall bandwidth of a physical device can often be underutilized when one logical device is idle while other logical devices are experiencing heavier workloads.

Aspects of the present disclosure address the above and other deficiencies by implementing a QoS manager to facilitate quality of service control of logical devices in memory sub-systems. The QoS manager can receive bandwidth QoS parameters associated with logical device partitions for a physical device and divide the device bandwidth into sub-bands, where each sub-band is associated with one of the logical devices. The QoS manager can then determine the I/O operations in queue for a particular logical device and perform the queued operations using an earned credit scheme, where a logical device earns credits based on the QoS parameters. Additionally, the QoS manager can operate with awareness of all logical devices associated with a physical device such that any underutilized bandwidth of one logical device can be redirected to another logical device during periods of more active workloads.

Advantages of the present disclosure include, but are not limited to, significantly improved bandwidth performance for logical devices partitioned across a physical device in a memory sub-system. Implementing the QoS manager of the present disclosure can ensure QoS TOPS and bandwidth agreements for each logical device within its own performance sub-band using the credit scheme tuned to the QoS parameters. Additionally, since the QoS manager of the present disclosure is aware of the QoS parameters and current performance of each of the logical devices, any underutilized capabilities of the physical device can be redirected to any logical device that is operating under heavier workloads. Moreover, since the bandwidth performance of the memory device is improved, the performance of a memory sub-system that includes the memory device also improves, since fewer I/O bottlenecks are encountered across the memory sub-system.

The memory sub-system controller115can additionally include QoS management component113that can be used to facilitate quality of service control of logical devices for memory sub-system110. In some embodiments, memory sub-system controller115includes at least a portion of QoS management component113. For example, the memory sub-system controller115can include a processor117(e.g., processing device) configured to execute instructions stored in local memory119for performing the operations described herein. In some embodiments, the QoS management component113is part of the host system120, an application, or an operating system. In other embodiments, local media controller135includes at least a portion of QoS management component113and is configured to perform the functionality describe herein.

The QoS management component113can receive bandwidth and TOPS parameters associated with logical device partitions for a physical device (e.g., memory devices130,140of memory sub-system110). The QoS management component113can divide the bandwidth and/or TOPS capability for memory devices130,140into sub-bands, where each sub-band is associated with one of the logical devices. The QoS manager can then determine the I/O operations in queue for a particular logical device and perform the queued operations using an earned credit scheme, where a logical device earns credits based on its associated QoS parameters. In various implementations, the QoS management component113can operate with awareness of other logical devices associated with memory devices130,140such that any underutilized bandwidth or TOPS capability of one logical device can be redirected to another logical device during periods of more active workloads. Further details with regards to the operations of the QoS management component113are described below.

FIG. 2illustrates an example interface between a host system120and a memory sub-system controller115implementing a QoS management component113in accordance with some embodiments of the present disclosure. In various implementations, host system120, memory sub-system controller115, and QoS management component113correspond to host system120, memory sub-system controller115, and QoS management component113ofFIG. 1.

In some implementations, memory sub-system controller115includes an NVMe controller211coupled to PCIe port210which enables communications with host system120across PCIe bus205. As noted above, a memory sub-system can include memory devices that are partitioned into multiple logical devices230(e.g., logical devices230-A,230-B,230-C). As noted above, in various implementations, the resources (e.g., storage capacity, I/O bandwidth capacity, etc.) of a physical memory device, such as memory device130, can be partitioned into multiple logical devices230, where each logical device230represents a portion of the resources of the physical device. Each logical device230can be presented to host system120as a representation of an independent physical device. In such instances, the total capacity of a physical device can be apportioned to the logical devices230(sometimes referred to as virtualized “physical functions”), either evenly across the logical devices230or in different capacity profiles or configurations based on particular client or customer implementations. For example, a 1 Terabyte (Tb) physical device can be partitioned into multiple logical devices with equivalent capacities (e.g., 10 partitions of 100 Gigabytes (Gb), 1000 with 1 Gb, etc.), or differing capacities (e.g., 5 partitions of 100 Gb and 20 partitions of 25 Gb.). Similarly, the bandwidth and/or IOPS capacity of a memory device can be divided across the logical partitions230. In one embodiment, the bandwidth represents the total processing capacity of a memory device based on an amount of data in a given period of time, while the TOPS capacity represents a number of separate transactions that can be performed in a given period of time, and is potentially influenced by the size of those transactions For example, a memory device with an bandwidth capacity of 6 Gb/second can be partitioned such that logical device230-A is configured with a maximum bandwidth of 2 Gb/sec, whereas logical device230-B is configured with 1 Gb/sec. In other implementations, other configurations can be used for the logical devices230.

In some implementations, NVMe controller211can be configured with one or more logical or virtual NVMe controllers that are each associated with one of the logical devices230. Additionally, each logical device230can be associated with a corresponding first-in-first-out (FIFO) command queue (e.g., FIFO212-A,212-B,212-C) that queues I/O commands for the logical devices230. As I/O commands are received from the host120via PCIe port210, the NVMe Controller211can determine the destination logical device230to which to direct the command and add the command to the corresponding FIFO queue for the destination logical device230.

In various implementations, the I/O commands received from host120can be directed to one or more blocks of data stored by the logical devices230. For example, a single I/O command (e.g., a command directed to a memory device configured as an NVMe device) can access a single block of data stored on logical device230-A. In such instances, the NVMe controller211can determine that the command is to be directed to logical device230-A and can store that command in the FIFO queue for logical device230-A (e.g., FIFO212-A). Once the command is processed by memory sub-system controller115, the single NVMe I/O command can be translated into a format that is understood by the media controller of the memory device (e.g., a front end media controller (FEMC) command). In some implementations, a single NVMe I/O command can be a command to access data made up of multiple blocks. In such instances one NVMe I/O command can later be translated into multiple FEMC commands that are processed by the memory device. For example, a single NVMe I/O command directed to 4 blocks of data can cause 4 FEMC commands to be later executed by the memory device.

In various implementations, each of logical devices230can be configured with individual quality of service (QoS) parameters. As noted above, the QoS parameters for the logical devices can be based on the bandwidth or TOPS capabilities of the memory device that has been partitioned. For example, given a memory device capable of providing a maximum bandwidth of 12 Gb/sec, the individual logical devices230can be partitioned and configured with a fraction of that total capacity (e.g., 2 Gb/sec, 5 Gb/sec, etc.). In such instances, the total device bandwidth can be divided into sub-bands, where each logical device230is associated with one of the sub-bands. In some implementations, the entire bandwidth of a memory device does not need to be apportioned across all logical devices. In other words, once the logical devices have been apportioned some of the bandwidth of the memory device, if additional bandwidth remains unallocated, it can be held in reserve for use during periods of observed increases in I/O workload.

The logical devices230can be configured with QoS parameters associated with a particular mode of operation. Depending on the embodiment and mode, separate thresholds can be configured for different parameters (i.e., for bandwidth and TOPS). In one example, a logical device can be configured to not exceed a specified bandwidth or TOPS value (e.g., “limit mode”). The limit mode sets a maximum performance ceiling (e.g., a limit threshold) for a logical device that QoS management component113can enforce on each physical function. In some implementations, each assigned limit should be set below the device's max bandwidth (e.g., no logical device can have a performance ceiling above the capability of the drive). However, in some implementations, the sum of all limits for all logical devices can exceed the maximum capacity for the memory device (referred to as “oversubscribed”). In one embodiment, there can be separate limit thresholds corresponding to bandwidth and TOPS. In such an embodiment, the TOPS limit threshold should not exceed the bandwidth limit threshold and QoS management component113can enforce both thresholds.

In another example, a logical device can be configured to not fall below a specified bandwidth or TOPS value (e.g., “reservation mode”). The reservation mode sets a minimum performance floor (e.g., a reservation threshold) for a logical device that QoS management component113will guarantee for each physical function (assuming that the host workload intensity allows that threshold to be met). In some implementations, the sum of all reservations should not exceed the maximum capabilities for the memory device (referred to as “undersubscribed”). In one embodiment, there can be separate reservation thresholds corresponding to bandwidth and TOPS. In such an embodiment, the TOPS reservation threshold should not be higher than the bandwidth reservation threshold. Depending on the received workload, QoS management component113can guarantee at least one of these thresholds.

In another example, a logical device can be configured with a combination of a limit and reservation (e.g., “reservation with limit mode”). This mode defines a performance window within which the logical device should perform, defining both a floor and a ceiling bandwidth and IOPs value that QoS management component113can enforce on each physical function. As described above, the corresponding TOPS thresholds should not be higher than the corresponding bandwidth thresholds.

QoS management component113is responsible for management of the QoS parameters for each of the logical devices230for the memory subsystem. The QoS management component determines whether there are any queued commands in the FIFOs212and if so, performs one or more I/O operations for the logical devices while adhering to the QoS modes for the associated logical devices230. The QoS management component113facilitates this process by implementing a credit scheme where credits are earned for each logical device in accordance with its assigned QoS performance mode as described above. A timer (e.g., a clock) is configured for each logical device such that the logical device earns credits based on the reservation threshold for the logical device and the timer cycle. Thus, a logical device can be configured to earn credits at a rate associated with the QoS parameters for the logical device (e.g., at a rate associated with the bandwidth frequency for the QoS mode) based on the cycles of the timer. In one embodiment, this timer can be referred to as a variable frequency credit-replenish clock.

QoS management component113can include a QoS credit manager225to manage the credits for each of the logical devices230. In some implementations, a credit can be used to process a portion of the data associated with an I/O command. As noted above, a logical device associated with an NVMe memory device can be configured such that each I/O command can access data made up of one or more blocks. In such instances, the credit scheme can be configured such that the cost to access one block is one credit. Thus, one credit is needed for each block of the data associated with an I/O operation in order to complete the entire I/O operation. For example, if the I/O operation is associated with 10 blocks, then 10 credits can be used to complete the I/O operation. The QoS credit manager225can identify the QoS parameters for a logical device230and configure its associated timer so that the logical device can earn credits based on the associated reservation performance bandwidth or IOPs settings (based on the data size of the I/O command). The QoS credit manager225can additionally deduct credits spent by the received I/O operations that are directed to a particular logical device230. In some implementations, the QoS credit manager225can be configured such that each logical device230is initiated with a beginning credit value that is greater than zero. Configuring the credit scheme in this manner can facilitate initial command processing more efficiently, since the logical devices should not need to wait for an extended period of time to accrue sufficient credits to perform I/O operations. An example of the management of credits for a logical device is illustrated below in conjunction withFIG. 3.

In various implementations, credits earned by one logical device230can be shared with another logical device230. In other words, should logical device230-A experience a period of heavy workload, resulting in the exhaustion of its credit balance (e.g., no available credits for I/O commands in queue), it could utilize stored credits accumulated by logical devices230-B or230-C if those logical devices do not have any I/O commands queued in their associated FIFOs212. Similarly, if there is any unallocated bandwidth for a memory device, that too can be utilized by logical device230-A. In this latter case, unallocated bandwidth can be assigned a logical device timer to maintain a credit store that can be shared with allocated logical devices during periods of high intensity workloads.

QoS management component113can additionally include an arbiter component220to monitor the FIFOs212for received I/O commands and to select a FIFO from which to retrieve the next I/O command to be processed. In various implementations, the arbiter component220can select the FIFOs212in a round robin scheduling fashion and utilize the credit information managed by QoS credit manager225to determine whether to process a command from the selected FIFO212. In one example, arbiter component220can select FIFO212-A, and determine whether FIFO212-A has a command that can be fetched. If so, arbiter component220can then access the credit information maintained by QoS credit manager225to determine whether logical device230-A has at least one credit available. If so, then arbiter component220can fetch the command and forward it to logical device230-A. If the command is directed to multiple blocks, arbiter component220can determine how many credits are needed to process all of the blocks associated with the command. If the number of credits needed to process the I/O command exceeds the number of credits available for the logical device, all of the blocks associated with command can be processed, but the total credits available can be driven to a negative number. The negative credit value can still be refreshed according to the timer clock.

If the arbiter component220determines that logical device230-A does not have any available credits to process the I/O command, several alternatives can be performed. In one example, arbiter component220can skip the command and move to the next available FIFO212(e.g., FIFO212-B) to repeat the process for that FIFO212. Alternatively, arbiter component220can determine whether any of the other logical devices230have credits available but do not have any commands queued in their associated FIFO212to be processed. In such instances, arbiter component220can utilize the processing slot for the idle logical device to process the command for logical device230-A. For example, arbiter component220can determine that logical device230-B has credits available but no commands queued in FIFO212-B, so the command queued in FIFO212-A for logical device230-A can still be processed even though logical device230-A has insufficient credits accumulated to execute the command.

In various implementations, arbiter component220can be configured to perform arbitration on a number of different types of requests issued by or directed to the physical functions (e.g., logic devices230-A,230-B,230-C). In one example, arbiter component220can perform arbitration based on a timer (e.g., “timer arbitration”). In such instances, arbiter component220can select the FIFO212for the logical device230based on the credit timer frequency associated with the logical device and issue what is referred to as a “timer request.” Thus, a timer request can be generated in synchronization with each cycle of the above mentioned credit-replenish clock. In other words, a logical device230needs to have at least one credit for its associated FIFO to be selected. Otherwise, the FIFO for that logical device230is bypassed and the next FIFO is selected. When a timer request is serviced by arbiter component220, the number of timer requests is decremented by the number of blocks fetched, and could become negative. Timer credits are replenished on each cycle of the credit-replenish clock.

In another example, arbiter component220can perform arbitration based on an available opportunity to execute a command (e.g., “opportunistic arbitration”). In such instances, if the selected FIFO212does not include a command, arbiter component220can select a different FIFO, such as FIFO212-A, that does have commands queued for processing whether the associated logical devices have sufficient credit or not. Such an action can be referred to as an “opportunistic request” and is generated when a physical function has at least one IOPS extra credit and one bandwidth extra credit available. Arbiter component220, however, will only service an opportunistic request when the timer request for another physical function is left unconsumed, because there was no I/O operation for that physical function at that time.

In another example, arbiter component220can perform arbitration based on a burst of requests (e.g., “burst arbitration”). In such instances, if a command is fetched based on a timer (e.g., timer arbitration) an additional command can be fetched immediately and processed. Such an action can be referred to as a “burst request” and is generated asynchronously between timer events when at least one burst credit (or timer credit) is available. Similar to a timer request, when a burst request is serviced by arbiter component220, the number of burst credits is decremented by the number of blocks fetched, and could become negative. In one embodiment, burst requests of all physical functions are serviced in a round-robin fashion. Burst arbitration allows each physical function to process short I/O bursts quicker and can be beneficial depending on the host workload. In a sustained type of workload, the burst requests are immediately consumed, and then for each burst credit returned, only timer requests are processed.

As described above, opportunistic requests take advantage of timer requests left unconsumed by other physical functions. When the sum of all reservations is strictly undersubscribed, the device bandwidth left unassigned can be used by QoS management component113to generate timer events that can be shared by the all physical functions for opportunistic requests. These opportunistic requests can be issued by physical functions that have been configured in reservation mode or reservation with limit mode and were provisioned with extra credit allowances to permit them to reach the corresponding limits. To allow reaching the TOPS limit, for example, the TOPS extra credit allowance can be calculated as the differential between the IOPS limit and the TOPS reservation. Similarly, the bandwidth extra credit allowance is calculated as the additional number of blocks that need to be transferred, above the bandwidth reservation, such that bandwidth limit could be reached.

In one embodiment, the bandwidth and TOPS extra credit allowances are calculated to be used during a period of time which can be referred to as a “session.” One session follows another, and at the beginning of each session the extra credit allowances are replenished. During a session, the physical function makes opportunistic requests as long as it has at least one available credit in each type of the extra credit allowances. When an opportunistic request is serviced, the IOPS extra credit allowance is decremented by one and the bandwidth extra credit allowance is decremented by the number of blocks requested.

FIG. 3illustrates an example of credit management for a logical device in accordance with some embodiments of the present disclosure. In some embodiments, this process can be performed by the QoS management component113ofFIG. 1and as described above with respect toFIG. 2.

As shown inFIG. 3, the logical device has been configured such that its number of credits has been initialized to 10 at startup (e.g., timer cycle300). The logical device is associated with a credit timer330where credits are earned for the logical device in accordance with the QoS parameters associated with the logical device. Thus, as shown, an additional credit is earned for the logical device that can be applied to processing I/O requests held in a corresponding FIFO queue, after each successive timer cycle.FIG. 3illustrates a series of I/O commands fetched from the associated FIFO, the credit costs associated with processing the commands, and the credit accumulations caused by the credit timer cycles of credit timer330.

At timer cycle301, the arbiter component of the QoS management component (e.g., arbiter component220inFIG. 2) selects the FIFO for the logical device and identifies I/O command320in the corresponding FIFO queue. In one embodiment, I/O command320includes 8 sub-commands directed to different blocks. Alternatively, I/O command320can represent a single sub-command that is to access 8 different blocks. In both cases, since I/O command320accesses 8 blocks, a total of 8 credits will be needed to satisfy the command. The first sub-command of I/O command320is processed at timer cycle301, decrementing the total credits from 10 to 9. As shown, the 8 sub-commands for the 8 blocks of I/O command320are processed between clock cycle301and302, thus decrementing the credit count from 10 to 2 (illustrated by the “credits after subtraction” value of 2).

At cycle302, the credit timer330has accumulated an additional credit that changes the credit count from 2 to 3 (illustrated by the “credits after increment” value of 3). At cycle303, the credit timer330has accumulated an additional credit that changes the credit count from 3 to 4 (illustrated by the “credits after increment” value of 4). In some implementations, since no additional commands are in queue between timer cycles302and303A, the arbiter component can select another FIFO queue to determine whether any commands are available for processing in another FIFO queue.

At cycle303A, a second set of I/O commands321is fetched by the arbiter component220. For example, arbiter component220can fetch the second set of I/O commands321based on a burst arbitration scheme. As shown, the first sub-command decrements the credit count from 4 to 3 (illustrated by the “credits after subtraction” value of 3). At cycle304, an additional credit is accumulated that changes the credit count from 3 to 4 (illustrated by the “credits after increment” value of 4), and that credit is immediately used by the second sub-command of I/O command321, which changes the credit count back to 3. The two additional sub-commands are subsequently processed which further reduces the credit count to 1 at cycle304A (illustrated by the “credits after subtraction” value of 1).

At cycle305, the credit timer330has accumulated an additional credit that changes the credit count from 1 to 2 (illustrated by the “credits after increment” value of 2). During the same cycle, a third set of I/O commands322is fetched by the arbiter component. The first sub-command is processed, which decrements the credit count from 2 to 1 (illustrated by the “credits after subtraction” value of 1). The second sub-command is then processed, which further decrements the credit count from 1 to 0 at cycle305A (illustrated by the “credits after subtraction” value of 0).

At cycle305A, although the logical device has exhausted its earned credits, the remaining two sub-commands of I/O command322can still be processed. Thus, by cycle305B, the credit count is decremented from 0 to −2. As shown inFIG. 3, one additional credit is accumulated at cycle306, and another additional credit is accumulated at cycle307, bringing the total credit count back to 0. Another credit is accumulated at cycle308, bringing the total credit count from 0 to 1 (illustrated by the “credits after increment” value of 1). As long as the logical device has one or more earned credits, arbiter component220can fetch a new I/O command associated with the logical device to be processed. By implementing the credit timer in accordance with the QoS parameters for the logical device, the ceiling threshold for the logical device can be preserved.

FIG. 4illustrates an example physical host interface between a host system and a memory sub-system implementing quality of service management in accordance with some embodiments of the present disclosure. In one embodiment, memory sub-system controller115of memory sub-system110is connected to host system120over a physical host interface, such as PCIe bus410. In one embodiment, memory sub-system controller115manages one or more NVMe controllers402-408. Depending on the embodiment, there can be a single NVMe controller, as illustrated above with respect toFIG. 2, or multiple virtual NVMe controllers402-408, which are virtual entities that appear as physical controllers to other devices, such as host system120, connected to PCIe bus410by virtue of a physical function412-418associated with each virtual NVMe controller402-408.FIG. 4illustrates four virtual NVMe controllers402-408and four corresponding physical functions412-418. In other embodiments, however, there can be any other number of NVMe controllers, each having a corresponding physical function. All of the virtual NVMe controllers402-408have the same priority and same functionality.

Each of virtual NVMe controllers402-408manages storage access operations for the underlying memory device130. For example, virtual NVMe controller402can receive data access requests from host system120over PCIe bus410, including requests to read, write, or erase data. In response to the request, virtual NVMe controller402can identify a physical memory address in memory device130pertaining to a logical memory address in the request, perform the requested memory access operation on the data stored at the physical address and return requested data and/or a confirmation or error message to the host system120, as appropriate. Virtual NVMe controllers404-408can function in the same or similar fashion with respect to data access requests for memory device130.

As described above, each of physical functions412-418is associated with each of virtual NVMe controllers402-408in order to allow each virtual NVMe controller402-408to appear as a physical controller on PCIe bus410. For example, physical function412can correspond to virtual NVMe controller402, physical function414can correspond to virtual NVMe controller404, physical function416can correspond to virtual NVMe controller406, and physical function418can correspond to virtual NVMe controller408. Physical functions412-418are fully featured PCIe functions that can be discovered, managed, and manipulated like any other PCIe device, and thus can be used to configure and control a PCIe device (e.g., virtual NVMe controllers402-408). Each physical function412-418can have some number virtual functions associated with therewith. The virtual functions are lightweight PCIe functions that share one or more resources with the physical function and with virtual functions that are associated with that physical function. Each virtual function has a PCI memory space, which is used to map its register set. The virtual function device drivers operate on the register set to enable its functionality and the virtual function appears as an actual PCIe device, accessible by host system120over PCIe bus410.

Each physical function412-418can be assigned to any one of virtual machines432-436in the host system120. When I/O data is received at a virtual NVMe controller402-408from a virtual machine432-436, a virtual machine driver provides a guest physical address for a corresponding read/write command. The physical function number can be translated to a bus, device, and function (BDF) number and then added to a direct memory access (DMA) operation to perform the DMA operation on the guest physical address. In one embodiment, controller115further transforms the guest physical address to a system physical address for the memory sub-system110.

Furthermore, each physical function412-418can be implemented in either a privileged mode or normal mode. When implemented in the privileged mode, the physical function has a single point of management that can control resource manipulation and storage provisioning for other functions implemented in the normal mode. In addition, a physical function in the privileged mode can perform management options, including for example, enabling/disabling of multiple physical functions, storage and quality of service (QoS) provisioning, firmware and controller updates, vendor unique statistics and events, diagnostics, secure erase/encryption, among others. Typically, a first physical function can implement a privileged mode and the remainder of the physical functions can implement a normal mode. In other embodiments, however, any of the physical functions can be configured to operate in the privileged mode. Accordingly, there can be one or more functions that run in the privileged mode.

Host system120runs multiple virtual machines432,434,436, by executing a software layer424, often referred to as “hypervisor,” above the hardware and below the virtual machines, as schematically shown inFIG. 4. In one illustrative example, the hypervisor424can be a component of a host operating system422executed by the host system120. Alternatively, the hypervisor424can be provided by an application running under the host operating system422, or can run directly on the host system120without an operating system beneath it. The hypervisor424can abstract the physical layer, including processors, memory, and I/O devices, and present this abstraction to virtual machines432,434,436as virtual devices, including virtual processors, virtual memory, and virtual I/O devices. Virtual machines432,434,436can each execute a guest operating system which can utilize the underlying virtual devices, which can, for example, map to the memory device130managed by one of virtual NVMe controllers402-408in memory sub-system110. One or more applications can be running on each virtual machine under the guest operating system.

Each virtual machine432,434,436can include one or more virtual processors. Processor virtualization can be implemented by the hypervisor424scheduling time slots on one or more physical processors such that from the guest operating system's perspective, those time slots are scheduled on a virtual processor. Memory virtualization can be implemented by a page table (PT) which is a memory structure translating guest memory addresses to physical memory addresses. The hypervisor424can run at a higher privilege level than the guest operating systems, and the latter can run at a higher privilege level than the guest applications.

In one implementation, there can be multiple partitions on host system120representing virtual machines432,434,436. A parent partition corresponding to virtual machine432is the root partition (i.e., root ring 0) that has additional privileges to control the life cycle of other child partitions (i.e., conventional ring 0), corresponding, for example, to virtual machines434and436. Each partition has corresponding virtual memory, and instead of presenting a virtual device, the child partitions see a physical device being assigned to them. When host system120initially boots up, the parent partition can see all of the physical devices directly. The pass through mechanism (e.g., PCIe Pass-Through or Direct Device Assignment) allows the parent partition to assign an NVMe device (e.g., one of virtual NVMe controllers402-408) to the child partitions. The associated virtual NVMe controllers402-408can appear as a virtual storage resource to each of virtual machines432,434,436, which the guest operating system or guest applications running therein can access. In one embodiment, for example, virtual machine432is associated with virtual NVMe controller402, virtual machine434is associated with virtual NVMe controller404, and virtual machine436is associated with virtual NVMe controller406. In other embodiments, one virtual machine can be associated with two or more virtual NVMe controllers. The virtual machines432,434,436, can identify the associated virtual NVMe controllers using a corresponding bus, device, and function (BDF) number, as will be described in more detail below.

QoS management component113can implement access control services for each of virtual NVMe controllers402-408. The access control services manage what devices have access permissions for the virtual NVMe controllers402-408. The access permissions can define, for example, which of virtual machines432-436on host system120can access each of virtual NVMe controllers402-408, as well as what operations each of virtual machines432-436can perform on each of virtual NVMe controllers402-408. In one embodiment, QoS management component113controls access permissions for each of virtual NVMe controllers402-408individually. For example, in the privileged mode, QoS management component113can grant virtual machine432permission to read and write data using virtual NVMe controller402, but only permission to read data using virtual NVMe controller404. Similarly, in the privileged mode, QoS management component113can grant virtual machine432permission to read and write data using virtual NVMe controller404only. Any combination of access permissions can be defined for virtual NVMe controllers402-408. When a memory access request is received for one of virtual NVMe controllers402-408, QoS management component113can analyze the conditions of the request (e.g., requester, target, operation, requested data address, etc.) based on access policies defining the access control services. The access policies can be stored in local memory119. If the request satisfies the corresponding access policy (the conditions of the request match conditions specified in the corresponding access policy), QoS management component113can grant the access request. Otherwise, the request can be denied.

FIG. 5illustrates a method of quality of service control of logical devices for memory sub-systems in accordance with some embodiments of the present disclosure. The method500can be performed by processing logic that can include hardware (e.g., processing device, circuitry, dedicated logic, programmable logic, microcode, hardware of a device, integrated circuit, etc.), software (e.g., instructions run or executed on a processing device), or a combination thereof. In some embodiments, the method500is performed by QoS management component113ofFIG. 1. Although shown in a particular sequence or order, unless otherwise specified, the order of the processes can be modified. Thus, the illustrated embodiments should be understood only as examples, and the illustrated processes can be performed in a different order, and some processes can be performed in parallel. Additionally, one or more processes can be omitted in various embodiments. Thus, not all processes are required in every embodiment. Other process flows are possible.

At operation510, the processing logic provides a number of physical functions412-418, wherein each of the physical functions212-218corresponds to a virtual memory controllers, such as NVMe Controller211or one of virtual NVMe controllers402-408. Each of the physical functions412-418represents a corresponding logical device, such as one of logical devices230-A-230-C, as a physical device to the host system120on a communication interface, such as PCIe bus210. In one embodiment, the physical functions412-418are created in response to input received from the system administrator via a management interface.

At operation520, the processing logic presents the physical functions412-418to a host computing system, such as host system120, over the communication interface, such as PCIe bus210. In one embodiment, the host system120assigns each of the physical functions412-418to a different virtual machine, such as one of virtual machines432,434,436, running on the host system120. Each of the physical functions412-418provides a configuration space for a corresponding one of the logical devices, wherein each configuration space is individually addressable. In other embodiments, each of the physical functions412-418can be assigned to separate application, users, programs, etc. running on host system120, or even to separate host systems entirely.

At operation530, the processing logic receives requests to perform I/O operations from host system120. In one embodiments, the requests are received from an assigned virtual machine, such as one of virtual machines432,434,436, running on the host system120. The requests can pertain to the one or more memory devices, such as memory devices130.

At operation540, the processing logic iteratively processes the I/O operations corresponding to the logical devices230-A-230-C associated with the memory device130. In one embodiment, one or more of virtual NVMe controllers402-408can perform the requested I/O operation, such as a read, write or erase operation, and can return requested data and/or a confirmation or error message to the host system120, as appropriate. In one embodiment, arbiter component220of QoS management component220manages the memory access requests, as well as background operations performed on the memory device, in view of QoS parameters associated with the physical functions. In one embodiment, QoS credit manager225manages operation credits for the physical functions, wherein a number of credits associated with each physical function is based at least in part of the QoS parameters, as described in more detail with respect toFIG. 6. For example, QoS management component220progressively rotate through each of the logical devices to determine whether I/O operations directed to a current logical device are present (e.g., in a corresponding FIFO) and whether sufficient operation credits are available to process those I/O operations.

FIG. 6illustrates a method of processing I/O operations for logical devices of a memory device in view of quality of service parameters in accordance with some embodiments of the present disclosure. The method600can be performed by processing logic that can include hardware (e.g., processing device, circuitry, dedicated logic, programmable logic, microcode, hardware of a device, integrated circuit, etc.), software (e.g., instructions run or executed on a processing device), or a combination thereof. In some embodiments, the method600is performed by QoS management component113ofFIG. 1. Although shown in a particular sequence or order, unless otherwise specified, the order of the processes can be modified. Thus, the illustrated embodiments should be understood only as examples, and the illustrated processes can be performed in a different order, and some processes can be performed in parallel. Additionally, one or more processes can be omitted in various embodiments. Thus, not all processes are required in every embodiment. Other process flows are possible.

At operation610, the processing logic identifies a current logical device. In one embodiment, identifying the current logical device includes applying at least one of a burst arbitration scheme or a timer arbitration scheme to the logical devices. The current logical device represents one of logical devices230-A-230-C which is being operated on by QoS management component113. As part of an iterative process, QoS management component113operates on each of the logical devices in turn, returning to a first logical device after operating on a last logical device, and the operations described with respect to method600can be performed for each of the logical device.

At operation620, the processing logic determines one or more I/O operations in queue for the current logical device. In one embodiment, QoS management component113examines a corresponding queue for the current logical device, such as a corresponding FIFO212-A-212-C. The corresponding queue maintains requests to perform I/O operations at the current logical device. In one embodiment, the requests are received from host system120via PCIe bus205and are directed to an associated physical function of the memory sub-system controller115.

At operation630, the processing logic determines a number of operation credits associated with the current logical device. In one embodiment, the number of credits is based at least in part on a set of quality of service (QoS) parameters for the current logical device. In one embodiment, the set of QoS parameters for the current logical device indicates a subset of a bandwidth capability or an input/output operations per second (IOPS) capability of the memory device associated with the current logical device. In one embodiment, the set of QoS parameters for the current logical device includes at least one of a limit parameter associated with a maximum bandwidth and/or TOPS capability for the current logical device or a reservation parameter associated with a minimum bandwidth and/or TOPS capability for the current logical device.

At operation640, the processing logic determines whether the number of operation credits satisfies a threshold condition. In one embodiment, the number of operation credits satisfies the threshold condition when the number of operation credits is a positive number of operation credits. For example, as long as the current logical device, has at least one operation credit, QoS management component113can determine that the threshold condition is satisfied. In other embodiments, some other threshold number of credits can be defined. In one embodiment, the number of operation credits associated with the current logical device periodically increases in response to the passage of one or more timer cycles of a timer associated with the current logical device, as described above in detail with respect toFIG. 3.

Responsive to determining that the number of operation credits satisfies the threshold condition, at operation650, the processing logic performs the one or more I/O operations for the current logical device. In one embodiment, each of the one or more I/O operations can include one or more sub-operations. Each sub-operation corresponding to a memory access operation, such as a read operation a write/program operation, or an erase operation. In one embodiment, one operation credit is used to perform each of the sub-operations, and the operation credit count is decremented accordingly.

At operation660, the processing logic identifies a subsequent logical device of the multiple logical devices. Once the I/O operations are performed, or if it is determined that insufficient operation credits are available at operation640, QoS management component113can identify a subsequent logical device on which to operate accordingly to the relevant arbitration scheme being implemented on memory device130.

The example computer system700includes a processing device702, a main memory704(e.g., read-only memory (ROM), flash memory, dynamic random access memory (DRAM) such as synchronous DRAM (SDRAM) or Rambus DRAM (RDRAM), etc.), a static memory706(e.g., flash memory, static random access memory (SRAM), etc.), and a data storage system718, which communicate with each other via a bus730.

The data storage system718can include a machine-readable storage medium724(also known as a computer-readable medium) on which is stored one or more sets of instructions726or software embodying any one or more of the methodologies or functions described herein. The instructions726can also reside, completely or at least partially, within the main memory704and/or within the processing device702during execution thereof by the computer system700, the main memory704and the processing device702also constituting machine-readable storage media. The machine-readable storage medium724, data storage system718, and/or main memory704can correspond to the memory sub-system110ofFIG. 1.