Storage control system with data management mechanism and method of operation thereof

A storage control system, and a method of operation thereof, including: a recycle write queue for providing a recycle write; a host write queue for providing a host write; and a scheduler, coupled to the recycle write queue and the host write queue, for scheduling the recycle write and the host write for writing to a memory device.

TECHNICAL FIELD

The present invention relates generally to a storage control system and more particularly to a system for data management.

BACKGROUND ART

Data storage, often called storage or memory, refers to computer components and recording media that retain digital data. Data storage is a core function and fundamental component of consumer and industrial electronics, especially devices such as computers, televisions, cellular phones, mobile devices, and digital video cameras.

Recently, forms of long-term storage other than electromechanical hard disks have become feasible for use in computers. NOT-AND (NAND) flash is one form of non-volatile memory used in solid-state storage devices. The memory cells are arranged in typical row and column fashion with circuitry for accessing individual cells. The memory transistors of those cells are placed to store an analog value that can be interpreted to hold two logical states in the case of Single Level Cell (SLC) or more than two logical states in the case of Multi Level Cell (MLC).

A flash memory cell is light in weight, occupies very little space, and consumes less power than electromechanical disk drives. Construction of a storage system with this type of memory allows for much higher bandwidths and input/output operations per second (IOPS) than typical electromechanical disk drives. More importantly, it is especially rugged and can operate at a much high temperature range. It will withstand without adverse effects repeated drops, each of which would destroy a typical electromechanical hard disk drive. A problem exhibited by flash memory is that it tends to have a limited life in use.

Thus, a need still remains for better data management devices. In view of the increasing demand for data management devices, it is increasingly critical that answers be found to these problems. In view of the ever-increasing commercial competitive pressures, along with growing consumer expectations and the diminishing opportunities for meaningful product differentiation in the marketplace, it is critical that answers be found for these problems. Additionally, the need to reduce costs, improve efficiencies and performance, and meet competitive pressures adds an even greater urgency to the critical necessity for finding answers to these problems.

DISCLOSURE OF THE INVENTION

The present invention provides a method of operation of a storage control system, including: receiving a recycle write from a recycle write queue; receiving a host write from a host write queue; and scheduling the recycle write and the host write for writing to a memory device.

The present invention provides a storage control system, including: a recycle write queue for providing a recycle write; a host write queue for providing a host write; and a scheduler, coupled to the recycle write queue and the host write queue, for scheduling the recycle write and the host write for writing to a memory device.

BEST MODE FOR CARRYING OUT THE INVENTION

The drawings showing embodiments of the system are semi-diagrammatic and not to scale and, particularly, some of the dimensions are for the clarity of presentation and are shown exaggerated in the drawing FIGs.

The term “module” referred to herein can include software, hardware, or a combination thereof in the present invention in accordance with the context in which the term is used. For example, the software can be machine code, firmware, embedded code, and application software. Also for example, the hardware can be circuitry, processor, computer, integrated circuit, integrated circuit cores, a microelectromechanical system (MEMS), passive devices, environmental sensors including temperature sensors, or a combination thereof.

Referring now toFIG. 1, therein is shown a storage control system100with data management mechanism in an embodiment of the present invention. The storage control system100includes a memory sub-system102having a memory controller104and a memory array106. The storage control system100includes a host system108communicating with the memory sub-system102.

The memory controller104provides data control and management of the memory array106. The memory controller104interfaces with the host system108and controls the memory array106to transfer data between the host system108and the memory array106.

The memory array106includes an array of memory devices110including flash memory devices or non-volatile memory devices. The memory array106can include pages of data or information. The host system108can request the memory controller104for reading, writing, and deleting data from or to a logical address space of a storage device or the memory sub-system102that includes the memory array106.

Referring now toFIG. 2, therein is shown an exemplary hardware block diagram of the memory controller104. The memory controller104can include a control unit202, a storage unit204, a memory interface unit206, and a host interface unit208. The control unit202can include a control interface210. The control unit202can execute software212stored in the storage unit204to provide the intelligence of the memory controller104.

The control unit202can be implemented in a number of different manners. For example, the control unit202can be a processor, an embedded processor, a microprocessor, a hardware control logic, a hardware finite state machine (FSM), a digital signal processor (DSP), or a combination thereof.

The control interface210can be used for communication between the control unit202and other functional units in the memory controller104. The control interface210can also be used for communication that is external to the memory controller104.

The control interface210can receive information from the other functional units or from external sources, or can transmit information to the other functional units or to external destinations. The external sources and the external destinations refer to sources and destinations external to the memory controller104.

The control interface210can be implemented in different ways and can include different implementations depending on which functional units or external units are being interfaced with the control interface210. For example, the control interface210can be implemented with a dedicated hardware including an application-specific integrated circuit (ASIC), a configurable hardware including a field-programmable gate array (FPGA), a discrete electronic hardware, or a combination thereof.

The storage unit204can include both hardware and the software212. For example, the software212can include control firmware. The storage unit204can include a volatile memory, a nonvolatile memory, an internal memory, an external memory, or a combination thereof. For example, the storage unit204can be a nonvolatile storage such as non-volatile random access memory (NVRAM), Flash memory, disk storage, or a volatile storage such as static random access memory (SRAM).

The storage unit204can include a storage interface214. The storage interface214can also be used for communication that is external to the memory controller104. The storage interface214can receive information from the other functional units or from external sources, or can transmit information to the other functional units or to external destinations. The external sources and the external destinations refer to sources and destinations external to the memory controller104.

The storage interface214can include different implementations depending on which functional units or external units are being interfaced with the storage unit204. The storage interface214can be implemented with technologies and techniques similar to the implementation of the control interface210.

The memory interface unit206can enable external communication to and from the memory controller104. For example, the memory interface unit206can permit the memory controller104to communicate with the memory array106ofFIG. 1.

The memory interface unit206can include a memory interface216. The memory interface216can be used for communication between the memory interface unit206and other functional units in the memory controller104. The memory interface216can receive information from the other functional units or can transmit information to the other functional units.

The memory interface216can include different implementations depending on which functional units are being interfaced with the memory interface unit206. The memory interface216can be implemented with technologies and techniques similar to the implementation of the control interface210.

The host interface unit208allows the host system108ofFIG. 1to interface and interact with the memory controller104. The host interface unit208can include a host interface218to provide communication mechanism between the host interface unit208and the host system108.

The control unit202can operate the host interface unit208to send control or status information generated by the memory controller104to the host system108. The control unit202can also execute the software212for the other functions of the memory controller104. The control unit202can further execute the software212for interaction with the memory array106via the memory interface unit206.

The functional units in the memory controller104can work individually and independently of the other functional units. For illustrative purposes, the memory controller104is described by operation of the memory controller104with the host system108and the memory array106. It is understood that the memory controller104, the host system108, and the memory array106can operate any of the modules and functions of the memory controller104.

Referring now toFIG. 3, therein is shown a functional block diagram of a scheduler smoothing function302of the memory controller104ofFIG. 1. Generally, the scheduler smoothing function302can be employed in the storage control system100ofFIG. 1and more specifically in the memory controller104. However, it is to be understood that the preceding examples are not meant to be limiting and the scheduler smoothing function302can be employed in any type of system that requires data management.

Generally, the scheduler smoothing function302can include a host write queue304, a recycle write queue306, a scheduler308, and a memory write operation queue310. The memory write operation queue310can interleave host and recycle writes to the memory devices110ofFIG. 1including flash devices. In at least one embodiment, the scheduler308can receive inputs from one or both of the host write queue304and the recycle write queue306to create an output write sequence that is delivered and implemented by the memory write operation queue310within the storage control system100.

Per the embodiments described herein, the term “host write” is defined herein as a physical write of new data from the host system108ofFIG. 1to write to a particular logical address range. Per the embodiments described herein, the term “recycle write” is defined herein as a physical write of data that the drive is moving due to recycling. Per the embodiments described herein, the term “scheduler” is defined herein as a module in the memory controller104including a solid-state drive (SSD) responsible for determining which operations to perform next to the memory devices110.

In general, the host write queue304can perform operations such as reads and writes submitted by a system external to the memory sub-system102ofFIG. 1. The recycle write queue306can perform operations including recycle writes312, such as reads, writes, and erases that the storage control system100must perform in order to free up space for host writes314and to maintain data integrity.

During operation of the scheduler smoothing function302, the host write queue304can send a host write request to the scheduler smoothing function302. The host write request can be associated with a physical write of new data from the host system108to write to a particular logical address range. Per the embodiments described herein, the term “physical write” is defined herein as a write that goes to an end storage element of a system of memory, such as a volatile memory or a non-volatile memory including a NAND flash device.

During operation of the scheduler smoothing function302, the recycle write queue306can also send a recycle write request to the scheduler smoothing function302. The recycle write request can be associated with a physical write of data that the storage control system100is moving due to recycling. Per the embodiments described herein, the term “recycling” is defined herein as moving data from one page to another page, for purposes of either freeing up erase blocks to write new host data or to ensure that data on the erase blocks is preserved. Recycling can also be referred to as garbage collection.

Per the embodiments described herein, the term “erase block” is defined herein as a group of pages that is the smallest number of pages that can be erased at one time. Per the embodiments described herein, the term “page” is defined herein as the smallest group of data bytes that can be read from or written to in an erase block.

It is to be understood that write requests from the host write queue304and the recycle write queue306can include any type of write operation or request, such as competing writes, metadata writes, RAID/parity writes, etc.

After receiving a number of host write requests and/or recycle write requests, the scheduler smoothing function302can then determine which write operation or sequence of write operations to perform next to optimize performance of the storage control system100. By way of example and not by way of limitation, the storage control system100can be optimized by providing a steady host performance to the user, a consistent host command latency period to the user, and/or maintenance of a desired recycle ratio.

Per the embodiments described herein, the term “recycle ratio” is defined herein as a number of logical pages that are written for recycling compared to a total number of data writes. Per the embodiments described herein, the term “host performance” is defined herein generally as how much work the host system108achieves when interfacing to the memory controller104including the SSD. For example, key measurements for the host performance can include throughput, average latency, worst-case latency, and latency deviation. The key measurements can be applied to any combinations of host write and host read distributions, sizes, and queue depths.

Once the scheduler smoothing function302has determined the correct write operation or sequence of write operations to perform next, the scheduler smoothing function302can then send this instruction to the memory write operation queue310for implementation within the memory array106ofFIG. 1.

Functions or operations of the memory controller104as described above can be implemented in hardware, software, or a combination thereof. The memory controller104can be implemented with the control unit202ofFIG. 2, the storage unit204ofFIG. 2, the memory interface unit206ofFIG. 2, the host interface unit208ofFIG. 2, or a combination thereof.

For example, the host write queue304can be implemented with the control unit202and the storage unit204to store and provide the host writes314. Also for example, the recycle write queue306can be implemented with the control unit202and the storage unit204to store and provide the recycle writes312.

For example, the scheduler308can be implemented with the control unit202to receive inputs from one or both of the host write queue304and the recycle write queue306to create an output write sequence that is delivered and implemented by the memory write operation queue310. Also for example, the memory write operation queue310can be implemented with the control unit202and the storage unit204to receive sequence of write operations to perform next from the scheduler308.

The host write queue304and the recycle write queue306can be coupled to the scheduler308. The scheduler308can be coupled to the memory write operation queue310.

The storage control system100is described with module functions or order as an example. The modules can be partitioned differently. For example, the scheduler308and the memory write operation queue310can be combined. Each of the modules can operate individually and independently of the other modules.

Furthermore, data generated in one module can be used by another module without being directly coupled to each other. The host write queue304, the recycle write queue306, the scheduler308, and the memory write operation queue310can be implemented as hardware accelerators (not shown) within the control unit202or can be implemented as hardware accelerators (not shown) in the memory controller104or outside of the memory controller104.

Referring now toFIG. 4, therein is shown an exemplary diagram of the scheduler smoothing function302. Generally, the scheduler smoothing function302can include the host write queue304, the recycle write queue306, the scheduler308, and the memory write operation queue310. However, in this embodiment, the scheduler smoothing function302can be targeted to a particular recycle ratio.

Generally, the memory devices110ofFIG. 1are limited by the bandwidth of a heavy write load. In order for the memory devices110to display uniform and/or even performance, the memory devices110should maintain a substantially even ratio of the recycle writes312and the host writes314. Any deviation from this ratio from second to second can show up in Inputs and Outputs per Second (IOPS) and latency measurements. This ratio can be commonly referred to as a recycle ratio402(RR) and can be expressed as:

Generally, a correlation can exist between a write amplification404(WA) and the recycle ratio402required for a particular value of the write amplification404. This correlation can be expressed as:

The recycle ratio402can be expressed as a function of the write amplification404. Particularly, the recycle ratio402can be expressed as one minus a reciprocal of the write amplification404.

Accordingly, even though the embodiments described herein generally focus on the recycle ratio402, the principles can apply equally well to a method and/or system that utilizes the write amplification404measurements. Per the embodiments described herein, the term “write amplification” is defined herein as a ratio of physical writes to a media compared to the host writes314to the memory device110.

In at least one embodiment, the scheduler smoothing function302can target a desired value of the recycle ratio402. For example, if the scheduler smoothing function302knows a ratio of the host writes314that it should perform relative to total writes, it can target that ratio in its scheduling and as a result be as responsive to the host system108ofFIG. 1as possible while fulfilling the recycling work that the memory devices110need to continue functioning.

Accordingly, after receiving inputs from the host write queue304and the recycle write queue306, the scheduler smoothing function302can then organize these inputs pursuant to a predefined value of a target recycle ratio406. Per the embodiments described herein, the term “target recycle ratio” is defined herein as a recycle ratio that a scheduling algorithm is targeting. This is abbreviated as RRt.

For example, if the target recycle ratio406is 0.75, the scheduler smoothing function302can then output an instruction to the memory write operation queue310indicating that roughly three (3) of every four (4) writes should be the recycle writes312. The target recycle ratio406of 0.75 is depicted in the memory write operation queue310, wherein the order in which the scheduler smoothing function302can dispatch the write operations to the memory devices110is shown. However, it is to be understood that the target recycle ratio406is not limited to the preceding exemplary value of 0.75 and the target recycle ratio406can include any value between and including 0.0 and 1.0.

As such, when the behavior of the host system108varies and/or as the needs of the memory devices110change, the target recycle ratio406can change as well. Accordingly, in at least one embodiment of the invention, if the storage control system100ofFIG. 1determines that the scheduler smoothing function302would benefit from a new value of the target recycle ratio406, the scheduler smoothing function302can be reprogrammed to use the new value of the target recycle ratio406.

It has been discovered that the scheduler smoothing function302permits enhanced flexibility for the needs of the host system108and the memory devices110. For example, the scheduler smoothing function302permits the storage control system100to perform the recycle writes312when there is none of the host writes314to schedule. As such, this enhanced flexibility allows the memory devices110to catch up or get ahead on the recycle writes312so that it does not have to perform as many later on if the host system108decides to perform more of the host writes314.

It has been discovered that the enhanced flexibility of the scheduler smoothing function302also permits the memory devices110to perform a burst of the host writes314even if the recycle ratio402would normally indicate doing more of the recycle writes312. The scheduler smoothing function302of the present embodiments permits a burst of the host writes314because the memory devices110can perform additional recycle writes later thereby providing better performance to the host system108in cases where the host write activity is burst oriented.

Referring now toFIG. 5, therein is shown a first exemplary graph of the recycle ratio402. Generally, the first exemplary graph plots the recycle ratio402on the Y-axis against a spare pool size502on the X-axis. Per the embodiments described herein, the term “spare pool” is defined herein as erased memory that is available to be written to and can be in units of erase blocks. In at least one embodiment, the first exemplary graph can be used to determine a value for the target recycle ratio406ofFIG. 4that the storage control system100ofFIG. 1and/or the scheduler smoothing function302ofFIGS. 3 and 4can use.

Generally, the storage control system100, when under a constant host workload, can dynamically determine a steady-state recycle ratio or the recycle ratio402from the first exemplary graph. In at least one embodiment, the steady state recycle ratio or value can be determined by having the storage control system100sample an available spare pool size or the spare pool size502and assigning a corresponding recycle ratio or the recycle ratio402. As such, the recycle ratio402can be set based on a current size of a spare pool or the spare pool size502.

Accordingly, in at least one embodiment, as the spare pool shrinks in size, the storage control system100can increase the recycle ratio402. Conversely, as the spare pool increases in size, the memory system can decrease the recycle ratio402. One possible correlation between the spare pool size502and the recycle ratio402is depicted inFIG. 5.

Generally, the first exemplary graph includes a first region504and a second region506. The first region504can include a substantially horizontal portion of the graph wherein the value of the recycle ratio402remains substantially constant around a value of one (1) over a range of the spare pool size502that varies from zero (0) erase blocks left to a value deemed “critically low.” The numerical value assigned to the qualitative term “critically low” can be predetermined or it can be empirically determined.

In at least one embodiment, factors used to empirically determine the value can include, but are not limited to, age of the memory devices110ofFIG. 1, recent host performance requirements, latency history, bad block management, etc. In another embodiment, a “critically low” value can occur when a number of free erase blocks approaches approximately 0.1% of a total number of the erase blocks. In yet another embodiment, a “critically low” value can occur when a number of free or empty erase blocks is equal to or less than approximately 20 out of the 4,000 erase blocks on each die or each of the memory devices110.

The second region506can include a substantially linear portion of the graph with a decreasing slope, wherein the value of the recycle ratio402decreases from around a value of one (1) to a value of zero (0). The value of the recycle ratio402can decrease over a range of the spare pool size502that varies from a value deemed “critically low” to a value corresponding to the “maximum” spare pool size. The numerical value assigned to the qualitative term “maximum” can be predetermined or it can be empirically determined.

In at least one embodiment, if the spare pool grows to a maximum size, the memory devices110can stop performing recycle operations because the recycle ratio402can be set to zero. However, it will be appreciated by those skilled in the art that the memory devices110need not totally stop performing recycle operations and can continue performing background-recycling operations. Generally, the value of the spare pool size502between “critically low” and “maximum” can act as a buffer region to help determine the potentially optimal steady state value for the recycle ratio402.

It will be appreciated by those skilled in the art that if the recycle ratio402is not high enough to maintain a steady state for the storage control system100, the size of the spare pool or the spare pool size502can decrease. Accordingly, in response, the first exemplary graph of the present embodiments would adjust the recycle ratio402to a higher value. In such cases, one of two things can happen: 1) the recycle ratio402can be set to a high enough value to be sustainable, or 2) the spare pool can become “critically low” and servicing of host operations can cease.

If the recycle ratio402is set to a high enough value to be sustainable, the storage control system100should reach a steady state in regards to a number of recycle operations that are performed for every host operation. It will be appreciated by those skilled in the art that when in this steady state, the spare pool size502changes very little, if at all.

If the spare pool becomes critically low and servicing of host operations ceases, the first exemplary graph can command the storage control system100to perform only recycle operations. In such cases, once the spare pool size502gets above this critically low mark, the recycle ratio402can be set to a value that allows some host operations to be serviced. It will be appreciated by those skilled in the art that this critically low mark should be used only in extreme cases as a safety net to prevent device failure or extended inoperable responses.

Once a steady state for a particular host workload is determined, the recycle ratio402and the host performance should be constant. However, when the host changes the workload, a different steady state can be required by the storage control system100. For example, if the new workload requires a higher value of the recycle ratio402, the spare pool can shrink and the storage control system100can adapt by increasing the recycle ratio402until it reaches a new steady state.

Conversely, if the new workload requires a lower value of the recycle ratio402, the spare pool can slowly grow. It will be appreciated by those skilled in the art that in response to the larger spare pool, the recycle ratio402can decrease pursuant to the first exemplary graph thereby permitting more of the host writes314ofFIG. 3to be performed.

As such, a method and/or system have been discovered for dynamically adjusting the recycle ratio402of the memory devices110. The dynamic adjustment of the recycle ratio402of the present embodiments happens directly because of the spare pool shrinking or growing in size, so the memory devices110do not have to try to predict the recycle ratio402that can be needed by the host system108ofFIG. 1.

It will be appreciated by those skilled in the art that additional equations and/or curves can be used to translate the spare pool size502into a target value of the recycle ratio402. Accordingly, the function or algorithm used to determine the recycle ratio402does not have to be linear in regards to the spare pool size502. The function may be exponential, logarithmic, mapping, etc.

It will be appreciated by those skilled in the art that if the recycle ratio402reaches a value of one (1) (e.g., the maximum value), the scheduler smoothing function302need not service the host writes314.

It will also be appreciated by those skilled in the art that the current embodiments permit the memory devices110to perform some recycling operations including the recycle writes312ofFIG. 3when there are no host operations including the host writes314.

Referring now toFIG. 6, therein is shown a second exemplary graph of the recycle ratio402. Generally, the second exemplary graph plots the recycle ratio402on the Y-axis against the spare pool size502on the X-axis. In at least one embodiment, the second exemplary graph can be used to determine a target value for the recycle ratio402that the storage control system100ofFIG. 1and/or the scheduler smoothing function302ofFIGS. 3 and 4can use. Per this embodiment, the second exemplary graph can include periods of little change in the recycle ratio402and periods of larger change in the recycle ratio402.

The second exemplary graph can include a transition region602and a stable region604. The transition region602includes a portion of the graph wherein the recycle ratio402experiences a relatively large change over a relatively small range of the spare pool size502. In at least one embodiment, the recycle ratio402can change by five percent (5%) or more in the transition region602.

The stable region604marks a portion of the second exemplary graph wherein the recycle ratio402remains relatively constant over a range of values of the spare pool size502. By way of example and not by way of limitation, the spare pool size502can change by five percent (5%) or more in the stable region604, while the recycle ratio402can remain relatively constant over that range. As such, the second exemplary graph permits a correlation between the recycle ratio402and the spare pool size502that can adjust in ranges.

It will be appreciated by those skilled in the art that additional equations and/or curves can be used to translate the spare pool size502into the target value of the recycle ratio402. Accordingly, the function or algorithm used to determine the recycle ratio402does not have to be linear in regards to the spare pool size502. The function can be exponential, logarithmic, mapping, etc.

Referring now toFIG. 7, therein is shown a third exemplary graph of the recycle ratio402. The third exemplary graph can include workload transitions. Generally, the third exemplary graph plots the recycle ratio402on the Y-axis against a time702on the X-axis. In at least one embodiment, the third exemplary graph can depict a transitional period704for the recycle ratio402that occurs between different steady states706that the storage control system100ofFIG. 1and/or the scheduler smoothing function302ofFIGS. 3 and 4can use.

Generally, when the host system108ofFIG. 1changes workloads, there can be the transitional period704when the recycle ratio402is higher than its new steady state. During this time, host performance may not be steady.

A dotted line depicts a transitional period response of the transitional period704for the embodiments described herein, wherein host performance degradation is minimized. It will be appreciated by those skilled in the art that host performance degradation is minimized with the transitional period response of the dotted line because the change in the recycle ratio402is minimized. A dashed line above the dotted line depicts another transitional period response of the transitional period704wherein the host experiences greater host performance degradation due to the larger increase in the recycle ratio402.

It has been discovered that by using the recycle ratio402functions described herein and/or the spare pool size502ofFIG. 5described herein that the uneven host performance caused by workload transitions is significantly reduced.

Referring now toFIG. 8, therein is shown a control flow of the scheduler smoothing function302. The scheduler smoothing function302can be implemented in the memory controller104ofFIG. 1with the host write queue304ofFIG. 3, the recycle write queue306ofFIG. 3, the scheduler308ofFIG. 3, and the memory write operation queue310ofFIG. 3. The control flow depicts the scheduler smoothing function302implemented as a smoothing feedback loop. Generally, the smoothing feedback loop teaches how to perform the mixing of the host writes314ofFIG. 3and the recycle writes312ofFIG. 3that the scheduler smoothing function302implements.

In at least one embodiment, the scheduler smoothing function302can implement the recycle ratio402ofFIG. 4by using the smoothing feedback loop to determine whether the scheduler smoothing function302needs to schedule one of the host writes314or the recycle writes312next. In such cases, the smoothing feedback loop can track and/or manipulate a current recycle ratio802(RRc) that the scheduler smoothing function302has sent out. Per the embodiments described herein, the term “current recycle ratio” is defined herein as a recycle ratio that a scheduling algorithm has been achieving over a predetermined recent history. This is abbreviated as RRc.

The smoothing feedback loop can begin with an update RRcmodule803. The update RRcmodule803ensures that the smoothing feedback loop utilizes the most current recycle ratio or the most current value of the recycle ratio402for its algorithm or calculation. In at least one embodiment, the update RRcmodule803can determine the current recycle ratio802or a current value of the recycle ratio402by utilizing inputs from a schedule host write module812and/or a schedule recycle write module816. The current value of the recycle ratio402of the update RRcmodule803can then be fed to a comparison module804.

In the comparison module804, the current recycle ratio802can be compared to the target recycle ratio406(RR). Generally, if the current recycle ratio802is greater than or equal to the target recycle ratio406, the smoothing feedback loop can choose a first path. If the current recycle ratio802is less than the target recycle ratio406, the smoothing feedback loop can choose a second path.

Accordingly, in at least one embodiment, if the current recycle ratio802is greater than or equal to the target recycle ratio406(RRc≥RRt), the smoothing feedback loop can then move to a host scheduler module808. Upon receiving an input from the comparison module804, the host scheduler module808can determine if there is one of the host writes314to schedule. If there is one of the host writes314to schedule, the smoothing feedback loop can then move to the schedule host write module812. The schedule host write module812can then send its schedule host write request to the update RRcmodule803to be used in determining the current recycle ratio802.

If there is none of the host writes314to schedule at the host scheduler module808, the smoothing feedback loop can then move to a recycle scheduler module810. If there is one of the recycle writes312to schedule, the smoothing feedback loop can then move to the schedule recycle write module816. The schedule recycle write module816can then send its schedule recycle write request to the update RRcmodule803to be used in determining the current recycle ratio802. If there is none of the recycle writes312to schedule when the smoothing feedback loop reaches the recycle scheduler module810, the smoothing feedback loop can then move back to the comparison module804.

In another embodiment, if the current recycle ratio802is less than the target recycle ratio406(RRc<RRt) or there is none of the recycle writes312to schedule from the recycle scheduler module810when the smoothing feedback loop reaches the comparison module804, the smoothing feedback loop can then move to a host write burst module814. If the smoothing feedback loop determines that a host write burst can be performed at the host write burst module814, the smoothing feedback loop can then move to the host scheduler module808and can proceed as described above. If the smoothing feedback loop determines that a host write burst cannot be performed at the host write burst module814, the smoothing feedback loop can then move to the recycle scheduler module810and can proceed as described above.

It will be appreciated by those skilled in the art that the smoothing feedback loop optimizes host performance by keeping the current recycle ratio802close to the target recycle ratio406when a steady stream of host activity is observed. By way of example and not by way of limitation, when the steady stream of the host activity is observed, the current recycle ratio802is kept within plus or minus ten percent (10%) of the target recycle ratio406.

It will be appreciated by those skilled in the art that the smoothing feedback loop optimizes host performance by allowing the current recycle ratio802to drop relative to the target recycle ratio406for a short period of time when a burst of the host activity is observed. By way of example and not by way of limitation, the current recycle ratio802varies by about ten percent (10%) from the target recycle ratio406in such instances.

It will be appreciated by those skilled in the art that the smoothing feedback loop optimizes host performance by allowing the current recycle ratio802to grow or increase relative to the target recycle ratio406for a period of time when a lull in host activity is observed. By way of example and not by way of limitation, the current recycle ratio802varies by about ten percent (10%) from the target recycle ratio406in such instances.

It has been discovered that such a method and/or system enhances the flexibility of the memory devices110ofFIG. 1by allowing the smoothing feedback loop and/or the scheduler smoothing function302to handle various scenarios and track how far it has deviated from the target recycle ratio406. At the same time, the smoothing feedback loop and/or the scheduler smoothing function302are permitted to converge on the actual value of the target recycle ratio406later.

Functions or operations of the memory controller104as described above can be implemented in hardware, software, or a combination thereof. The memory controller104can be implemented with the control unit202ofFIG. 2, the storage unit204ofFIG. 2, the memory interface unit206ofFIG. 2, the host interface unit208ofFIG. 2, or a combination thereof.

For example, the update RRcmodule803can be implemented with the control unit202to ensure that the smoothing feedback loop utilizes the most current value of the recycle ratio402and determine the current value of the recycle ratio402by utilizing inputs from the schedule host write module812and/or the schedule recycle write module816. Also for example, the comparison module804can be implemented with the control unit202to compare the current recycle ratio802to the target recycle ratio406.

For example, the host scheduler module808can be implemented with the control unit202to determine if there is one of the host writes314to schedule. Also for example, the recycle scheduler module810can be implemented with the control unit202to determine if there is one of the recycle writes312to schedule.

For example, the schedule host write module812can be implemented with the control unit202to schedule the host writes314. Also for example, the host write burst module814can be implemented with the control unit202to determine if a host write burst can be performed. Further, for example, the schedule recycle write module816can be implemented with the control unit202to schedule one of the recycle writes312.

The update RRcmodule803can be coupled to the comparison module804, the schedule host write module812, and the schedule recycle write module816. The comparison module804can be coupled to the host scheduler module808, the recycle scheduler module810, and the host write burst module814.

The host scheduler module808can be coupled to the recycle scheduler module810, the schedule host write module812, and the host write burst module814. The recycle scheduler module810can be coupled to the host write burst module814and the schedule recycle write module816.

The storage control system100ofFIG. 1is described with module functions or order as an example. The modules can be partitioned differently. For example, the recycle scheduler module810and the schedule recycle write module816can be combined. Each of the modules can operate individually and independently of the other modules.

Furthermore, data generated in one module can be used by another module without being directly coupled to each other. The update RRcmodule803, the comparison module804, and the host scheduler module808can be implemented as hardware accelerators (not shown) within the control unit202or can be implemented as hardware accelerators (not shown) in the memory controller104or outside of the memory controller104. The recycle scheduler module810, the schedule host write module812, the host write burst module814, and the schedule recycle write module816can be implemented as hardware accelerators (not shown) within the control unit202or can be implemented as hardware accelerators (not shown) in the memory controller104or outside of the memory controller104.

Referring now toFIG. 9, therein is shown a functional block diagram of a moving average smoothing function902of the scheduler smoothing function302ofFIG. 3. Generally, the moving average smoothing function902can be used to implement the scheduler smoothing function302. The functional block diagram can include a smoothing functional circuit. Note that although this diagram is in circuit form, it can be implemented in software, hardware, or a combination thereof as well.

In at least one embodiment, a moving average of recent writes can be used to adjust the current recycle ratio802. In such cases, the moving average of the recent writes can be sampled from a flash write operation queue or the memory write operation queue310.

The memory write operation queue310can include one or more write commands, including the host writes314and/or the recycle writes312. The host writes314and the recycle writes312within the memory write operation queue310can be recent write operations.

The recent write operations can be given weighting factors or weights908using multiplication modules910. Based on the weights908, the moving average smoothing function902can determine whether the host writes314or the recycle writes312should happen next. The moving average smoothing function902can also feed the decision into a recent history window so that it can influence the next decision. Outputs of the multiplication modules910can be fed to a summation module912to calculate the current recycle ratio802.

Functions or operations of the memory controller104ofFIG. 1as described above can be implemented in hardware, software, or a combination thereof. The memory controller104can be implemented with the control unit202ofFIG. 2, the storage unit204ofFIG. 2, the memory interface unit206ofFIG. 2, the host interface unit208ofFIG. 2, or a combination thereof.

For example, the multiplication modules910can be implemented with the control unit202to multiply the recent write operations, including the recycle writes312and the host writes314, and the weights908. Also for example, the summation module912can be implemented with the control unit202to calculate the current recycle ratio802based on the outputs or results of the multiplication modules910. The multiplication modules910can be coupled to the summation module912and the memory write operation queue310.

The storage control system100ofFIG. 1is described with module functions or order as an example. The modules can be partitioned differently. For example, the multiplication modules910and the summation module912can be combined. Each of the modules can operate individually and independently of the other modules.

Furthermore, data generated in one module can be used by another module without being directly coupled to each other. The multiplication modules910and the summation module912can be implemented as hardware accelerators (not shown) within the control unit202or can be implemented as hardware accelerators (not shown) in the memory controller104or outside of the memory controller104.

Referring now toFIG. 10, therein is shown a functional block diagram of an exponential moving average smoothing function1002of the scheduler smoothing function302ofFIG. 3. Generally, the exponential moving average smoothing function1002can be used to implement the scheduler smoothing function302. The functional block diagram can include a smoothing functional circuit. Note that although this diagram is in circuit form, it can be implemented in software, hardware, or a combination thereof as well.

A preferred method of tracking the current recycle ratio802using a moving average can include an exponential moving average because it can be the easiest to compute. The exponential moving average can be the easiest to compute because it can require only the current recycle ratio802, denoted as RRC(n), not a buffer of previous decisions.

The exponential moving average can require no divisions, but instead, just or only a subtraction module1004, a multiplication module1006, and an addition module1008each time. In addition, the exponential moving average can behave well on fixed-point hardware. If all writes for a chosen window size are given equal weights, the smoothing function can be a simple-moving average function that is similar to a batch smoothing function, which will be subsequently described.

The functional block diagram depicts a decision module1010, which determines a decision, denoted as Δ(n), of whether the host writes314ofFIG. 3or the recycle writes312ofFIG. 3should happen next. The decision module1010can provide the decision as an input to the subtraction module1004. The subtraction module1004can subtract the current recycle ratio802from the decision. InFIG. 10, Δ(n) can have a value of “0” or “1” when one of the host writes314or the recycle writes312, respectively, is to occur at time n.

A subtraction result from the subtraction module1004can be multiplied by an exponential average parameter or a weight1012, denoted as α, by the multiplication module1006, a result of which can then be added to the current recycle ratio802by the addition module1008to generate an updated current recycle ratio1014, denoted as RRC(N+1). The updated current recycle ratio1014can be fed to a unit delay module1016to delay the updated current recycle ratio1014by a predetermined unit of time to generate a value of the current recycle ratio802for subsequent processing.

The functional block diagram depicts an initialization module1018. The initialization module1018configures the current recycle ratio802(RRc) for the scheduler smoothing function302. For example, a good starting point is to set the current recycle ratio802to be equal to the target recycle ratio406(RRt) ofFIG. 4.

In the drive, the target recycle ratio406can be calculated every time the spare pool size502ofFIG. 5changes, and the current recycle ratio802can be immediately set to a new ratio that is equal to the target recycle ratio406so the scheduler308ofFIG. 3can start recycling to the new ratio. If the current recycle ratio802were not set to be equal to the target recycle ratio406when the target recycle ratio406moved or changed and if there was a gap between the current recycle ratio802and the target recycle ratio406, we could end up with a burst of only the host writes314or the recycle writes312getting serviced.

Functions or operations of the memory controller104ofFIG. 1as described above can be implemented in hardware, software, or a combination thereof. The memory controller104can be implemented with the control unit202ofFIG. 2, the storage unit204ofFIG. 2, the memory interface unit206ofFIG. 2, the host interface unit208ofFIG. 2, or a combination thereof.

For example, the subtraction module1004can be implemented with the control unit202to subtract the current recycle ratio802from the decision of whether the host writes314or the recycle writes312should happen next. Also for example, the multiplication module1006can be implemented with the control unit202to multiply the subtraction result from the subtraction module1004by the weight1012.

For example, the addition module1008can be implemented with the control unit202to add the result of the multiplication module1006to the current recycle ratio802to generate the updated current recycle ratio1014. Also for example, the decision module1010can be implemented with the control unit202to determine the decision of whether the host writes314or the recycle writes312should happen next.

For example, the unit delay module1016can be implemented with the control unit202to delay the updated current recycle ratio1014by the predetermined unit of time to generate the value of the current recycle ratio802. Also for example, the initialization module1018can be implemented with the control unit202to generate and supply initialization parameters for the target recycle ratio406ofFIG. 4.

The subtraction module1004can be coupled to the multiplication module1006, the decision module1010, and the initialization module1018. The decision module1010can be coupled to the addition module1008. The addition module1008can be coupled to the unit delay module1016and the initialization module1018.

The storage control system100ofFIG. 1is described with module functions or order as an example. The modules can be partitioned differently. For example, the subtraction module1004and the multiplication module1006can be combined. Each of the modules can operate individually and independently of the other modules.

Furthermore, data generated in one module can be used by another module without being directly coupled to each other. The subtraction module1004, the multiplication module1006, the addition module1008, the decision module1010, the unit delay module1016, and the initialization module1018can be implemented as hardware accelerators (not shown) within the control unit202or can be implemented as hardware accelerators (not shown) in the memory controller104or outside of the memory controller104.

Referring now toFIG. 11, therein is shown a functional block diagram of a batch smoothing function1102of the scheduler smoothing function302ofFIG. 3. Generally, the batch smoothing function1102can be used to implement the scheduler smoothing function302. The functional block diagram can include a smoothing functional circuit.

This type of smoothing function considers work in batches, using the target recycle ratio406ofFIG. 4to decide how many of the host writes314and the recycle writes312to schedule for that batch. That is, at the start of each batch, the smoothing function can create a schedule1104of an order in which host and recycle work can occur for that batch.

For example,FIG. 11depicts the storage control system100ofFIG. 1having the target recycle ratio406of 0.75. At the start of each batch, the scheduler308can create the schedule1104with the host writes314and the recycle writes312, which the scheduler308plans to perform for that batch.

The scheduler308can then work its way down the scheduler308, pulling from an appropriate write queue for a host slot1106or a recycle slot1108, for which the scheduler308is currently processes. The appropriate write queue can be the host write queue304or the recycle write queue306. The scheduler308can pull the host writes314or the recycle writes312from the appropriate write queue for the host slot1106or the recycle slot1108, respectively.

The scheduler308can update the memory write operation queue310with an order by which the host writes314and the recycle writes312are executed. When a batch completes, the scheduler308can create a new schedule or another of the schedule1104, if necessary.

With the batch smoothing function1102, the scheduler308can still choose to deviate from the target recycle ratio406for periods of time, as indicated inFIG. 8. Thus, with the batch smoothing function1102, there is still a feedback loop to ensure that a device or the memory controller104ofFIG. 1can be continually converging on the target recycle ratio406.

It has been discovered that the memory sub-system102ofFIG. 1including a solid-state drive (SSD) employing the scheduler smoothing function302for scheduling a mixture of the host writes314and the recycle writes312improves host performance. The host performance is improved by maintaining steady host performance and consistent host command latencies for providing even performance to the host system108ofFIG. 1.

It has also been discovered that the memory sub-system102including the SSD with the scheduler smoothing function302that utilizes the target recycle ratio406improves host performance. The host performance is improved by using the target recycle ratio406to schedule the mixture of the host writes314and the recycle writes312to maintain steady host performance and consistent host command latencies.

It has further been discovered that the memory sub-system102including the SSD with the scheduler smoothing function302that utilizes a target write amplification to schedule a mixture of the host writes314and the recycle writes312improves host performance by providing steady host performance and consistent host command latencies.

It has further been discovered that the memory sub-system102dynamically determining the recycle ratio402ofFIG. 4provides a steady state host performance and consistent host command latencies thereby improving host performance.

It has further been discovered that the memory sub-system102adjusting the recycle ratio402based on the spare pool size502ofFIG. 5provides a steady state host performance and consistent host command latencies thereby improving host performance.

It has further been discovered that the scheduler smoothing function302ceases servicing the host writes314only if the recycle ratio402reaches one (1) or the spare pool size502becomes critically low provides improved reliability by servicing only the recycle writes312to increase the spare pool size502to prevent device failure or extended inoperable responses.

It has further been discovered that the memory sub-system102including the SSD stops or ceases to perform recycle operations or the recycle writes312altogether if the recycle ratio402becomes zero improves host performance by servicing only the host writes314in response to the larger spare pool.

It has further been discovered that the memory sub-system102including the SSD continues to perform the recycling operations in the background at a low rate if the recycle ratio402becomes zero improves host performance by servicing more of the host writes314in response to the larger spare pool.

It has further been discovered that the memory sub-system102optimizes host performance by adjusting the recycle ratio402and the spare pool size502to help even out host performance during workload transitions or the transitional period704ofFIG. 7.

It has further been discovered that the memory sub-system102including the SSD performing some of the recycle writes312when there are no host operations or the host writes314provides improved reliability. The improved reliability is provided because the recycle writes312increase the spare pool size502thereby preventing device failure or extended inoperable responses.

It has further been discovered that a solid-state drive (SSD) or the memory sub-system102that updates the target recycle ratio406based on host and drive activity to update the scheduler smoothing function302improves host performance by maintaining steady host performance and consistent host command latencies. The target recycle ratio406is updated based on the host and drive activity based on an appropriate mixture of the host writes314and the recycle writes312.

It has further been discovered that the SSD or the memory sub-system102with the scheduler smoothing function302that employs the smoothing feedback loop improves host performance by maintaining steady host performance and consistent host command latencies. The host performance is improved using the smoothing feedback loop to converge on the target recycle ratio406.

It has further been discovered that the SSD or the memory sub-system102with the scheduler smoothing function302that schedules a mixture by utilizing the moving average smoothing function902ofFIG. 9improves host performance by maintaining steady host performance and consistent host command latencies. The moving average smoothing function902improves host performance using a moving average of recent scheduling activity, including the host writes314and the recycle writes312, and comparing it to the target recycle ratio406.

It has further been discovered that the SSD or the memory sub-system102with the scheduler smoothing function302that allows the SSD to get ahead on its maintenance/recycling work improves host performance by maintaining steady host performance and consistent host command latencies. The SSD gets ahead on its maintenance/recycling work by scheduling extra cycles for the recycle writes312when the host system108is not sending full of the host writes314.

It has further been discovered that the SSD or the memory sub-system102with the scheduler smoothing function302that allows the SSD to service a burst of the host writes314improves host performance by maintaining steady host performance and consistent host command latencies. The host performance is improved by the comparison module804ofFIG. 8, the host scheduler module808ofFIG. 8, and the schedule host write module812ofFIG. 8to provide a quick response to the host system108and catch up in the target recycle ratio406later.

It has further been discovered that the SSD or the memory sub-system102with the scheduler smoothing function302that schedules a mixture of the host writes314and the recycle writes312improves host performance by maintaining steady host performance and consistent host command latencies. The host performance is improved by choosing which types of writes to schedule, including the host writes314and the recycle writes312, in batches using the batch smoothing function1102in accordance with the target recycle ratio406.

The physical transformation of scheduling the recycle writes312and the host writes314based on the recycle ratio402for writing to the memory devices110ofFIG. 1results in movement in the physical world, such as people using the memory sub-system102based on the operation of the storage control system100. As the movement in the physical world occurs, the movement itself creates additional information that is converted back in to receiving the recycle writes312from the recycle write queue306and receiving the host writes314from the host write queue304for the continued operation of the storage control system100and to continue the movement in the physical world.

Referring now toFIG. 12, therein is shown a flow chart of a method1200of operation of a storage control system in a further embodiment of the present invention. The method1200includes: receiving a recycle write from a recycle write queue in a block1202; receiving a host write from a host write queue in a block1204; and scheduling the recycle write and the host write for writing to a memory device in a block1206.

Thus, it has been discovered that the storage control system of the present invention furnishes important and heretofore unknown and unavailable solutions, capabilities, and functional aspects for a storage control system with data management mechanism. The resulting method, process, apparatus, device, product, and/or system is straightforward, cost-effective, uncomplicated, highly versatile, accurate, sensitive, and effective, and can be implemented by adapting known components for ready, efficient, and economical manufacturing, application, and utilization.