Employing multiple queueing structures within a userspace storage driver to increase speed

Performance in multi-core data storage systems is increased while allowing for portability and fast failover in the event of a failure of a driver stack by a data storage system employing several queues to reduce lock contention. Queueing is performed with two levels of several queues each within a userspace scheduling driver within a userspace container. The userspace scheduling driver may dequeue into a userspace management driver that communicates with a kernel-based hardware driver by way of a kernel helper driver. An apparatus, system, and computer program product for performing a similar method are also provided.

BACKGROUND

A data storage system is an arrangement of hardware and software that typically includes one or more storage processors coupled to an array of non-volatile data storage devices, such as magnetic disk drives, electronic flash drives, and/or optical drives. The storage processors service host input/output (I/O) operations received from host machines. The received I/O operations specify storage objects (e.g. logical disks or “LUNs”) that are to be written to, read from, created, or deleted. The storage processors run software that manages incoming I/O operations and that performs various data processing tasks to organize and secure the host data received from the host machines and stored on the non-volatile data storage devices

Some data storage systems employ a storage stack to process and transform I/O operations from one format to another to increase speed and versatility. Once an I/O operation is transformed into a set of low-level I/O operations directed at physical extents of storage on storage drives, these low-level I/O operations may be queued and executed according to various policies to ensure fairness and increase efficiency.

SUMMARY

Unfortunately, conventional data storage systems that utilize several parallel processing cores may experience performance limitations when a large number of low-level I/O operations are directed to a physical drive within a short period of time. Such limitations are largely due to lock contention on the queue between the several processing cores. Contention may become more noticeable when modern flash-based drives capable of processing several concurrent I/O operations are used, as those devices are capable of processing several hundred thousand I/O operations (or more) per second, which can easily overwhelm a single queue having a lock contention issue.

Thus, it would be desirable to decrease performance degradation due to locking. Such a result may be accomplished by a data storage system employing several queues to reduce lock contention. It would further be desirable to perform this queuing within a userspace driver within a userspace container to allow for portability and fast failover to a new userspace container in the event of a failure of the driver stack. This may be accomplished by performing the queueing with two levels of several queues within a userspace scheduling driver within a userspace container. The userspace scheduling driver may dequeue into a userspace management driver that communicates with a kernel-based hardware driver by way of a kernel helper driver.

In some embodiments, a method of processing storage requests directed to a storage device of a computing device having a plurality of processing cores (hereinafter “cores”) is performed. The method includes (a) sending, by a first storage driver operating within userspace of the computing device, storage requests initiated by a first core of the computing device to a first userspace queue, the first userspace queue being dedicated to storage requests from the first core, (b) sending, by the first storage driver operating within userspace, storage requests initiated by a second core of the computing device to a second userspace queue, the second userspace queue being dedicated to storage requests from the second core, the second core being distinct from the first core, and the second userspace queue being distinct from the first userspace queue, (c) sending, by the first storage driver operating within userspace, storage requests from the first userspace queue and the second userspace queue to a set of userspace dispatch queues, the first userspace queue and the second userspace queue not belonging to the set of userspace dispatch queues, (d) sending, by the first storage driver operating within userspace, storage requests from the set of userspace dispatch queues to a second storage driver operating within userspace of the computing device, the second storage driver being distinct from the first storage driver, and (e) sending, by the second storage driver operating within userspace, by way of a kernel helper function, the storage requests received from the first storage driver to a hardware device driver for the storage device for performance by the storage device, the hardware device driver for the storage device operating within a kernel of the computing device. An apparatus, system, and computer program product for performing a similar method are also provided.

The foregoing summary is presented for illustrative purposes to assist the reader in readily grasping example features presented herein. However, the foregoing summary is not intended to set forth required elements or to limit embodiments hereof in any way.

DETAILED DESCRIPTION OF THE INVENTION

Embodiments are directed to techniques for increasing performance in multi-core data storage systems while allowing for portability and fast failover in the event of a failure of the driver stack. This may be accomplished by a data storage system employing several queues to reduce lock contention. The queueing is performed with two levels of several queues each within a userspace scheduling driver within a userspace container. The userspace scheduling driver may dequeue into a userspace management driver that communicates with a kernel-based hardware driver by way of a kernel helper driver.

FIG. 1depicts an example environment30including a computing device32serving as a data storage system (DSS). DSS computing device32may be any kind of computing device, such as, for example, a personal computer, workstation, server computer, enterprise server, DSS rack server, laptop computer, tablet computes, smart phone, mobile computer, etc. Typically, computing device30is a DSS rack server.

DSS computing device32includes network interface circuitry34, processing circuitry36, memory40, storage interface circuitry42, and persistent data storage drives44(depicted as storage device44A, optional storage device44B, . . . ). DSS computing device32may also include other components as are well-known in the art, including interconnection circuitry.

Network interface circuitry34may include one or more Ethernet cards, cellular modems, Fibre Channel (FC) adapters, Wireless Fidelity (Wi-Fi) wireless networking adapters, and/or other devices for connecting to a network (not depicted). Network interface circuitry34allows the DSS computing device32to communicate with one or more host devices (not depicted) capable of sending data storage commands to the DSS computing device for fulfillment.

Processing circuitry36may be any kind of processor or set of processors configured to perform operations, such as, for example, a microprocessor, a multi-core microprocessor, a digital signal processor, a system on a chip, a collection of electronic circuits, a similar kind of controller, or any combination of the above. Processing circuitry36includes multiple processing cores38(depicted as cores38(1),38(2),38(3), . . . ). Each core38may be a distinct physical core or it may be a virtual core (e.g., due to hyper-threading). Thus, for example, if DSS computing device32includes two microprocessors, each microprocessor having four physical cores with hyper-threading enabled, then the DSS computing device32would have a total of sixteen cores38. In some embodiments, DSS computing device32may be built as a set of two or more storage processors (SPs, not depicted) each mounted on a separate board, each SP having its own network interface circuitry34, processing circuity36, memory40, and storage interface circuitry42, but sharing the storage devices44between them. In such embodiments, a high-speed inter-SP bus may connect the SPs. There may be more than one SP installed in DSS30for redundancy and performance reasons. In these embodiments, each SP may be considered independently for purposes of this disclosure.

Persistent storage drives44may include any kind of persistent storage devices, such as, for example, hard disk drives, solid-state storage devices (SSDs), flash drives, etc. In a typical embodiment, one or more of the storage devices44(e.g., storage drive44A) is an SSD or a flash drive having multiple channels46(depicted as channels46A-i,46A-ii, . . . ) allowing more than one storage operation to be performed by that storage device44A simultaneously.

Storage interface circuitry42controls and provides access to persistent storage drives44. Storage interface circuitry42may include, for example, SCSI, SAS, ATA, SATA, FC, M.2, and/or other similar controllers and ports.

Memory40may be any kind of digital system memory, such as, for example, random access memory (RAM). Memory40stores an operating system (OS) kernel50in operation (e.g., a Linux, UNIX, Windows, MacOS, or similar operating system kernel). Memory40also includes a userspace portion48within which non-kernel OS applications (not depicted) as well as user applications (not depicted) and data (not depicted) may be stored.

As is well-known in the art, only the kernel50and applications running within the kernel50have direct access to the hardware of the DSS computing system32. Any application running within userspace48may access the hardware only by means of a system call to the kernel50. Although functions may execute faster if implemented within the kernel50(e.g., as hardware drivers, such as storage hardware drivers66and helper driver64), the more complex the kernel50becomes, the more likely the kernel50is to crash, which can require the entire DSS32(or SP) to reboot, which can cause significant downtime. Thus, it is desirable to implement complex functionality (such as a complex storage driver stack) within userspace48to avoid crashing the kernel50. It is further desirable to implement the storage driver stack entirely within its own userspace container52so that in the event of a crash, only the userspace container52need be restarted, while other applications running within userspace48may continue operating, which allows for even further decreased downtime in the event of a crash. Use of a userspace container52also allows for easy portability between SP. Implementation within userspace48also allows for easy upgrading of the kernel50without having to recompile and test new OS upgrades prior to upgrading.

As depicted, a storage stack for storage operations on the DSS computing device32is implemented using userspace drivers54,56,58,60, and62. Upper filesystem driver54is configured to receive file-based storage requests74from hosts, each file-based storage request74being directed at a filesystem (not depicted) that is ultimately backed by storage from one or more of the storage devices44of the DSS computing device32. Upper filesystem driver54translates those requests74into block-based storage requests76directed to particular blocks of storage of a volume or logical disk (not depicted) on which the filesystem rests. Upper filesystem driver54sends those block-based storage requests76to mapping driver56.

In some embodiments, mapping driver56implements the logical volume using a container file (not depicted) of a lower-deck filesystem (not depicted). Thus, in these embodiments, mapping driver56translates block-based requests76into file-based requests78aimed at the container file. Mapping driver56further implements the lower-deck filesystem on a second virtual volume (not depicted) made up of storage extents (not depicted) drawn from the one or more storage devices44. In some embodiments, mapping driver56further introduces address translations due to de-duplication, RAID, and other services. Thus, mapping driver56translates file-based requests78back into block-based requests80to particular storage drives44, which it sends down the storage stack to multi-core cache58.

Multi-core cache58is a layer of the driver stack that makes use of a dedicated portion (not depicted) of memory40(which may include some persistent or battery-backed memory, not depicted) to store data relating to the block-based storage requests for fast performance optimized for execution by several cores38operating in parallel. Typically, once a block-based storage request80is placed within multi-core cache58(and either placed in persistent memory or mirrored to a backup copy on another SP), it may be acknowledged up the stack, allowing the host to continue as if the original storage request74had been fully-executed, even though the data has not yet been flushed to the ultimate backing store on storage devices44. In the background, the cores38work to flush this cached data to the storage devices44.

Each core38is configured to perform this flushing by sending one or more low-level data storage commands82that are each aimed at a particular address range of a particular storage drive44to the next layer of the storage stack, which is scheduling driver60. For any given storage drive44A, scheduling driver60, which may also be referred to as a Physical Package driver, schedules the execution of commands82directed at that storage drive44A, which is important when there are many simultaneous commands82directed to the same storage drive82coming from several different cores38.

Each core38sends its respective storage commands82(depicted as storage commands82(1),82(2),82(3), corresponding to cores38(1),38(2),38(3), respectively) to a particular per-core queue68(depicted as per-core userspace queues68A(1),68A(2),68A(3), . . . ). Each per-core queue68is dedicated to storage commands82from a particular core38(x) aimed at a particular storage device44Y. For each particular storage device44Y, there may be up to as many per-core queues68as there are cores38within processing circuitry36. In some embodiments, it is possible that certain cores38of processing circuitry36may be configured not to send storage commands82to flush the multi-core cache58(e.g., certain cores38may be dedicated to other tasks). In these embodiments, the number of per-core queues68for each particular storage device44Y may instead be limited to the number of cores38that are available for sending storage commands82. Thus, for example, if processing circuitry36includes four cores38, but only three of those cores38(1),38(2),38(3) are available for sending storage commands82, then there would be three per-core userspace queues68A(1),68A(2),68A(3) for storage device44A.

Upon receiving each storage command82, scheduling driver60places it into the appropriate per-core userspace queue68(at the head of each queue). In some embodiments, scheduling driver60may perform various re-ordering and/or merging operations on the various storage commands82within each per-core userspace queue68. In some embodiments, scheduling driver60may perform load-balancing operations by shifting various storage commands82between the per-core userspace queues68.

Scheduling driver60dequeues storage commands82from the tail of each per-core userspace queue68A for a particular storage device44A to an appropriate userspace dispatch queue70of a set of such dispatch queues70A (depicted as dispatch queues70A-i,70A-ii, . . . ) associated with that particular storage device44A. If there are the same number of per-core userspace queue68A for a particular storage device44A as there are dispatch queues70A for that particular storage device44A, then each per-core userspace queue68A dequeues directly to the head of a dedicated dispatch queue70A. Thus, for example, if there were only two per-core userspace queues68A(1),68A(2), then per-core userspace queue68A(1) would dequeue from its tail to the head of dispatch queue70A-i, and per-core userspace queue68A(2) would dequeue from its tail to the head of dispatch queue70A-ii. As depicted, since there are three per-core userspace queue68A(1),68A(2),68A(3) but only two dispatch queues70A-i,70A-ii, per-core userspace queue68A(1) dequeues from its tail by sending a storage command84to the head of dispatch queue70A-i, while per-core userspace queues68A(2),68A(3) alternate dequeuing from their respective tails by respectively sending storage commands86a,86bto the head of dispatch queue70A-ii.

Scheduling driver60typically refrains from performing re-ordering, merging, and load-balancing operations on the dispatch queues70.

Scheduling driver60dequeues storage commands88from the tail of each dispatch queue70A for a particular storage device44A aimed at a respective channel46A of that particular storage device44A. Thus, for example, as depicted, scheduling driver60dequeues storage commands88-ifrom the tail of dispatch queue70A-i aimed at channel46A-i of storage drive44A and storage commands88-iifrom the tail of dispatch queue70A-ii aimed at channel46A-ii of storage drive44A. It should be understood that, in some embodiments, instead of dequeuing aimed at particular channels46, scheduling driver60may simply dequeue in a round-robin manner from the dispatch queues70A for particular storage drive44A, relying on a built-in queue (not depicted) of that particular storage drive44A to execute the operations in parallel on its various channels46A.

It should be understood that scheduling driver60does not dequeue the storage commands88directly to the channels46A (or directly to the storage device44A), since a userspace driver such as scheduling driver60cannot communicate directly with the hardware. In addition, there are additional management tasks such as link initialization, link services, PHY management, and I/O support that are performed by intervening management driver62. Since management driver62also runs within userspace48, it is able to communicate with the storage hardware driver66running in the kernel50through a kernel helper driver64. See, for example, U.S. Pat. No., 9,612,756 issued on Apr. 4, 2017, the entire contents and teachings of which are incorporated by reference herein by this reference. Thus, management driver62forwards storage commands88-i,88-iias respective storage commands90-i,90-iito storage hardware driver66A for storage drive44A. To the extent that certain communications between management driver62within userspace48and storage hardware driver66A within the kernel50are prohibited (or not possible), helper driver64is used to forward such communications across the barrier between userspace48and the kernel50. Eventually, storage hardware driver66A forwards storage commands90-i,90-iias respective storage commands92-i,92-iito storage drive44A or its respective channels46A-i,46A-ii.

In some embodiments, memory40may also include a persistent storage portion (not depicted). Persistent storage portion of memory40may be made up of one or more persistent storage devices, such as, for example, disk drives, solid state drives, and the like. Persistent storage portion of memory40or persistent storage drives44is configured to store programs and data even while the DSS computing device32is powered off. The OS, applications, and drivers54,56,58,60,62,64,66are typically stored in this persistent storage portion of memory40or on persistent storage drives44so that they may be loaded into a system portion of memory40from this persistent storage portion of memory40or persistent storage drives44upon a system restart. These applications and drivers54,56,58,60,62,64,66, when stored in non-transient form either in the volatile portion of memory40or on persistent storage drives44or in persistent portion of memory40, form a computer program product. The processing circuitry36running one or more of these applications and drivers54,56,58,60,62,64,66thus forms a specialized circuit constructed and arranged to carry out the various processes described herein.

FIG. 2illustrates an example method100performed by the various drivers54,56,58,60,62,64,66of the storage stack and/or the kernel50. It should be understood that any time a piece of software (e.g., drivers54,56,58,60,62,64,66, kernel50, etc.) is described as performing a method, process, step, or function, in actuality what is meant is that a computing device (e.g., DSS computing device32) on which that piece of software is running performs the method, process, step, or function when executing that piece of software on its processing circuitry36. It should be understood that one or more of the steps or sub-steps of method100may be omitted in some embodiments. Similarly, in some embodiments, one or more steps or sub-steps may be combined together or performed in a different order. Method100is performed by DSS computing device32.

Steps110and120(and, in embodiments in which step130is performed, also step130) may be performed in parallel. Being performed in parallel means that the order of execution of these steps110,120(and130) is unimportant; they may be performed simultaneously, in an overlapping manner, or any of them may be performed prior to or subsequent to the other(s).

In step110, scheduling driver60operating within userspace48(and, in some embodiments, more particularly within a dedicated userspace container52) sends storage requests (e.g., storage commands82(1)) that were initiated by first core38(1) (towards particular storage drive44A) to a first userspace per-core queue68A(1), the first userspace per-core queue68A(1) being dedicated to storage requests82(1) that came from that first core38(1) directed to particular storage drive44A.

In step120, scheduling driver60operating within userspace48(and, in some embodiments, more particularly within a dedicated userspace container52) sends storage requests (e.g., storage commands82(2)) that were initiated by second core38(2) (towards particular storage drive44A) to a second userspace per-core queue68A(2), the second userspace per-core queue68A(2) being dedicated to storage requests82(2) that came from that second core38(2) directed to particular storage drive44A.

In optional step130(which may be omitted in systems that only have two cores38(1),38(2) or which only have two cores38(1),38(2) permitted to process flushes from the multi-core cache58), scheduling driver60operating within userspace48(and, in some embodiments, more particularly within a dedicated userspace container52) sends storage requests (e.g., storage commands82(3)) that were initiated by third core38(3) (towards particular storage drive44A) to a third userspace per-core queue68A(3), the third userspace per-core queue68A(3) being dedicated to storage requests82(3) that came from that third core38(3) directed to particular storage drive44A.

In some embodiments, scheduling driver60may perform step140, in which the contents of the various per-core queues68may be modified for efficiency reasons. Typically, step140is omitted for any per core queues68A that are associated with storage drives44A that are SSDs or are flash-based (or otherwise have minimal latency for random seeks). Step140may include one or more of sub-steps142,144,146.

In sub-step142, scheduling driver60re-orders storage commands82(x) stored within a per-core queue68A(x). For example, if there are two different storage commands82(x)-I and82(x)-II that are directed to extents of storage drive44A that are in close physical proximity to each other, scheduling driver60may re-order the queue68A(x) so that those two storage commands82(x)-I,82(x)-II are performed consecutively instead of having another storage commands82(x)-III directed to a distant extent intervening.

In sub-step144, scheduling driver60merges storage commands82(x) stored within a per-core queue68A(x). For example, if there are two different storage commands82(x)-I and82(x)-II that are both write commands directed at consecutive extents of storage drive44A, scheduling driver60may merge these two storage commands82(x)-I,82(x)-II into a single storage command82(x)-IV that writes to a merged larger extent.

In sub-step146, scheduling driver60load-balances between the per-core queues68A. For example, if per-core queue68A(1) has 1,000 pending storage requests82(1) therein and per-core queue68A(2) has only 17 pending storage requests82(2) therein, scheduling driver60may transfer some of the pending storage requests82(1) from the per-core queue68A(1) to per-core queue68A(2).

In step150, scheduling driver60operating within userspace48(and, in some embodiments, more particularly within a dedicated userspace container52) sends storage requests (e.g., storage commands84,86a,86b) from the userspace per-core queues68A to the set of userspace dispatch queues70A for storage drive44A.

In one embodiment, if there is only a single userspace dispatch queue70A-i for storage drive44A, then, in sub-step152, scheduling driver60dequeues the storage commands84,86a,86bfrom all of the userspace per-core queues68A to the single userspace dispatch queue70A-i for storage drive44A.

Alternatively, if there are at least two userspace dispatch queues70A-i,70A-ii, then sub-steps156and157(and possibly also158) are performed.

In sub-step156, scheduling driver60dequeues the storage commands84from at least one userspace per-core queue68A(1) to userspace dispatch queue70A-i for storage drive44A, while, in sub-step157, scheduling driver60dequeues the storage commands86afrom a different userspace per-core queue68A(2) to a different userspace dispatch queue70A-ii for storage drive44A. If there are more userspace per-core queues68A than userspace dispatch queue70A for storage drive44A, then, in sub-step158, scheduling driver60dequeues the storage commands86bfrom a third userspace per-core queue68A(3) to the same userspace dispatch queue70A-ii as in sub-step157.

In step160, scheduling driver60operating within userspace48(and, in some embodiments, more particularly within a dedicated userspace container52) sends storage requests (e.g., storage commands88) from the set of userspace dispatch queues70A to another driver operating within userspace48, namely management driver62. In some embodiments, the storage commands88are sent from particular dispatch queues70A to corresponding dispatch queues (not depicted) within management driver. In other embodiments, in optional sub-step165, scheduling driver60dispatches the storage commands88from the various userspace dispatch queues70A to management driver62(having only a single queue therein, not depicted) in an alternating manner according to a fairness policy.

In step170, the other driver operating within userspace48, namely management driver62, sends, by way of kernel helper driver64, the storage requests88received from scheduling driver60as storage requests90to storage hardware driver66A within the kernel50. The storage hardware driver66A is then able to pass these storage requests90on to the storage drive44A as storage requests92for execution by the storage drive44A. In some embodiments, storage hardware driver66A passes particular storage requests92-ito first channel46A-i and other storage requests92-iito second channel46A-ii. In other embodiments, a local queue (not depicted) within storage drive44A distributes the storage requests92between the various channels46A for execution as they become available.

Thus, techniques have been presented for increasing performance in multi-core data storage systems32while allowing for portability and fast failover in the event of a failure of the driver stack. This may be accomplished by a data storage system32employing several queues68,70to reduce lock contention. The queueing is performed with two levels of several queues68,70each within a userspace scheduling driver60within a userspace container52. The userspace scheduling driver60may dequeue into a userspace management driver62that communicates with a kernel-based hardware driver66by way of a kernel helper driver64.

As used throughout this document, the words “comprising,” “including,” “containing,” and “having” are intended to set forth certain items, steps, elements, or aspects of something in an open-ended fashion. Also, as used herein and unless a specific statement is made to the contrary, the word “set” means one or more of something. This is the case regardless of whether the phrase “set of” is followed by a singular or plural object and regardless of whether it is conjugated with a singular or plural verb. Further, although ordinal expressions, such as “first,” “second,” “third,” and so on, may be used as adjectives herein, such ordinal expressions are used for identification purposes and, unless specifically indicated, are not intended to imply any ordering or sequence. Thus, for example, a “second” event may take place before or after a “first event,” or even if no first event ever occurs. In addition, an identification herein of a particular element, feature, or act as being a “first” such element, feature, or act should not be construed as requiring that there must also be a “second” or other such element, feature or act. Rather, the “first” item may be the only one. Although certain embodiments are disclosed herein, it is understood that these are provided by way of example only and that the invention is not limited to these particular embodiments.

While various embodiments have been particularly shown and described, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the appended claims.

For example, although various embodiments have been described as being methods, software embodying these methods is also included. Thus, one embodiment includes a tangible non-transitory computer-readable storage medium (such as, for example, a hard disk, a floppy disk, an optical disk, flash memory, etc.) programmed with instructions, which, when performed by a computer or a set of computers, cause one or more of the methods described in various embodiments to be performed. Another embodiment includes a computer that is programmed to perform one or more of the methods described in various embodiments.

Furthermore, it should be understood that all embodiments which have been described may be combined in all possible combinations with each other, except to the extent that such combinations have been explicitly excluded.