User space and kernel space access to memory devices through private queues

A storage controller coupled to a storage array includes a device driver running in a kernel space that receives an administrative command from an application running in a user space of the storage controller and writes the administrative command to a first submission queue of a plurality of submission queues associated with a storage device in the storage array, where the first submission queue is reserved for use by the device driver. An input/output (I/O) command received from the application running in the user space, however, is written directly to a second submission queue of the plurality of submission queues without being routed through the kernel space, where the second submission queue being reserved for direct access by the application running in the user space.

BACKGROUND

As computer memory storage and data bandwidth increase, so does the amount and complexity of data that businesses manage daily. Large-scale distributed storage systems, such as data centers, typically run many business operations. A datacenter, which also may be referred to as a server room, is a centralized repository, either physical or virtual, for the storage, management, and dissemination of data pertaining to one or more businesses. A distributed storage system may be coupled to client computers interconnected by one or more networks. If any portion of the distributed storage system has poor performance, company operations may be impaired. A distributed storage system therefore maintains high standards for data availability and high-performance functionality.

DETAILED DESCRIPTION

A host computer system, such as storage controller, may be designed to interact with a storage device, which may be part of a larger storage array, to execute storage commands and manage operation of the storage device. The host computer system may include one or more queues used for processing commands (e.g., I/O related commands), implemented in system memory. Each queue may refer to a pair of corresponding submission and completion queues. The submission queues store storage commands to be fetched by the storage device, while the completion queues store results provided by the storage device indicating execution of corresponding storage commands. The host computer system may also include a kernel space device driver, executed by a processor. In certain systems, the kernel space device driver places commands onto the submission queues and reads results from the completion queues. In one embodiment, the storage commands may be generated by an application running in a user space on the host computer system, while the device driver runs in a kernel space on the host computer system. Throughout the description that follows, the terms hardware queue(s), I/O queue(s) admin queue(s), submission queue(s), completion queue(s), and queue pair(s) may be used when describing the above referenced data structure(s). Depending on the particular implementation, any of the above referenced queues may be used to store different types of commands or acknowledgments, such as administrative commands and/or I/O related commands. In addition, although reference may be made to a single queue (e.g., a submission queue), depending on the embodiment, that single queue may have a corresponding queue (e.g., a completion queue) which together make up queue pair. In other embodiments, multiple submission queues may be associated with the same completion queue.

Modern computing systems generally segregate virtual memory into a kernel space and one or more user spaces. This separation is generally done in order to provide memory protection and hardware protection from malicious or errant software behavior. In general, the kernel space is reserved for running a privileged operating system kernel, kernel extensions, and device drivers, such as kernel space device drivers, while the user spaces are where application software executes. Since the kernel space device driver that manages the submission queues and completion queues runs in the kernel space, storage commands from an application running in user space are generally passed from user space to kernel space, where they traverse a kernel stack before being placed onto the submission queue and retrieved by the storage device. The kernel stack may include the kernel space device driver, and possibly other modules, such as a block layer. Traversing this kernel stack can be a time consuming processes that increases a latency associated with execution of the storage commands.

In one embodiment, the system described herein introduces concurrent user space and kernel space access from the host computer system to storage devices through private, reserved I/O queues. In one embodiment, the kernel space device driver on the host machine reserves certain I/O queues for direct access by an application running in user space, while still maintaining other I/O queues for use by the kernel space device driver itself. In this manner, the application running in user space can place certain types of storage commands directly onto the reserved I/O queues, without routing those commands through the kernel space. The application may bypass the kernel space for certain types of commands including latency sensitive input/output (I/O) commands such as read operations, write operations, etc. For other types of commands, however, such as administrative commands or other latency tolerant commands, the application may utilize the kernel space device driver running in kernel space to place the commands in corresponding I/O queues. Accordingly, the application can reduce the latency associated with processing certain types of commands by writing these commands directly to a reserved I/O queue. In one embodiment, the storage device can fetch different commands from different I/O queues concurrently, including I/O queues reserved for use by both the user space and the kernel space.

FIG. 1is a block diagram illustrating a storage system100in which embodiments of the present disclosure may be implemented. Storage system100may include storage controllers110,150and storage array130, which is representative of any number of data storage arrays or storage device groups. As shown, storage array130includes storage devices135A-n, which are representative of any number and type of storage devices (e.g., solid-state drives (SSDs)). Storage controllers110and150may be coupled directly to initiator device125and storage controllers110and150may be coupled remotely over network120to initiator device115. Initiator devices115and125are representative of any number of clients which may utilize storage controllers110and150for storing and accessing data in storage system100. It is noted that some systems may include only a single client or initiator device, connected directly or remotely, to storage controllers110and150.

In one embodiment, controller110is designated as the “primary” controller, which performs most or all of the I/O operations on the array130. If, however, a software crash, hardware fault or other error occurs, the “secondary” controller150may be promoted to serve as the primary controller and take over all responsibilities for servicing the array130. In one embodiment, storage controllers110and150are identical and any description of controller110herein may be equally attributed to storage controller150.

Storage controller110may include software and/or hardware configured to provide access to storage devices135A-n. In one embodiment, storage controller110is connected to storage device135A-n over a data bus, such as a Peripheral Component Interconnect express (PCIe) bus that operates within the framework of the Non-Volatile Memory express (NVMe) communication interface. In general, the NVMe communication interface makes use of submission and completion queues located on the host side for submitting storage commands and receiving confirmation of the execution of those commands on the storage devices. One of storage devices135A-n may fetch commands from the submission queue, execute the fetched command, and return results to the completion queue. In another embodiment, the submission and completion queues are located on the storage devices. NVMe, as a logical device interface, capitalizes on the low latency and internal parallelism of flash-based storage devices, mirroring the parallelism of contemporary CPUs, platforms and applications. NVMe allows levels of parallelism possible in modern solid state drives (SSDs) to be fully exploited by the host hardware and software. As a result, NVMe reduces I/O overhead and provides various performance improvements in comparison to previous logical device interfaces, including multiple, long command queues, and reduced latency.

Although storage controller110is shown as being separate from storage array130, in some embodiments, storage controller110may be located within storage array130. Storage controller110may include or be coupled to a base operating system (OS), a volume manager, and additional control logic for implementing the various techniques disclosed herein. In one embodiment, the OS is designed with flash storage in mind, so while it can use conventional SSDs to store data, it does not depend on a 512 byte random overwrite capability. Even on conventional SSDs, storage controller110can achieve better performance by writing and discarding data in large chunks. This style of I/O is sometimes called “flash friendly I/O.” This also makes it a much easier task to convert the OS to use the physical addressing of storage devices, as compared to conventional filesystems.

In one embodiment, the logic of storage controller110includes one or more user space applications142and one or more kernel space device drivers144. In one embodiment, there may be a kernel space device driver144on storage controller110corresponding to each of storage devices135A-n in storage array130. As new storage devices are connected to controller110, new device drivers may be installed. These drivers may be similarly discarded when a corresponding device is disconnected from storage controller110. Applications wishing to communicate with one of storage devices135A-n, such as user space application142, may do so via device driver144. In one embodiment, multiple applications can access the same kernel space device driver144concurrently. In one embodiment, storage controller150includes a separate instance(s) of kernel space device driver154and one or more other user space applications152. In one embodiment, storage controller110includes a number of I/O queues for storing commands to be fetched by one of storage devices135A-n. Kernel space device driver144may reserve certain queues for direct access by user space application142in order to reduce the latency associated with processing certain types of commands (e.g., I/O commands). Additional details regarding these I/O queues are provided below with respect toFIG. 2.

Storage controller110may include and/or execute on any number of processing devices and may include and/or execute on a single host computing device or be spread across multiple host computing devices, depending on the embodiment. In some embodiments, storage controller110may generally include or execute on one or more file servers and/or block servers. Storage controller110may use any of various techniques for replicating data across devices135A-n to prevent loss of data due to the failure of a device or the failure of storage locations within a device. Storage controller110may also utilize any of various deduplication techniques for reducing the amount of data stored in devices135A-n by deduplicating common data.

In one embodiment, storage controller110may utilize logical volumes and mediums to track client data that is stored in storage array130. A medium is defined as a logical grouping of data, and each medium has an identifier with which to identify the logical grouping of data. A volume is a single accessible storage area with a single file system, typically, though not necessarily, resident on a single partition of a storage device. The volumes may be logical organizations of data physically located on one or more of storage device135A-n in storage array130. Storage controller110may maintain a volume to medium mapping table to map each volume to a single medium, and this medium is referred to as the volume's anchor medium. A given request received by storage controller110may indicate at least a volume and block address or file name, and storage controller110may determine an anchor medium targeted by the given request from the volume to medium mapping table.

In various embodiments, multiple mapping tables may be maintained by storage controller110. These mapping tables may include a medium mapping table and a volume to medium mapping table. These tables may be utilized to record and maintain the mappings between mediums and underlying mediums and the mappings between volumes and mediums. Storage controller110may also include an address translation table with a plurality of entries, wherein each entry holds a virtual-to-physical mapping for a corresponding data component. This mapping table may be used to map logical read/write requests from each of the initiator devices115and125to physical locations in storage devices135A-n. A “physical” pointer value may be read from the mappings associated with a given medium during a lookup operation corresponding to a received read/write request. The term “mappings” is defined as the one or more entries of the address translation mapping table which convert a given medium ID and block number into a physical pointer value. This physical pointer value may then be used to locate a physical location within the storage devices135A-n. The physical pointer value may be used to access another mapping table within a given storage device of the storage devices135A-n. Consequently, one or more levels of indirection may exist between the physical pointer value and a target storage location.

In alternative embodiments, the number and type of client computers, initiator devices, storage controllers, networks, storage arrays, and data storage devices is not limited to those shown inFIG. 1. At various times one or more clients may operate offline. In addition, during operation, individual client computer connection types may change as users connect, disconnect, and reconnect to storage system100. Further, the systems and methods described herein may be applied to directly attached storage systems or network attached storage systems and may include a host operating system configured to perform one or more aspects of the described methods. Numerous such alternatives are possible and are contemplated.

Network120may utilize a variety of techniques including wireless connections, direct local area network (LAN) connections, wide area network (WAN) connections such as the Internet, a router, storage area network, Ethernet, and others. Network120may comprise one or more LANs that may also be wireless. Network120may further include remote direct memory access (RDMA) hardware and/or software, transmission control protocol/internet protocol (TCP/IP) hardware and/or software, router, repeaters, switches, grids, and/or others. Protocols such as Fibre Channel, Fibre Channel over Ethernet (FCoE), iSCSI, and so forth may be used in network120. The network120may interface with a set of communications protocols used for the Internet such as the Transmission Control Protocol (TCP) and the Internet Protocol (IP), or TCP/IP. In one embodiment, network120represents a storage area network (SAN) which provides access to consolidated, block level data storage. The SAN may be used to enhance the storage devices accessible to initiator devices so that the devices135A-n appear to the initiator devices115and125as locally attached storage.

Initiator devices115and125are representative of any number of stationary or mobile computers such as desktop personal computers (PCs), servers, server farms, workstations, laptops, handheld computers, servers, personal digital assistants (PDAs), smart phones, and so forth. Generally speaking, initiator devices115and125include one or more processing devices, each comprising one or more processor cores. Each processor core includes circuitry for executing instructions according to a predefined general-purpose instruction set. For example, the x86 instruction set architecture may be selected. Alternatively, the ARM®, Alpha®, PowerPC®, SPARC®, or any other general-purpose instruction set architecture may be selected. The processor cores may access cache memory subsystems for data and computer program instructions. The cache subsystems may be coupled to a memory hierarchy comprising random access memory (RAM) and a storage device.

In one embodiment, initiator device115includes initiator application112and initiator device125includes initiator application122. Initiator applications112and122may be any computer application programs designed to utilize the data on devices135A-n in storage array130to implement or provide various functionalities. Initiator applications112and122may issue requests to read or write data from certain logical volumes data within storage system100. Initiator applications112and122may communicate with the kernel on its corresponding device to send data over a network interface. This data travels over network120, hitting a network interface on storage controller110, which utilizes the kernel to get the data to user space application142. Application142can utilize kernel space device driver144of storage controller110to service the read or write requests, as described in detail herein.

FIG. 2is a block diagram illustrating user space and kernel space access to memory devices through private queues, according to an embodiment. In one embodiment, the virtual memory of storage controller110is divided into a user space212and a kernel space214. Kernel space214is reserved for running a privileged operating system kernel, kernel extensions, such as block layer246, and device drivers, such as kernel space device driver144. User space212is reserved for running application software, such as user space application142. This separation between user space212and kernel space214provides memory protection and hardware protection from malicious or errant software behavior. This arrangement of modules may be a logical separation, and in other embodiments, these modules, interfaces or other components can be combined together or separated in further components. In one embodiment, storage device135A is connected to storage controller110over PCIe bus240. In one embodiment, storage device135A may be external to storage controller110as part of storage array130. In another embodiment, storage controller110and storage device135A may be part of a single device or appliance. In other embodiments, storage controller110may include different and/or additional components which are not shown to simplify the description. Storage device135A may include one or more mass storage devices which can include, for example, flash memory or solid-state drives (SSDs). In one embodiment, storage device135A also includes memory such as random-access memory (RAM); dynamic random-access memory (DRAM); erasable programmable memory (e.g., EPROM and EEPROM); flash memory; or any other type of storage medium.

In one embodiment, user space application142may generate or issue storage commands to be executed on storage device135A. These commands may include I/O commands, such as read operations, write operations, etc., or administrative commands, such as firmware updates, debug operations, asynchronous event notifications, etc. In one embodiment, storage controller110includes a number of queues220, into which the storage commands may be placed and eventually fetched by storage device135A for execution. In general, kernel space device driver144is responsible for management and allocation of queues220and for placing the storage commands into the queues220.

For certain commands, user space application142can pass the commands to kernel space214for processing. Depending on the type of command, user space application142may make a system call232to a block layer246in kernel space214. The block layer can translate a file referenced by the system call to storage blocks on storage device135A and provide an indication of the corresponding storage blocks to kernel space device driver144. In another embodiment, user space application142may pass the command directly to kernel space device driver144. Upon receipt, kernel space device driver may send (235,236) the command to a queue224or226that has been reserved for use by the kernel space device driver144. In one embodiment, queue224is an admin queue, while queue226is an I/O queue. In one embodiment, it is at the discretion of user space application142which types of commands are routed through the kernel space214, although kernel space device driver144may be configured to handle both administrative commands and I/O commands.

Since kernel space device driver144runs in kernel space214, storage commands from user space application142that are passed to kernel space device driver before being placed onto one of the queues220may experience some additional latency associated with traversing the kernel stack (i.e., block layer246, kernel space device driver144and any other components in kernel space214). Accordingly, in one embodiment, user space application142is configured to place certain commands directly into one of queues220without routing the commands through kernel space214. For example, user space application142may place (231) certain latency-sensitive I/O commands directly into I/O queue222. This prevents any delay associated with routing the commands through kernel space214. In one embodiment, storage controller110includes memory mapped registers called doorbell registers. When the user space application142writes a command to the memory of a particular queue, it also updates the doorbell register to point to this newly written memory location (e.g., an index into the queue for NVMe). Since the queue memory is previously registered with the storage device on queue creation for the doorbell register, the device knows where to look for the new command.

In one embodiment, to configure I/O queue222for direct access from user space212, user space application may send a system call or command233to kernel space device driver144for reservation of the I/O queue. Kernel space device driver144may determine whether or not an I/O queue has already been reserved for user space application142, and if not, may allocate I/O queue222. In one embodiment, system call233requests that the kernel map some of the user space memory so writes to it get sent by direct memory access (DMA) or memory mapped I/O, to the device. In one embodiment, the request sent to the kernel is an I/O control system call (e.g., ioctl) to create the queue and the corresponding memory mapping. The queue creation turns into a command sent to the drive, and the memory mapping is handled by an I/O memory management unit (IOMMU). Once allocated, kernel space device driver144may return a confirmation that the queue has been reserved for user space application142, and user space application142is free to directly place storage commands onto I/O queue222.

In one embodiment, storage device135A may retrieve the stored commands from queues220for execution. In one embodiment, storage device135A uses a polling approach where it periodically queries queues220to determine whether any pending storage commands are present. In another embodiment, an interrupt approach is used, where storage controller110sends an interrupt to storage device135A to notify it that queues220include pending storage commands. In one embodiment, PCIe bus240operates within the framework of the NVMe communication interface. According to this framework, storage device135A may fetch commands from the queues220, execute the fetched commands, and return results to corresponding completion queues (not shown). In one embodiment, each of queues220includes a corresponding completion queue to form a set of NVMe queue pairs. In one embodiment, storage controller110may include a separate set of I/O queue pairs for each of storage devices135A-n in storage array130. In another embodiment, the queues are located on one of the storage device in storage array130. In this embodiment, user space application142uses a polling approach where it periodically queries the submission queues to determine whether there is room to send new storage commands or the completion queues to determine whether there are any received confirmations. In another embodiment, an interrupt approach is used, where the storage device sends an interrupt to user space application142to notify it that the submission queues have room or that the completion queues have confirmations.

FIG. 3is a flow diagram illustrating a method for concurrent user space and kernel space access to memory devices through private queues, according to an embodiment. The method300may be performed by processing logic that comprises hardware (e.g., circuitry, dedicated logic, programmable logic, microcode, etc.), software (e.g., instructions run on a processing device to perform hardware simulation), firmware, or a combination thereof. In one embodiment, method300may be performed by storage controller110, as shown inFIGS. 1 and 2.

Referring toFIG. 3, at block310, method300receives an administrative command from an application142in a user space212of the storage controller110. In one embodiment application142issues an administrative command, such as a firmware update, debug operation, or other command which is not comparatively sensitive to processing latency. At block320, method300routes the administrative command through kernel space device driver144in a kernel space214of the storage controller110. In one embodiment, application142makes a system call corresponding to the administrative command which is received by block layer246. Block layer246may translate a file referenced by the system call to corresponding storage blocks on storage device135A and provide an indication of the storage blocks to kernel space device driver144in kernel space214. In other embodiments, application142may pass the administrative command directly to kernel space device driver144. At block330, method300sends the administrative command to a first queue224of a plurality of queues220, where the first queue224is reserved for use by the kernel space device driver144. In one embodiment, kernel space device driver144places the administrative command onto queue224from where it may be retrieved by storage device135A.

At block340, method300receives a first I/O command from the application142in user space212of the storage controller110. In one embodiment application142issues the I/O command, such as a read command, a write command, or other command which is sensitive to processing latency. At block350, method300sends the first I/O command directly to a second I/O queue222of the plurality of queues220without routing the first I/O command through the kernel space214of the storage controller110. In one embodiment, user space application can place the I/O command onto I/O queue222, which has been reserved for direct access by user space application142.

At block360, method300receives a second I/O command from application142in user space212of the storage controller110. The second I/O command may be the same as or similar to the first I/O command, in that it includes a read command or a write command. In another embodiment, the second I/O command may be received from some other application besides user space application142. In one embodiment, however, the second I/O command may be passed through kernel space214rather than written directly onto an I/O queue220. At block370, method300routes the second I/O command through kernel space device driver144in kernel space214of the storage controller110. Kernel space device driver144is configured to process any type of command including either administrative commands or I/O commands. Thus, it is at the discretion of user space application142whether to place commands directly onto an I/O queue or to route the commands through kernel space214. At block380, method300sends the second I/O command to a third I/O queue226of a plurality of queues220, where the third I/O queue226is reserved for use by the kernel space device driver144. As a result, method300can place commands on queues220concurrently and can write commands from the first queue224, the second I/O queue222, and the third I/O queue226to storage device135A concurrently. In one embodiment, storage device135A can retrieve pending commands from any of queues220concurrently, as well. In this manner, both applications in user space212and drivers in kernel space214can have concurrent access to storage device135A.

FIG. 4is a flow diagram illustrating a method for allocation of I/O queue pairs for direct user space access, according to an embodiment. The method400may be performed by processing logic that comprises hardware (e.g., circuitry, dedicated logic, programmable logic, microcode, etc.), software (e.g., instructions run on a processing device to perform hardware simulation), firmware, or a combination thereof. In one embodiment, method400may be performed by kernel space device driver144, as shown inFIGS. 1 and 2.

Referring toFIG. 4, at block410, method400receives a memory allocation for an I/O queue pair. At lock420, method400receives a system call or command from application142running in user space212requesting allocation of an I/O queue pair, including a submission queue and completion queue, at the memory addresses allocated at block410. In one embodiment, to configure I/O queue222(and a corresponding completion queue) for direct access from user space212, user space application142may send a system call or command233to kernel space device driver144for reservation of the I/O queue pair. At block430, method400allocates the second I/O queue222(and the corresponding completion queue) for direct access by user space application142. Once the I/O queue pair is allocated, at block440, method400sends a confirmation of the reservation of the I/O queue pair to user space application142.

FIG. 5is a flow diagram illustrating a method for issuing commands to kernel space and directly to I/O queues, according to an embodiment. The method500may be performed by processing logic that comprises hardware (e.g., circuitry, dedicated logic, programmable logic, microcode, etc.), software (e.g., instructions run on a processing device to perform hardware simulation), firmware, or a combination thereof. In one embodiment, method500may be performed by user space application142, as shown inFIGS. 1 and 2.

Referring toFIG. 5, at block510, method500sends a first command to a kernel space device driver144in a kernel space214of the storage controller110, the kernel space device driver144to send the first command to a first queue224of a plurality of queues220associated with a storage device135A coupled to the storage controller110, the first queue224being reserved for use by the kernel space device driver144. At block520, method500sends a system call or command to kernel space device driver144for reservation of a second I/O queue222for direct access by user space application142. At block530, method500receives a confirmation of the reservation of I/O queue222. At block540, method500sends a second command directly to the second I/O queue222without routing the second command through the kernel space214of the storage controller110.

The exemplary computer system600includes a processing device602, a main memory604(e.g., read-only memory (ROM), flash memory, dynamic random access memory (DRAM), a static memory606(e.g., flash memory, static random access memory (SRAM), etc.), and a data storage device618, which communicate with each other via a bus630. Data storage device618may be one example of any of the storage devices135A-n inFIGS. 1 and 2. Any of the signals provided over various buses described herein may be time multiplexed with other signals and provided over one or more common buses. Additionally, the interconnection between circuit components or blocks may be shown as buses or as single signal lines. Each of the buses may alternatively be one or more single signal lines and each of the single signal lines may alternatively be buses.

The data storage device618may include a machine-readable storage medium628, on which is stored one or more set of instructions622(e.g., software) embodying any one or more of the methodologies of functions described herein, including instructions to cause the processing device602to execute user space application142, kernel space device driver144or initiator application112or122. The instructions622may also reside, completely or at least partially, within the main memory604and/or within the processing device602during execution thereof by the computer system600; the main memory604and the processing device602also constituting machine-readable storage media. The instructions622may further be transmitted or received over a network620via the network interface device608.

Embodiments of the claimed subject matter include, but are not limited to, various operations described herein. These operations may be performed by hardware components, software, firmware, or a combination thereof.