Data streaming for computational storage

Methods, apparatuses, and computer-readable media for streaming arbitrarily large amounts of data through computational storage programs of a computational storage device. A computational storage device comprises a storage media, a computational storage processor, and a controller. A firmware of the controller comprises a plurality of streaming drivers, each associated with a data source or data destination of the storage device. The firmware further comprises a buffer abstraction layer operable to read data from a data source through an associated ingress streaming driver of the plurality of streaming drivers to provide a source data stream for a computational storage program executing on the computational storage processor. The buffer abstraction layer is further operable to receive a destination data stream from the computational storage program and write data to a data destination through an associated egress streaming driver of the plurality of streaming drivers.

BRIEF SUMMARY

The present disclosure relates to technologies for streaming large amounts of data through computational storage programs. According to some embodiments, a storage device comprises a storage media, a computational storage processor, and a controller. A firmware of the controller comprises a plurality of streaming drivers, each associated with a data source or data destination of the storage device. The firmware further comprises a buffer abstraction layer operable to read data from a data source through an associated ingress streaming driver of the plurality of streaming drivers to provide a source data stream for a computational storage program executing on the computational storage processor. The buffer abstraction layer is further operable to receive a destination data stream from the computational storage program and write data to a data destination through an associated egress streaming driver of the plurality of streaming drivers.

According to further embodiments, a method for streaming large amounts of data through computational storage programs comprises steps of receiving a definition of a data source, a data destination, and a computational storage program associated with a computational storage operation in a storage device. An ingress streaming driver implemented in the storage device is provisioned for the data source, and an egress streaming driver implemented in the storage device is provisioned for the data destination. The computational storage program is executed in the storage device while reading input data from the ingress streaming driver and writing output data to the egress streaming driver through a buffer abstraction layer.

According to further embodiments, a non-transitory computer readable medium comprises processor-executable instructions that, when executed by a controller of a storage device, cause the controller to provision an ingress streaming driver for a data source associated with a computational storage operation and provision an egress streaming driver for a data destination associated with the computational storage operation. The controller then executes a computational storage program on a computational storage processor of the storage device while reading input data from the ingress streaming driver and writing output data to the egress streaming driver through a buffer abstraction layer.

These and other features and aspects of the various embodiments will become apparent upon reading the following Detailed Description and reviewing the accompanying drawings.

DETAILED DESCRIPTION

The following detailed description is directed to technologies for streaming large amounts of data through computational storage programs. In data-intensive computing applications, the movement of data between storage devices and host compute resources presents a performance bottleneck. Computational storage comprises technologies and storage architectures that shift data compute tasks to processing resources closer to the storage media, offloading processing from host CPUs and reducing the storage-to-CPU bottleneck. For example, a computational storage device (“CSD”) may comprise a conventional storage device, such as an HDD or an SSD, with dedicated processing hardware added to perform operations directly on data stored therein. Computational storage devices and architectures lend themselves to high volume, data-intensive operations such as compression/decompression, encryption, video transcoding, and other algorithmic operations for edge computing, machine learning, real-time data analytics, and HPC applications.

FIG.1shows the components of a typical storage device102. The storage device102includes non-volatile storage media104, such as NAND FLASH memory, NOR flash memory, other FLASH technology memories, phase-change memory (“PCM”), racetrack memory, magnetic media, optical media, and/or the like. The storage device further includes a controller106than manages data stored on the storage media104and processes data read and write requests from a host computing device, referred to herein generally as host108. The host108may include a server computer, storage-area controller, personal computer, laptop, game console, and the like. The storage device102may further include a non-transitory computer-readable memory embedded in or accessible by the controller106and containing a firmware (not shown) comprising processor-executable code allowing the controller to process storage requests from the host108and perform other necessary management of the storage media104.

The host108is connected to the storage device102through a host interface110. According to some embodiments, the host interface110may be compatible with the NVM Express (“NVMe”) interface for the high-bandwidth connection of the storage device102to the host108over a PCI Express (“PCIe”) bus. Alternatively or additionally, the host interface110may be compatible with one or more of a SATA interface standard, an IDE interface standard, a SCSI interface standard, a SAS interface standard, a USB interface standard, a CF interface standard, an MMC interface standard, an SD interface standard, and the like. In further embodiments, the storage device102may be connected to the host(s)108through the host interface110via a switch/fabric/intermediate controller (not shown) or some other intermediate device.

According to embodiments, an illustrative computational storage device120may include dedicated processing resources added to the storage device102, such as a computational storage processor (“CSP”)122and a connected computational program memory (“CPM”)124, as further shown inFIG.1. In some embodiments, the CSP122may comprise one or more processing resources, such as FPGAs, ASICs, and/or CPU cores with a limited instruction set to enable efficient algorithmic processing of raw data in the CSD120. The CPM124may comprise static random access memory (“SRAM”) or dynamic random access memory (“DRAM”) onboard the storage device102. In some embodiments, the computational storage device120may include a standardized interface and/or runtime environment, referred to herein as a CS engine130, for execution of programs or modules, referred to generally as CS programs132. The CS engine130may define an execution model and an interface for the CS programs132to perform computational storage operations in the CSD120. In some embodiments, the CS engine130may be implemented in the firmware of the CSD120.

For example, the enhanced Berkeley Packet Filter (“eBPF”) specification may be implemented by the CS engine130and used to write programs that can be applied to data in the storage device102. The eBPF specification defines an instruction set and a program structure for programs intended to run at a kernel layer in an OS and have direct access to data packets, e.g., network, storage, and the like, to process or pre-process data before passing it up to the application layer. Because of its design, the eBPF specification lends itself to being an excellent development environment for CS programs132. Accordingly, an engineer may be allowed to write a program on a host in C, Python, or any other applicable language, and cross-compile that program into an eBPF image. The eBPF image may then be uploaded to the CSD120for execution on the CSP122to perform the desired operation(s) on data contained within the device. This provides the advantage of data no longer needing to be read onto the host108for processing since the data can be processed directly in the CSD120.

One approach for implementing CS programs132in a CSD120may include designating the CPM124to serve as both the source and target for computational storage operations on the device. For example, the storage controller106may read data from the storage media104into the CPM124. CS programs132executing on the CSP122may access the data in the CPM124, processes it, and write the resulting processed data back to the CPM. The processed data may then be written by the controller106back to the storage media104, sent to the host108through the host interface110, and/or the like. It will be appreciated that, traditionally, storage devices102have minimal SRAM/DRAM available for use as CPM124as compared to host systems, which often have large memory capacities. This is primarily due to costs involved in implementing the additional SRAM/DRAM and supporting circuitry. Because the data is read into and processed in the limited CPM124, this approach creates a bottleneck in the size of data that can be processed by the CS programs132.

According to embodiments described herein, a data streaming environment may be implemented in a computational storage device that allows arbitrarily large amounts of data to be processed by computational storage programs without being restricted by the size of onboard computational program memory. In some embodiments, a data streaming environment200includes an abstraction layer, such as the buffer abstraction layer202shown inFIG.2, implemented between the CS program132and a data source204and data destination206defined by the desired operation. The buffer abstraction layer202interfaces with the data source204and data destination206using streaming drivers, such as the ingress streaming driver208and egress streaming driver210further shown inFIG.2, to stream data from and to a variety of sources and destinations.

By utilizing the data streaming environment200shown inFIG.2and described herein, only a portion of data is required to be stored in the CPM124at a time. This allows the CS program132to read data from the data source, e.g., the storage media104, as it is needed and write data to the data destination, e.g., the host108, as it is available without having to be concerned with (or containing code for) buffering data in the CPM124or being limited in processing by the amount of CPM available. The buffer abstraction layer202, ingress streaming driver208, and egress streaming driver210handle all of the reading of data from the data source204and writing of the resulting processed data to the data destination206as well as any data buffering and memory management of the CPM124. By implementing the data streaming environment200, the buffer abstraction layer202can create data pipelines to keep packet-oriented CS programs132, such as eBPF programs, fed with data without the program requiring knowledge of the underlying data transfers while minimizing use of the CPM124. The data streaming environment200may further abstract the CS programs132away from the data that they consume and produce.

FIG.3shows one example configuration of the data streaming environment200in an NVMe SSD supporting computational storage processing, according to some embodiments. The configuration includes a CS program132executing on the CSP122for processing data read from the storage media104of the drive before it is passed to the host108. For example, an eBPF program132may be present in the firmware of the CSD120for performing decryption and/or decompression of data as it is read from the FLASH media104. In another example, the host108may upload an eBPF program132to the CSD120for transcoding video files from the format in which they stored on the FLASH media104to a different format suitable for playback on or streaming from the host108.

According to some embodiments, a given computational storage operation may be associated with a data definition that specifies a data source204, a CS program132that operates on input data from the data source, and a data destination210for the output data produced by the program. The specification of the data source204may include both the source of the input data, e.g., media, memory, host, etc., as well as the locations or extent of input data to be processed from the source in the computational storage operation. The CSD120may include a set of ingress and egress streaming drivers208,210that can be attached to different data sources and destinations204,206in the drive. The streaming drivers208,210manage the physical input and output of data from their associated physical sources and destinations and expose read and write operations to the buffer abstraction layer202. In some embodiments, the streaming drivers208,210may expose memory operations for devices that may not be memory (such as media operations). The streaming drivers208,210may be implemented in the firmware of the CSD120, or they may be implemented in the hardware of the drive, with associated interfaces implemented within the buffer abstraction layer202.

The buffer abstraction layer202provides the data pipelines to the CS program132for the consumption and production of data, i.e., consuming of input data from the storage media104and production of output data to the host. The buffer abstraction layer202provisions the appropriate ingress and egress streaming drivers208,210for the data source204and data destination206required by the CS program132based on the data definition associated with the computational storage operation. In the current example, this may include an ingress streaming driver208for reading from the media interface of the storage media104(e.g., an LBA read interface302for the FLASH) and an egress streaming driver210for writing directly to host memory (e.g., a host DMA engine304for transfer of data over the PCIe bus306). In some embodiments, the provisioning of the ingress and egress streaming drivers208,210may be the result of API calls by the CS program132to the buffer abstraction layer.

The buffer abstraction layer202may then manage the reading of data in the ingress streaming driver208and the writing of data in the egress streaming driver210according to the consumption and production of data by the eBPF program132. In other words, when data is consumed by the eBPF program132, the buffer abstraction layer208notifies the ingress streaming driver208to read more data from the FLASH media104. Conversely, when data is produced by the eBPF program132, the buffer abstraction layer202notifies the egress streaming driver210to transfer the data to the host memory using DMA. According to embodiments, the eBPF program132is dissociated from the underlying device input and output operations and operates only with sequential data streams for both consumption and production.

According to some embodiments, to flow control the data streams, the buffer abstraction layer202may pause the CS program132if no data is available to read by the ingress streaming driver208from the data source204. Similarly, the buffer abstraction layer202may pause the CS program132if the circular buffer in the egress steaming driver210is full awaiting space for new writes to the data destination206. In further embodiments, the ingress and egress streaming drivers208,210may support circular buffers stored in the CPM124that are managed by the drivers based upon read/write operations. As data is read by the buffer abstraction layer202and provided to the eBPF program132, it can be freed in the ingress streaming driver208and any associated buffers in the CPM124. Similarly, as data is written to the data destination206by the egress streaming driver210, the driver frees any associated buffer memory. The buffer abstraction layer202thus provides the capability for allowing arbitrarily large memory operations to occur regardless of the size of the CPM124.

In some embodiments, the buffer abstraction layer202works with the ingress streaming driver208and the egress streaming driver210to initiate data transfers based upon watermarks set in the abstraction layer. For example, a watermark for the ingress streaming driver208could set that specifies that additional reads from the FLASH media (data source204) are to be performed if input data buffered in the CPM124falls below 16 KB. Similarly, because it is inefficient to transfer small amounts of data over DMA to the host108(data destination206), a watermark could be set for the egress streaming driver210that would only initiate DMA transfers if 4 KB of output data is available to transfer. Once all of the specified input data from the data source204has been read from the ingress streaming driver208and processed by the eBPF program132and any output data flushed to the egress streaming driver210, the program may be terminated.

The data may be unstructured data (such as arbitrary blocks from the storage media104) or structured data (such as text-based comma-separated-value data for consumption by an application on the host108). Further, by abstracting the CS program132from the data source204and data destination206, any source or destination may be implemented from/to which the program may consume or produce data. For example, the following table provides some examples of combinations of data source204and data destinations206for which ingress and egress streaming drivers208,210may be provided in the firmware/hardware of a CSD120, according to embodiments described herein:

DestinationSource DriverDriverDescriptionMediaHost MemoryProcess data from media and stream(DMA)results to host memory.MediaMediaProcess data from media and streamresults back to the media.Host MemoryMediaProcess data from host memory and(DMA)stream the results into media.CPMCPMProcess data in the CPM back to theCPM.MediaPeer DeviceStream data from a device to anotherMemoryPCIe device (CPM) for processing; aka(DMA)peer-to-peer streaming.Host MemoryHost MemoryProcess data in host memory and stream(DMA)(DMA)the result back to host memory.

FIG.4illustrates one routine400for streaming data to and from a computational storage program132in a computational storage device120utilizing the data streaming environment200described herein. According to some embodiments, the routine400may be performed by the buffer abstraction layer202and/or other modules in the firmware of the CSD120, executing in a controller106and/or computational storage processor122of the device. In other embodiments, the routine400may be performed by firmware and hardware of a storage controller controlling an array of storage devices.

The routine400includes step402, where a data definition is received associated with a computational storage operation to be performed in the CSD120. For example, the host108may request execution of a computational storage program from the controller106of the CSD120. The host108may specify a slot number for a CS program132, a descriptor indicating the data source204, such as such as FLASH media and a list of LBA ranges, watermarks or buffer thresholds for reading input data from the data source, a descriptor indicating the data destination206, such as host memory address(es) and lengths/ranges (e.g., a scatter/gather list for DMA transfers), watermarks or buffer thresholds for writing output data to the data destination, and the like. The host108may further upload the CS program132, such as eBPF bytecode, for loading in the specified slot. In other embodiments, the data definition may be predefined for certain computational storage operations in the CSD120and stored in the firmware. For example, in a CS-enabled SSD may include a data definition and CS program120in the firmware for encryption and/or decryption of data written to and read from the storage media104by the host108through the controller106.

Next, the routine400proceeds from step402to step404, where an ingress streaming driver is provisioned for the data source204based on the data definition received in step402. For example, the controller106may load a streaming driver from the firmware for reading data from the FLASH media104of the SSD through the LBA read interface302, i.e. by accessing data in the FLASH by logical block address. Similarly, at step406, an egress streaming driver is provisioned for the data destination206based on the data definition. For example, the controller106may load a streaming driver for writing to host memory through a host DMA engine304, i.e., by direct memory access over the PCIe bus306. In addition, the controller106may initialize the buffer abstraction layer202and/or the circular buffers for buffering of input and output data for the data streams in the CPM124, as well as establish the watermarks for the reading and writing of data to and from the ingress and egress streaming drivers208,210based on the corresponding descriptors in the received data definition.

From step406, the routine400proceeds to step408, where the CS program132specified by the data definition is executed on the CSP122while streaming input data from and output data to the buffer abstraction layer202. The buffer abstraction layer202in turn reads input data from the ingress streaming driver208and writes data to the egress streaming driver210to manage the data streams. For example, an eBPF program132may be loaded into the CS engine130and executed to consume data from the FLASH media104(data source204) and produce data to the host108(data destination206) through the buffer abstraction layer202. According to embodiments, the buffer abstraction layer202manages data flow in the data streams from the ingress streaming driver208and to the egress streaming driver210during execution of the eBPF program132. As data is consumed by the eBPF program132, the buffer abstraction layer202notifies the ingress streaming driver208to read more data from the FLASH media104based on the specified watermarks/thresholds for the data source204. Conversely, when data is produced by the eBPF program132, the buffer abstraction layer202notifies the egress streaming driver210to DMA the data to the host108based on the specified watermarks/thresholds for the data destination206. The ingress streaming driver208manages the reads from the data source204, e.g., LBA reads to the FLASH media104, and the egress streaming driver210manages writes to the data destination206, e.g., DMA writes to the memory of the host108.

If the buffer abstraction layer202determines that there is no data available to be read by the ingress streaming driver208from the data source204, then the execution of the CS program132may be paused until more input data is available from the data source204, as shown at steps410and412of the routine400. For example, if the buffer abstraction layer202detects that the circular buffer in the CPM124for the ingress streaming driver208is empty, the buffer abstraction layer can temporarily halt execution of the eBPF program132until the ingress streaming driver can read more data from the FLASH media104. Similarly, if the buffer abstraction layer202detects that the circular buffer utilized by the egress steaming driver210is full, the buffer abstraction layer can temporarily halt execution of the eBPF program132until the egress streaming driver can write more data to the host memory (the data destination206), as further shown at steps414and416.

As shown at step418, once all of the source data from the data source204specified in the data definition has been read by the ingress streaming driver208, the routine400proceeds to step420, where the CS program132is notified through the buffer abstraction layer202to complete its operation. Upon completion of the CS program132, the controller106may then free the ingress and egress streaming drivers208and210along with any buffer memory and processing resources, and notify the host108of completion of the operation. From step420, the routine400ends.

Based on the foregoing, it will be appreciated that technologies for streaming arbitrarily large amounts of data through computational storage programs are presented herein. While the embodiments herein describe CS programs132comprising eBPF programs executing in a runtime in the computational storage device120, other CS program architectures, interfaces, and/or runtime environments may be imagined by one skilled in the art for execution of computational storage functions and programs in a computational storage device. For example, CS programs132may comprise ARM code, perhaps translated from eBPF, for execution on ARM-based CSPs122and that utilize ARM exceptions to identify when data is exhausted or full in the circular buffers of the streaming drivers208,210. It is intended that this disclosure include all such interface and/or runtime environments. Moreover, it will be appreciated that the embodiments described in this disclosure may be utilized in any storage device for the inclusion of computational storage processing in the device. This may include solid-state drives, hybrid magnetic and solid-state drives, magnetic hard disk drives, optical disk drives, USB flash drives, memory cards and cartridges, storage controllers for arrays of flash memory devices, storage controllers for arrays of high-speed magnetic disk drives, and the like.

The logical operations, functions, or steps described herein as part of a method, process or routine may be implemented (1) as a sequence of processor-implemented acts, software modules, or portions of code running on a controller or computing system and/or (2) as interconnected machine logic circuits or circuit modules within the controller or computing system. The implementation is a matter of choice dependent on the performance and other requirements of the system. Alternate implementations are included in which operations, functions or steps may not be included or executed at all, may be executed out of order from that shown or discussed, including substantially concurrently or in reverse order, depending on the functionality involved, as would be understood by those reasonably skilled in the art of the present disclosure.

Many variations and modifications may be made to the above-described embodiments without departing substantially from the spirit and principles of the present disclosure. Further, the scope of the present disclosure is intended to cover any and all combinations and sub-combinations of all elements, features, and aspects discussed above. All such modifications and variations are intended to be included within the scope of the present disclosure, and all possible claims to individual aspects or combinations of elements or steps are intended to be supported by the present disclosure.