System and method of using persistent memory to support small-sized data append for object store

A system and a method are disclosed that efficiently supports an append operation in an object storage system. A size of data received with a request for an append operation from an application is determined based on a data-alignment characteristic of a storage medium. Data that is not aligned with the data-alignment characteristic is stored in persistent memory and aggregated with other data from the application that is not aligned with the data-alignment characteristic, while data that is aligned with the data-alignment characteristic is stored directly in the storage medium. Aggregated data that becomes aligned with the data-alignment characteristic as additional requests for append operations are received are migrated to the storage medium.

TECHNICAL FIELD

The subject matter disclosed herein relates to object storage systems. More specifically, the subject matter disclosed herein relates to a system and a method that efficiently supports an append operation in an object storage system.

BACKGROUND

An append operation is a common operation for an object storage device. Typical object storage devices provide some round-about or work-around techniques that approximate an append operation, but such object storage devices generally do not have a native and efficient way of supporting an append operation. One technique that is used for an append operation is a read-modify-write operation, which may be a costly and time consuming storage-media access technique.

FIG.1pictorially depicts an append operation100that is essentially a read-modify-write operation. At101, an original object is read from storage. At102, new data is appended to the original object. At103, the updated object is written back to storage.

Additionally, small write sizes, that is, a write size that is less than native block size of a storage media, are generally not handled efficiently by the storage media of an object storage device. Consequently, a large number of small-sized append operations that may be performed for a large number of small sized writes may actually increase latency and may wear the media of a storage device at a fast rate.

SUMMARY

An example embodiment provides an object storage system that may include a storage medium, persistent memory and a controller. The storage medium may include a data-alignment characteristic. The persistent memory may include non-volatile memory that is accessible in a random access, byte-addressable manner. The controller may be coupled to the storage medium and the persistent memory, and the controller may be configured to: receive data from an application for a append operation; determine a size a first portion of the data to be a size of the data divided by a first integer multiple of a size of the data-alignment characteristic of the storage medium in which the first integer multiple may be greater than 0; determine a size of a second portion of the data to be a remainder of the size of the data divided by the first integer multiple of the size of the data-alignment characteristic of the storage medium; store in the storage medium the first portion of the data based on the first integer multiple being greater than 0; stage in the persistent memory the second portion of the data based on a size of the second portion of the data being greater than 0 in which the second portion of the data may be staged in the persistent memory with other second portions of data received with previously received append operations; and store in the storage medium staged second portions of the data that collectively having a size that is a second integer multiple of the size of the data-alignment characteristic of the storage medium, the second integer multiple being greater than 0. In one embodiment, the controller may be further configured to: use a mapping table to indicate that the first portion of the data is stored in the storage medium based on the first integer multiple being greater than 0; and use the mapping table to indicate that the second portion of the data is stored in persistent memory based on the size of the second portion of the data being greater than 0.

An example embodiment provides an object storage system that may include a storage medium, persistent memory and a controller. The storage medium may include a data-alignment characteristic. The persistent memory may include non-volatile memory that is accessible in a random access, byte-addressable manner. The controller may be coupled to the storage medium and the persistent memory, and the controller may be configured to: receive first data from an application for a first append operation; determine a size a first portion of the first data to be a size of the first data divided by a first integer multiple of a size of the data-alignment characteristic of the storage medium in which the first integer multiple being greater than 0; determine a size of a second portion of the first data to be a remainder of the size of the first data divided by the first integer multiple of the size of the data-alignment characteristic of the storage medium; store in the storage medium the first portion of the first data based on the first integer multiple being greater than 0; stage in the persistent memory the second portion of the first data based on a size of the second portion of the first data being greater than 0; receive second data from the application for a second append operation that is subsequent to the first append operation; determine a size a first portion of the second data to be a size of the second data divided by a second integer multiple of a size of a data-alignment characteristic of the storage medium, the second integer multiple being greater than 0; determine a size of a second portion of the second data to be the size of the second data divided by the second integer multiple of the size of the data-alignment characteristic of the storage medium; store in the storage medium the first portion of the second data based on the second integer multiple being greater than 0; stage in the persistent memory the second portion of the second data based on a size of the second portion of the second data being greater than 0; and store in the storage medium the second portion of the first data and the second portion of the second data staged in the persistent memory that collectively having a size that is a third integer multiple of the size of the data-alignment characteristic of the storage medium, the third integer multiple being greater than 0.

An example embodiment provides a method to store data from an application in an object storage system using an append operation in which the method may include: receiving first data from the application for a first append operation; determining a size a first portion of the first data to be a size of the first data divided by a first integer multiple of a size of a data-alignment characteristic of the storage medium in which the first integer multiple may be greater than 0; determining a size of a second portion of the first data to be a remainder of the size of the first data divided by the first integer multiple of the size of the data-alignment characteristic of the storage medium; storing in the storage medium the first portion of the first data based on the first integer multiple being greater than 0; staging in a persistent memory the second portion of the first data based on a size of the second portion of the first data being greater than 0 in which the persistent memory may include non-volatile memory that is accessible in a random access, byte-addressable manner; receiving second data from the application for a second append operation that is subsequent to the first append operation; determining a size a first portion of the second data to be a size of the second data divided by a second integer multiple of a size of a data-alignment characteristic of the storage medium in which the second integer multiple may be greater than 0; determining a size of a second portion of the second data to be the size of the second data divided by the second integer multiple of the size of the data-alignment characteristic of the storage medium; storing in the storage medium the first portion of the second data based on the second integer multiple being greater than 0; staging in the persistent memory the second portion of the second data based on a size of the second portion of the second data being greater than 0; and storing in the storage medium the second portion of the first data and the second portion of the second data staged in the persistent memory that collectively having a size that is a third integer multiple of the size of the data-alignment characteristic of the storage medium, the third integer multiple being greater than 0.

DETAILED DESCRIPTION

In the following detailed description, numerous specific details are set forth in order to provide a thorough understanding of the disclosure. It will be understood, however, by those skilled in the art that the disclosed aspects may be practiced without these specific details. In other instances, well-known methods, procedures, components and circuits have not been described in detail not to obscure the subject matter disclosed herein.

The terminology used herein is for the purpose of describing some example embodiments only and is not intended to be limiting of the claimed subject matter. As used herein, the singular forms “a,” “an” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms “comprises” and/or “comprising,” when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof. The terms “first,” “second,” etc., as used herein, are used as labels for nouns that they precede, and do not imply any type of ordering (e.g., spatial, temporal, logical, etc.) unless explicitly defined as such. Furthermore, the same reference numerals may be used across two or more figures to refer to parts, components, blocks, circuits, units, or modules having the same or similar functionality. Such usage is, however, for simplicity of illustration and ease of discussion only; it does not imply that the construction or architectural details of such components or units are the same across all embodiments or such commonly-referenced parts/modules are the only way to implement some of the example embodiments disclosed herein.

As used herein, the term “module” refers to any combination of software, firmware and/or hardware configured to provide the functionality described herein in connection with a module. The software may be embodied as a software package, code and/or instruction set or instructions, and the term “hardware,” as used in any implementation described herein, may include, for example, singly or in any combination, hardwired circuitry, programmable circuitry, state machine circuitry, and/or firmware that stores instructions executed by programmable circuitry. The modules may, collectively or individually, be embodied as circuitry that forms part of a larger system, for example, but not limited to, an integrated circuit (IC), system on-chip (SoC) and so forth. The various components and/or functional blocks disclosed herein may be embodied as modules that may include software, firmware and/or hardware that provide functionality described herein in connection with the various components and/or functional blocks.

The subject matter disclosed herein provides an efficient way to support an append operation in an object storage device that avoids a read-modify-write operation. A size of data received with a request for an append operation from an application is determined based on a data-alignment characteristic of a storage medium. Data that is not aligned with the data-alignment characteristic is stored in persistent memory and aggregated with other data from the application that is not aligned with the data-alignment characteristic, while data that is aligned with the data-alignment characteristic is stored directly in the storage medium. Aggregated data that becomes aligned with the data-alignment characteristic as additional requests for append operations are received are migrated to the storage medium.

A table may be created that indicates where object data associated with an application is located. That is, the table may include locations in the storage medium and in the persistent memory where object data has been stored.

FIG.2depicts a block diagram of an object storage system200that provides an efficient technique to support an append operation according to the subject matter disclosed herein. The object storage system200may include a controller201, a data buffer202, a persistent memory203, and a storage media204. The controller201is communicatively coupled to each of the data buffer202, the persistent storage203, and the storage media204. The various components depicted inFIG.2for the object storage system200may be implemented as one or more modules.

The controller201may include, for example, at least one microprocessor, at least one digital signal processor, at least one microcontroller, or the like. There may be a memory (not shown) that is coupled to the controller201that may store command code to be used by the controller201or a user data.

The buffer202may be configured to receive data and commands from an application206executing in a host system205, and send data and status information to the host system205. Although only one application206has been depicted as executing in the host system205, it will be understood that any number of applications206may be executing in the host system205. Additionally, it will also be understood that more than one host system205may be communicatively coupled to the object storage device200.

The persistent memory203may be non-volatile memory that is accessible in a random access, byte-addressable manner. In one example embodiment, the persistent memory203may be, for example, dynamic random access memory (DRAM) or static random access memory (SRAM) having a power supply provided with battery backup. In another example embodiment, the persistent memory203may be a nonvolatile (NV) dual inline memory module (DIMM). In still another embodiment, the persistent memory203may be a storage class memory (SCM). The persistent memory203provides persistent storage, for example, across a power-failure.

The storage media204may be a non-volatile mass-storage device, such as, but not limited to one or more solid-state drives (SSDs) and/or one or more hard drives (HDs). In one embodiment, the persistent memory203may be used by the controller201to store command code and/or user data that is used by the controller201. The storage media204may have a native block size. In one embodiment, the native block size of the storage media may be 512 bytes.

In one embodiment, when the application206sends a request for an append operation and data associated with the append operation, the data buffer202receives the data. The controller201determines whether the size of data associated with the received append operation matches the native block size of the storage media204. In one embodiment, the size of the received data is divided by the size of the native block size. The portion of the received data that has a size that is an integer multiple greater than 0 of the native block size is written directly into the storage media204. Any remaining portion of the received data that has a size that is less than the size of the native block size (i.e., a remainder) is written to the persistent memory203for later storage into the storage media204.

Small-sized append operations are aggregated, or staged, in the persistent memory203, and a mapping entry in a table may be created for an object that may point to the address within the persistent memory203. Once a portion of an object in the persistent memory203grows to be large enough to efficiently be written to the storage media204(i.e., is an integer multiple greater than 0 of the native block size of the storage media204), then the object data in the persistent memory203may be migrated, or written, to the storage media204as a background operation. The mapping entry or entries in the table for the migrated data may be updated accordingly.

FIG.3depicts a portion of an example mapping table300that may be created in, for example, the persistent memory203that points to locations of the different portions of object data written to the object storage system200by the application206. That is, an object may be represented by a unique key that is able to point to multiple values, or locations, in the object storage system200. Each value may represent a segment or portion of an entire object. A segment may either reside in the storage media204or in the persistent memory203. For example, an object key may index values V1through Vnof an object. As depicted inFIG.3, the values V1and V2indicate locations in the storage media204. The values V3and Vnindicate locations in the persistent memory203. During a read operation, the mapping entries for an object may be used to assemble the object from the various segments or portions.

FIG.4pictorially depicts an example400of multiple small append operations that are aggregated, or staged, in the persistent memory203. Consider that the storage media204has a native block size of 512 bytes. At401, a first append operation (Append1) results in 124 bytes (a remainder) that are not byte aligned. The 124 bytes may be written into the persistent storage203, and an entry created in a table (not shown) for the 124 byte object. For Append1, (and also for Append2through Append4) there may have also been data that was byte aligned and written directly into the storage media204. Subsequently, at402a second append operation (Append2) results in 327 bytes that are not byte aligned. The 327 bytes may be written into the persistent storage203, and an entry created in the table (not shown) for the 327 byte object. A third and fourth append operations at403and404may be received that respectively result in 657 bytes and 2998 bytes being written into the persistent memory203. At this point (at405), the four example append operations Append1-Append4have a collective byte total of 4096 bytes. The 4096 bytes are written to the storage media204at406, and the corresponding entries for the four append operations Append1-Append4are updated to indicate that the data that has been migrated to the storage media204.

FIG.5pictorially depicts another example500of an append operation according to the subject matter disclosed herein. Consider an append operation of 5000 bytes and that the storage media204has a native block size of 512 bytes. The size of the data received with the append operation is divisible by the native block size of 512 with a remainder of 392 bytes. The portion of the data having a size of 4608 bytes is written directly into the storage media204, and the remaining 392 bytes are written into the persistent memory203. Entries are created in a table (not shown) to indicate a location of the 4608 bytes that are stored in the storage media204, and to indicate a location for the 392 bytes that are stored in the persistent memory203.