Data compression for grid-oriented storage systems

Methods, computing systems and computer program products implement embodiments of the present invention that include configuring multiple storage system resources to manage a storage entity distributed among the storage system resources. Upon receiving, by a first given storage system resource from a host computer, an I/O request for data in the storage entity, a second given storage system resource responsible for managing the storage entity data referenced in the I/O request is identified, and the I/O request is forwarded to the second given storage system resource. Upon receiving the forwarded I/O request, the second given storage system resource performs a data compression operation while processing the I/O request, and conveys a result of the I/O operation to the first given storage system resource. Upon receiving the result from the second given storage system resource, the first given storage system resource forwards the result to the host computer.

FIELD OF THE INVENTION

The present invention relates generally to data compression, and specifically to distributing compression of data stored in grid-oriented storage systems.

BACKGROUND

Grid-oriented storage (also known as grid storage) is a specialized approach to store data using multiple self-contained interconnected storage nodes so that any given storage node can communicate with any other storage node without the data having to pass through a centralized switch. In grid-oriented storage systems, each storage node comprises an interface node and a data node, wherein the interface node is configured to communicate with host computers and other storage nodes in the grid, and the data node is configured to manage data stored on the storage node.

Grid-oriented storage systems can implement a uniform data distribution scheme that optimizes load balancing, fault-tolerance and redundancy across the system. In operation, if one of the storage nodes fails, then network traffic can be rerouted to a redundant storage node. Additionally or alternatively, if a network path between two storage nodes is interrupted, network traffic can be rerouted via another network path.

SUMMARY

There is provided, in accordance with an embodiment of the present invention a method, including configuring multiple storage system resources to manage a storage entity distributed among the storage system resources, receiving, by a first given storage system resource, an input/output (I/O) request for data in the storage entity, identifying a second given storage system resource responsible for managing the storage entity data referenced in the I/O request, forwarding the I/O request to the second given storage system resource, and performing, by the second given storage system resource, a data compression operation while processing the I/O request.

There is also provided, in accordance with an embodiment of the present invention a grid-oriented storage system, including a network, multiple storage system resources coupled to the network and configured to manage a storage entity distributed among the storage system resources, wherein a first given storage system resource is configured to receive an input/output (I/O) request for data in the storage entity, to identify a second given storage system resource responsible for managing the storage entity data referenced in the I/O request, and to forward the I/O request to the second given storage system resource, and wherein the second given storage system resource is configured to perform a data compression operation while processing the I/O request.

There is further provided, in accordance with an embodiment of the present invention a computer program product, the computer program product including a non-transitory computer readable storage medium having computer readable program code embodied therewith, the computer readable program code including computer readable program code configured to arrange multiple storage system resources to manage a storage entity distributed among the storage system resources, computer readable program code executing on a first given storage system resource and configured, to receive an input/output (I/O) request for data in the storage entity, to identify a second given storage system resource responsible for managing the storage entity data referenced in the I/O request, and to forward the I/O request to the second given storage system resource, and computer readable program code executing on the second given storage system resource and configured to perform a data compression operation while processing the I/O request.

DETAILED DESCRIPTION OF EMBODIMENTS

Overview

Data compression is a data reduction technique that involves encoding information using fewer bits than the original representation. In storage systems, data can be compressed before being written to the physical layer, thereby saving space and reducing the number of input/output (I/O) requests conveyed to storage system resources in the facility. Storage systems can use data compression to reduce storage space needed for operations such as snapshots, local/remote mirroring and cloning. While data compression can save a significant amount of storage space, it can consume a significant amount of processing resources, and create challenges to perform random access operations to compressed data stored in the facility's storage devices.

In storage facilities using dual node compression storage, each of the storage nodes controls a set of physical volumes, maintains its own cache, and typically compresses data prior to conveying an I/O request to the physical layer. In order to improve the response time, users can receive acknowledgements for writes when they are stored in a cache and synchronized to the cache of a peer node as well.

In grid-oriented storage systems, each user volume can be divided into small slices and distributed among the different nodes. Each compression node is then responsible for a subset of the volume's non-contiguous slices, and therefore not responsible for the entire volume. Therefore, some dual node compression properties, such as temporal locality of the data or free space management in the volume are not accessible in grid storage systems.

Embodiments of the present invention provide methods and systems for distributing real-time data compression in grid-oriented storage systems. As described hereinbelow, multiple storage system resources are configured to manage a storage entity distributed among the storage system resources. In embodiments herein, examples of storage system resources include, but are not limited to, storage area network (SAN) systems and network attached storage (NAS) systems, and examples of storage entities that can be distributed among the storage system resources include, but are not limited to, logical/physical volumes, files, blocks, chunks and objects.

Upon a first given storage system resource receiving an input/output (I/O) request for data stored in the storage entity, a second given storage system resource responsible for managing (i.e., storing) the storage entity data referenced in the I/O request is identified. For example, if the I/O request comprises a read request, the second given storage system resource comprises the storage system resource storing the data to be retrieved. Likewise, if the I/O request comprises a write request, the second given storage system resource comprises the storage system resource where the data is to be written. In some embodiments, the first and the second given storage system resources may comprise a single one of the storage system resources. For example, the first given storage system resource can receive a read request for data stored on the first given storage system resource.

Upon identifying the second given storage system resource, the first given storage system resource forwards the I/O request to the second given storage system resource, and the second given storage system resource performs a data compression operation while processing the I/O request. For example, if the I/O request comprises a request to read data, the second given storage system resource retrieves compressed data from a storage device and decompresses the compressed data. Likewise, if the I/O request comprises a request to write data, the second given storage system resource compresses the data and stores the compressed data to a storage device.

Systems implementing embodiments of the present invention can evenly distribute data compression among storage nodes, much like how data slices (described hereinbelow) are distributed between data nodes in grid-oriented storage systems. By distributing data compression operations among storage nodes, data stored on multiple storage nodes can be simultaneously compressed/decompressed by their respective storage nodes, thereby increasing performance. Additionally, grid-oriented storage systems may use less bandwidth when performing mirroring and/or data synchronization operations since compressed data is conveyed between the storage nodes.

FIG. 1is a block diagram that schematically illustrates a data processing storage subsystem20, in accordance with an embodiment of the invention. The particular subsystem (also referred to herein as a storage system) shown inFIG. 1is presented to facilitate an explanation of the invention. However, as the skilled artisan will appreciate, the invention can be practiced using other computing environments, such as other storage subsystems with diverse architectures and capabilities.

Storage subsystem20receives, from one or more host computers22, input/output (I/O) requests, which are commands to read or write data at logical addresses on logical volumes. Any number of host computers22are coupled to storage subsystem20by any means known in the art, for example, using a network. Herein, by way of example, host computers22and storage subsystem20are assumed to be coupled by a Storage Area Network (SAN)26incorporating data connections24and Host Bus Adapters (HBAs)28. The logical addresses specify a range of data blocks within a logical volume, each block herein being assumed by way of example to contain 512 bytes. For example, a 10 KB data record used in a data processing application on a given host computer22would require 20 blocks, which the given host computer might specify as being stored at a logical address comprising blocks 1,000 through 1,019 of a logical volume. Storage subsystem20may operate in, or as, a SAN system.

Storage subsystem20comprises a clustered storage controller34coupled between SAN26and a private network46using data connections30and44, respectively, and incorporating adapters32and42, again respectively. In some configurations, adapters32and42may comprise host bus adapters (HBAs) or high-speed Infiniband or 10 G Ethernet network interface cards (“NIC”). Clustered storage controller34implements clusters of storage modules36, each of which includes a processor52, an interface38(in communication between adapters32and42), and a cache40. Each storage module36is responsible for a number of storage devices50by way of a data connection48as shown.

As described previously, each storage module36further comprises a given cache40. However, it will be appreciated that the number of caches40used in storage subsystem20and in conjunction with clustered storage controller34may be any convenient number. While all caches40in storage subsystem20may operate in substantially the same manner and comprise substantially similar elements, this is not a requirement. Each of the caches40may be approximately equal in size and is assumed to be coupled, by way of example, in a one-to-one correspondence with a set of physical storage devices50, which may comprise disks. In one embodiment, physical storage devices may comprise such disks. Those skilled in the art will be able to adapt the description herein to caches of different sizes.

Each set of storage devices50comprises multiple slow and/or fast access time mass storage devices, herein below assumed to be multiple hard disks.FIG. 1shows caches40coupled to respective sets of storage devices50. In some configurations, the sets of storage devices50comprise one or more hard disks, or solid state drives (SSDs) which can have different performance characteristics. In response to an I/O command, a given cache40, by way of example, may read or write data at addressable physical locations of a given storage device50. In the embodiment shown inFIG. 1, caches40are able to exercise certain control functions over storage devices50. These control functions may alternatively be realized by hardware devices such as disk controllers (not shown), which are linked to caches40.

Each storage module36is operative to monitor its state, including the states of associated caches40, and to transmit configuration information to other components of storage subsystem20for example, configuration changes that result in blocking intervals, or limit the rate at which I/O requests for the sets of physical storage are accepted.

Routing of commands and data from HBAs28to clustered storage controller34and to each cache40may be performed over a network and/or a switch. Herein, by way of example, HBAs28may be coupled to storage modules36by at least one switch (not shown) of SAN26, which can be of any known type having a digital cross-connect function. Additionally or alternatively, HBAs28may be coupled to storage modules36.

In some embodiments, data having contiguous logical addresses can be distributed among modules36, and within the storage devices in each of the modules. Alternatively, the data can be distributed using other algorithms, e.g., byte or block interleaving. In general, this increases bandwidth, for instance, by allowing a volume in a SAN or a file in network attached storage to be read from or written to more than one given storage device50at a time. However, this technique requires coordination among the various storage devices, and in practice may require complex provisions for any failure of the storage devices, and a strategy for dealing with error checking information, e.g., a technique for storing parity information relating to distributed data. Indeed, when logical unit partitions are distributed in sufficiently small granularity, data associated with a single logical unit may span all of the storage devices50.

While such hardware is not explicitly shown for purposes of illustrative simplicity, clustered storage controller34may be adapted for implementation in conjunction with certain hardware, such as a rack mount system, a midplane, and/or a backplane. Indeed, private network46in one embodiment may be implemented using a backplane. Additional hardware such as the aforementioned switches, processors, controllers, memory devices, and the like may also be incorporated into clustered storage controller34and elsewhere within storage subsystem20, again as the skilled artisan will appreciate. Further, a variety of software components, operating systems, firmware, and the like may be integrated into one storage subsystem20.

Storage devices50may comprise a combination of high capacity hard disk drives and solid state disk drives. In some embodiments each of storage devices50may comprise a logical storage device. In storage systems implementing the Small Computer System Interface (SCSI) protocol, the logical storage devices may be referred to as logical units, or LUNs. While each LUN can be addressed as a single logical unit, the LUN may comprise a combination of high capacity hard disk drives and/or solid state disk drives.

While the configuration inFIG. 1shows storage controller34comprising four modules36and each of the modules coupled to four storage devices50, a given storage controller comprising any multiple of modules36coupled to any plurality of storage devices50is considered to be with the spirit and scope of the present invention.

FIG. 2is a block diagram that schematically illustrates components of module36(also referred to herein as a storage system resource), andFIG. 3is a block diagram of a logical volume70(i.e., an example of a storage entity) managed by storage controller34, in accordance with an embodiment of the present invention. In embodiments of the present invention, storage controller34is configured as a grid-oriented storage system, and modules36(and their respective components) comprise storage system resources of the grid-oriented storage system.

InFIG. 2, modules36and their respective components can be differentiated by appending a letter to the identifying numeral, so that the modules comprise modules36A-36D. In addition to storage devices50and processor52, module36comprises a memory60that stores cache40, interface38and a compression application62. Processor52executes compression application62to compress and decompress data, as described hereinbelow.

As shown inFIG. 3, volume70comprises slices64A-64P, wherein the slices comprise a set of partitions on storage devices50. In the example shown inFIG. 2, the slices of volume70are distributed among storage devices50so that storage devices50A store slices64A-64D, storage devices50B storage slices64E-64H, storage devices50C storage slices64I-64L, and storage devices50D storage slices64M-64P.

In some embodiments (as used by Equation 1 hereinbelow), each given slice64may comprise a non-contiguous range of addresses, and system20can use a “modulo N” calculation (i.e., “every N-th block”) when storing the given slice to a given storage device50. For example, if N=5, then slice64A comprises blocks (i.e., storage regions on the storage devices) 1, 6, 11, 16, etc., slice64B comprises blocks 2, 7, 12, 17, etc., slice64C comprises blocks 3, 8, 13, 18 and so forth.

In operation, interface38in a first given module36receives an I/O request from a given host computer22, identifies a second given module36that manages the slice for the data referenced in the I/O request, and conveys the I/O request to the second given module36. Upon receiving the I/O request, compression application62on the second given module performs a data compression operation as part of performing the I/O operation.

In a first example, interface38A receives, from a given host computer22, a write request to store data to slice64G, and forwards the write request to module36B. Upon interface38B receiving the write request from interface38A, compression application62B performs the compression operation by compressing the data, and processor52B stores the compressed data for slice64G. Upon storing the compressed data for slice64G (or cache40B), interface38B conveys a completion indication to module36A, and upon receiving the completion indication, interface38A forwards the completion indication to the given host computer.

In a second example, interface38D receives, from a given host computer22, a read request to retrieve data from slice64J, and forwards the read request to module36C. Upon interface38C receiving the read request from interface38D, processor52C retrieves compressed data for (i.e., associated with) slice64J, and compression application62C performs the compression operation by decompressing the retrieved data. Upon decompressing the retrieved data, interface38C conveys the decompressed data to module36D, and upon receiving the decompressed data, interface38D forwards the decompressed data to the given host computer.

In storage controller34, data distribution (i.e., of slices64) and data compression are independent from one another, thereby minimizing any performance impact of data compression while maintaining load-balancing advantages of the grid-oriented storage configuration implemented by the storage controller. In embodiments of the present invention, I/O operations for compressed volumes (e.g., volume70) pass through compression applications62(also referred to as a compression node), which compress and decompress the data back and forth to interfaces38(also referred to as the data/interface layer) based on the distribution of slices64. In order to allow multiple compression nodes to work on a single user volume, the volume is divided into regions (e.g., slices64). Since the system contains many regions, the compression node itself works with sections, which are aggregates of regions, wherein each section number can be calculated as:
section_number=region number % number_of_sections  (1)

One advantage of using sections comprises load balancing between compression nodes, since each compression node can manage several sections. Upon a given compression node or module failure, its respective sections can be redistributed and re-balanced between the remaining compression nodes. Additionally, using sections can enable modules36to utilize compression objects, wherein each section of a volume (which is built from many regions) comprises a separate compression object. Using sections to define compression objects can conserve the total number of compression objects required by storage controller34.

In some embodiments, the interface layer (i.e., interface38) can directs I/O operation requests to the appropriate compression node by looking up which section is handled by which compression node in a compression distribution table (not shown). To determine the section (i.e., compressed object) number, the following formula can be employed:
((Logical LBA+per_volume_offset)/region_size) % number_of_sections  (2)

Processor52comprises a general-purpose central processing unit (CPU) or special-purpose embedded processors, which are programmed in software or firmware to carry out the functions described herein. The software may be downloaded to modules36in electronic form, over a network, for example, or it may be provided on non-transitory tangible media, such as optical, magnetic or electronic memory media. Alternatively, some or all of the functions of the processor may be carried out by dedicated or programmable digital hardware components, or using a combination of hardware and software elements.

These computer readable program instructions may also be stored in a computer readable storage medium that can direct a computer, a programmable data processing apparatus, and/or other devices to function in a particular manner, such that the computer readable storage medium having instructions stored therein comprises an article of manufacture including instructions which implement aspects of the function/act specified in the flowchart and/or block diagram block or blocks.

Distributed Data Compression in a Grid Storage System

FIG. 4is a flow diagram that schematically illustrates a method for storage controller34to compress data stored in volume70that is distributed among slices64, in accordance with an embodiment of the present invention. In a configuration step80, modules36A-36D in storage controller34are configured as a grid-oriented storage system, using embodiments described supra. While the example inFIGS. 2-3shows volume70distributed over four modules36of a single storage controller34, distributing the volume (or any other type of storage entity) over any number of modules in any number of storage controllers is considered to be within the spirit and scope of the present invention.

In a receive step82, a first given module36receives an I/O request from a given host computer22, and in an identification step84, processor52in the first given module identifies a second given module36that manages the data referenced by the I/O request. For example if processor52B receives a request to either read data from or write data for slice64J, then the first given module comprises module36B, and the second given module comprises module36C.

In operation, the I/O request received from the given host computer comprises an address (e.g., a logical address) for the data on storage devices50. Since system20stores compressed data, the actual address where the data is stored on the storage devices most likely differs from the address in the I/O request. In some embodiments, system20may maintain a mapping table that stores mappings between the requested addresses and the actual addresses of the compressed data. Therefore, in embodiments herein, a given processor52can process a write request by storing compressed data “for” a given slice, and the processor can process a read request by retrieving compressed data “for” (i.e., associated with) a given slice.

In a first forwarding step86, the first given module forwards the I/O request to the second given module, and in a processing step88, the second given module receives the forwarded I/O request, and performs a compression operation while processing the I/O request. As described supra, if the I/O request comprises a write request then the compression operation comprises processor52compressing the data prior to storing the compressed data for a given slice64. Likewise, if the I/O request comprises a read request, then the compression operation comprises processor52in the second given module decompressing compressed data retrieved for (i.e., associated with) a given slice64.

In a convey step90, processor52in the second given module conveys a result of the I/O operation to the first given module. As described supra, if the I/O request comprises a write request, then the result comprises a message indicating that the write operation was completed successfully (or not completed successfully). Likewise, if the I/O request comprises a write request, then the result comprises the data that was retrieved and decompressed for a given slice64. Finally, in a second forwarding step92, upon receiving the result from the second given module, processor52in the first given module forwards the result to the given host computer that issued the I/O request, and the method continues with step82.