Disk image introspection for storage systems

A method for disk image access in a storage system comprises receiving an input/output (I/O) request to a disk image in a file system of a storage system, and in response to the I/O request to the disk image in the file system, reconstructing a file system request. Introspection is performed on the disk image to determine an original file system request based on the I/O request to the disk image. The method further includes laying out a disk image in the file system of the storage system, and performing type-specific introspection on the disk image. Based upon the results of the introspection, data ranges in the disk image are mapped to block or files within the file system. A mapping of disk image data ranges to block or files in the file system is maintained for responding to further I/O requests.

BACKGROUND

The present invention related generally to file systems, and in particular, to virtual machine disk image introspection for storage systems.

Virtual machines in information technology systems often generate input/output (I/O) workloads that comprise small requests with random addresses. Therefore, from the point of performance, streaming throughput is less important than the number of I/O requests per second (IOPS). IOPS intensive workloads are typically highly sensitive to latency. Further, virtual machines perform I/O to disk images, which are effectively file representations of real disks. A disk image can thus be formatted as any guest file system (e.g., EXT3, XFS, NTFS), or any other higher-level structure such as a database (DB). The disk image is stored in a server file system (e.g., GPFS, NFS, EXT3, VMFS, iSCSI) and operations to the disk image are handled by treating the disk image as a typical operating system file. A server file system attempts to optimize I/O requests to the disk image file system without useful knowledge about the structure of the disk image content, resulting in suboptimal I/O latency and throughput. Further, original file system requests are lost, replaced with less predictable I/O requests. This causes data and metadata caching problems and disk layout alignment issues, wherein I/O request to a single block in a misaligned disk image causes two server blocks to be accessed at the server file system.

BRIEF SUMMARY

Embodiments of the invention enhance disk image access in a storage system. In one embodiment, a method of providing virtual machine disk image access in a storage system comprises receiving an I/O request to a disk image in a file system of a storage system, and in response to the I/O request to the disk image in the file system, reconstructing the original disk image file system request. Introspection is performed on the disk image to determine an original file system request based on the I/O request to the disk image.

In one embodiment, the method further includes laying out a disk image in the server file system of the storage system, and performing type-specific introspection on the disk image. Based upon the results of the introspection, data ranges in the disk image are mapped to block or files within the server file system. A mapping of disk image data ranges to block or files in the server file system is maintained for responding to further I/O requests.

DETAILED DESCRIPTION

The following description is made for the purpose of illustrating the general principles of the invention and is not meant to limit the inventive concepts claimed herein. Further, particular features described herein can be used in combination with other described features in each of the various possible combinations and permutations. Unless otherwise specifically defined herein, all terms are to be given their broadest possible interpretation including meanings implied from the specification, as well as meanings understood by those skilled in the art and/or as defined in dictionaries, treatises, etc. The description may disclose several preferred embodiments for file systems, as well as operation and/or component parts thereof. While the following description will be described in terms of file systems and processes for clarity and placing the invention in context, it should be kept in mind that the teachings herein may have broad application to all types of systems, devices and applications.

Embodiments of the invention provide disk image introspection for storage systems in information technology systems such as for cloud computing. It is understood in advance that although this disclosure includes a detailed description on cloud computing, implementation of the teachings recited herein are not limited to a cloud computing environment. Rather, embodiments of the present invention are capable of being implemented in conjunction with any other type of computing environment now known or later developed.

Characteristics are as follows:

Deployment Models are as follows:

Embodiments of the invention provide introspecting a disk image in a storage system for optimizing data access operations on the disk image. In one embodiment, the invention provides a system implementing a method for introspecting and analyzing a disk image in the storage system for identifying a file system of the disk image.

According to embodiments of the invention, a mapping of data ranges in a disk image file system (i.e., guest file system) to block or files within a storage server file system (i.e., server file system of a storage system) is maintained. Information about storage server disk image type is used to transform storage server file system (i.e., server) requests to guest (or host) file system requests. Whenever a write occurs on the disk image, the mapping is updated. Read/write requests (for the disk image) are reconstructed into original file system requests. Knowledge of the original file system requests is utilized for optimizing operations on the disk image, such as prefetching or caching metadata.

In one embodiment, the mapping information is stored in a memory, and if the program or node fails, a new mapping is generated. A database may store this information to handle failures, such that on restart the mapping information is saved on disk.

In the description herein, an original file request means the original file system request in the virtual machine (guest or host system), which is different from the file system in which the disk image (which contains the guest file system) is stored.

One embodiment of the invention provides a system for reconstructing virtual machine file system requests at the storage system level. As such, the storage system has knowledge of the operating system information about the data to be stored. This allows the file system of the storage system to view disk images as transparent objects that can be optimized.

In one implementation, the invention makes data caching and prefetching more accurate and effective, reducing cache pollution and increasing the cache-hit ratio. Further, unlike conventional approaches where metadata requests to the disk image are converted to I/O requests and all information regarding directories and their directory entries are lost, according to embodiments of the invention the entire disk image file system metadata structure is known and can be effectively navigated. Embodiments of the invention are useful with network attached storage (NAS) protocols.

According to an embodiment of the invention, a system for introspecting and analyzing disk image files in a storage server places disk image file on storage devices, and reconstructs and optimizes requests in relation to file system stored by the disk image (e.g., virtual machine file system requests). Information about server disk image type is used to transform storage server file system (i.e., server) requests to guest file system requests. For example:read (disk_image_file, offset, len) where offset and len are relative the disk image file is reconstructed as getattr(x) where x is a file/inode in the virtual machine's file system, or read (file_within_disk_image, offset, len) where the offset and len are now relative to the file stored by the virtual machine's file system, etc.
Further, information about server disk image type is used to arrange bytes on a server disk subsystem. The server disk image type can be any well structured format (e.g., file system (fs), DB). As such, the storage system utilizes the same disk image information as the guest operating system to optimize I/O access requests. This increases accuracy of prefetching, prioritizing metadata, and performing per-file data prefetching. In one example, a file system block in the disk image is stored across multiple file system blocks in the storage system. In this case, the layout of the disk image may be changed such that all of its file system blocks are stored in a single file system block in the storage system. This would reduce the amount of read-modify-write in the storage system and increase performance. Further, the type of file system in the disk image may be automatically determined, or may be specified by a user.

FIG. 1shows a block diagram of an information technology system10according to an embodiment of the invention. The system10comprises a virtual machine (VM)11including a hypervisor, and a storage system12, such as a NAS system, according to an embodiment of the invention. The storage system12includes a disk image introspection system13, according to an embodiment of the invention, wherein the disk image introspection system13comprises an I/O request analyzer module14and an introspection module14A comprising a disk image analyzer module15, a layout manager module16, a layout generator module17, and a prefetch manager module18.

In one embodiment, the I/O request analyzer14uses the disk image analyzer15to map incoming I/O requests from the VM hypervisor11to a superblock, file, inode, dentry, indirect block, or data block. For example, the disk image analyzer15uses a Linux VFS interface to understand a disk image layout. The layout manager16manages mapping of disk image byte ranges to the storage system file layout19. The prefetch manager18uses the disk image analyzer and the I/O request analyzer results to determine data to prefetch in the disk image. The layout generator17uses the disk image analyzer results to divide up the disk image and create a layout for the disk image in the storage system12(FIG. 3B).

In one embodiment of the invention, file system requests in a storage system are reconstructed as follows. I/O request are received for a disk image in the storage system, and introspection on the disk image is performed to reverse engineer the I/O request and determine the original file system request. A set of one or more I/O requests are determined to perform on the storage system to satisfy the original I/O request to the disk image. A response is provided to the original I/O request. It is determined if any additional I/O requests are likely to be performed to the storage system, and if so, the results are cached to optimize possible future I/O requests to the disk image.

FIG. 2shows a process20that utilizes disk image introspection for data prefetching, according to an embodiment of the invention, comprising:Process block21: Guest application (or OS) accesses a disk image DI1.Process block22: Virtual machine (guest system) hypervisor opens disk image DI1in the storage system.Process block23: Storage system opens disk image.Process block24: Storage system determines disk image type (manually or automatically) and analyzes disk image structure by reverse engineering using a file system specific module.Process block25: Storage system performs initial optimizations on disk image (e.g., prefetch all metadata information into a cache).Process block26: Guest system application issues a file system request.Process block27: The hypervisor converts file system request to a read or write request from/to the disk image located in the storage system.Process block28: The storage system reverse engineers the read or write request into the original file system request. This is accomplished by using a file system module specific for the type of file system located in the disk image (e.g., a read of bytes10to20could be converted into a lookup request for an inode).Process block29: The storage system uses knowledge of the original file system request to perform optimization operations on the disk image. File system specific optimizations (from file system module in process block23above) are leveraged. An example for an ext3 disk image involves leveraging an ext3 readahead algorithm to determine prefetching strategy. Further, storage system specific optimizations are performed. For example, for a lookup of an inode, a read request of all dentries in that directory is performed.
The process blocks26-29inFIG. 2are repeated for additional incoming requests.

FIG. 3Aillustrates a process30for laying out a disk image in a file system of a storage system (e.g., storage server according to an embodiment of the invention). Type-specific introspection on the disk image is performed.FIG. 3Aillustrates the runtime aspects of said reverse engineering process block28inFIG. 2, by converting I/O requests to the disk image into the original file system request.FIG. 3Bshows an example mapping31of blocks of disk image33and the blocks of file layout19in the file system in the storage system12.

Based upon the results of the introspection, disk image data ranges are mapped to blocks within the file system in the storage server12. A mapping of disk image data ranges to blocks in the file system in the storage server is maintained in the storage server12for facilitating I/O requests.

Embodiments of the invention can take the form of a computer simulation or program product accessible from a computer-usable or computer-readable medium providing program code for use by or in connection with a computer, processing device, or any instruction execution system. As will be appreciated by one skilled in the art, aspects of the present invention may be embodied as a system, method or computer program product. Accordingly, aspects of the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment (including firmware, resident software, micro-code, etc.) or an embodiment combining software and hardware aspects that may all generally be referred to herein as a “circuit,” “module” or “system.” Furthermore, aspects of the present invention may take the form of a computer program product embodied in one or more computer readable medium(s) having computer readable program code embodied thereon.

Referring now toFIG. 4, a schematic of an example of a cloud computing node implementing an embodiment of the invention is shown. Cloud computing node100is only one example of a suitable cloud computing node and is not intended to suggest any limitation as to the scope of use or functionality of embodiments of the invention described herein. Regardless, cloud computing node100is capable of being implemented and/or performing any of the functionality set forth hereinabove.

Computer system/server112may be described in the general context of computer system executable instructions, such as program modules, being executed by a computer system.

Generally, program modules may include routines, programs, objects, components, logic, data structures, and so on that perform particular tasks or implement particular abstract data types.

Computer system/server112may be practiced in distributed cloud computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed cloud computing environment, program modules may be located in both local and remote computer system storage media including memory storage devices.

As shown inFIG. 4, computer system/server112in cloud computing node100is shown in the form of a general-purpose computing device. The components of computer system/server112may include, but are not limited to, one or more processors or processing units116, a system memory128, and a bus118that couples various system components including system memory128to processor116.

Computer system/server112typically includes a variety of computer system readable media. Such media may be any available media that is accessible by computer system/server112, and it includes both volatile and non-volatile media, removable and non-removable media.

Program/utility140, having a set (at least one) of program modules142, may be stored in memory128by way of example, and not limitation, as well as an operating system, one or more application programs, other program modules, and program data. Each of the operating system, one or more application programs, other program modules, and program data or some combination thereof, may include an implementation of a networking environment. Program modules142generally carry out the functions and/or methodologies of embodiments of the invention as described herein. Computer system/server112may also communicate with one or more external devices114such as a keyboard, a pointing device, a display124, etc.; one or more devices that enable a user to interact with computer system/server112; and/or any devices (e.g., network card, modem, etc.) that enable computer system/server112to communicate with one or more other computing devices. Such communication can occur via Input/Output (I/O) interfaces122. Still yet, computer system/server112can communicate with one or more networks such as a local area network (LAN), a general wide area network (WAN), and/or a public network (e.g., the Internet) via network adapter120. As depicted, network adapter120communicates with the other components of computer system/server112via bus118. It should be understood that although not shown, other hardware and/or software components could be used in conjunction with computer system/server112. Examples, include, but are not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data archival storage systems, etc.