Importing pre-existing data of a prior storage solution into a storage pool for use with a new storage solution

Mechanisms are provided for importing pre-existing data into a storage system utilizing a current storage management system that is different from an original storage management system used to create the pre-existing data. One or more data storage devices are integrated into the storage system in-place without modification of the pre-existing data stored on the one or more data storage devices. Metadata for the pre-existing data is created based on a linear progression of data in the pre-existing data. Read access requests targeting the pre-existing data are executed using the created metadata. Write access requests targeting the pre-existing data are executed by redirecting the write access requests to a copy of the pre-existing data created in another storage location.

BACKGROUND

The present application relates generally to an improved data processing apparatus and method and more specifically to mechanisms for importing pre-existing data of a prior storage solution into a storage pool for use with a new storage solution.

With the introduction of new data storage management solutions into a customer environment, the continued use of pre-existing data created within an older storage management solution often needs to be maintained. This continued use of pre-existing data is usually handled in one of three ways. In a first option, the pre-existing storage management solution is maintained side-by-side with the new storage management solution. This option limits the customer because no new functional capabilities with the newer storage management solution can be shared with the pre-existing data under continued management of the older storage management solution. Moreover, this option introduces additional management complexity because the old storage management solution must be maintained along with the new storage management solution.

In a second option, the data may be dumped to a raw format media, e.g., magnetic tape, and then re-imported into the new storage management solution as if it were new data. This solution tends to be impractical due to the massive amounts of data involved and the time required to move data between systems, e.g., between the host system and a raw format media system and then back from the raw format media system into the system configured with the new storage management solution.

As a third option, the new storage management solution may be developed so that it ensures compatibility with the old storage management solution and thus, merely adds-on to the pre-existing storage management solution. This option often limits the user to a product line from one exclusive vendor, i.e. the customer can only obtain storage management solutions from the same vendor since the customer requires backward compatibility to access the pre-existing data, and often does not expand the data management capability of the pre-existing data since the new storage management solution is limited in what it can do by the need to provide backward compatibility. This is primarily because the data in the storage management solution is often closely tied and described by internal storage solution metadata that is proprietary to the pre-existing storage management solution.

SUMMARY

In one illustrative embodiment, a method, in a data processing system, is provided for importing pre-existing data into a storage system utilizing a current storage management system that is different from an original storage management system used to create the pre-existing data. The method comprises integrating one or more data storage devices into the storage system in-place without modification of the pre-existing data stored on the one or more data storage devices. The method further comprises creating metadata for the pre-existing data based on a linear progression of data in the pre-existing data. Moreover, the method comprises executing read access requests targeting the pre-existing data using the created metadata. In addition, the method comprises executing write access requests targeting the pre-existing data by redirecting the write access requests to a copy of the pre-existing data created in another storage location.

In other illustrative embodiments, a computer program product comprising a computer useable or readable medium having a computer readable program is provided. The computer readable program, when executed on a computing device, causes the computing device to import pre-existing data into a storage system utilizing a current storage management system that is different from an original storage management system used to create the pre-existing data, by performing various ones of, and combinations of, the operations outlined above with regard to the method illustrative embodiment.

In yet another illustrative embodiment, system/apparatus is provided that comprises a storage management system and a storage system comprising one or more first data storage devices storing data created using the storage management system. The storage management system is configured to perform various ones of, and combinations of the operations outlined above with regard to the method illustrative embodiment.

DETAILED DESCRIPTION

The illustrative embodiments address the problems in the prior art by provide a mechanism for importing pre-existing data of a prior storage solution “in place” into a storage pool for use with a new storage solution. What is meant by the term “in place” is that the data does not have to be dumped into a raw format on an intermediate raw format media but instead, the data can continue to reside on the exact same physical media in which it originally resided under the older storage management solution.

The illustrative embodiment will be described in terms of data being provided in storage devices corresponding to logic unit numbers (LUNs). LUNs are numbers assigned to logical units of a storage system. A logical unit of a storage system comprises one or more storage devices addressed by the Small Computer System Interface (SCSI) protocol or similar protocols, such as Fibre Channel or iSCSI. A LUN may be used with any device that supports read/write operations, such as a tape drive, hard disk, solid state disk, or the like, but is most often used to refer to a logical disk created in a storage area network (SAN). While LUNs are the actual logical identifier of a set of storage locations on one or more physical storage devices, the term LUN is often used to refer to the physical devices or logical disks themselves and such convention will be used herein as well.

With the illustrative embodiments described herein, one or more pre-existing LUNs, i.e. one or more sets of storage locations, that store “old” data “in place,” are imported into a separate non-metadata storage tier of a multi-tiered storage system having a new storage management pool, i.e. a collection of multiple LUNs that may comprise a combination of pre-existing or “old” data created under a prior storage management system, and “new” data created under a new storage management system. The metadata to describe the linear layout of the pre-existing LUN(s) is maintained within the metadata storage tier of the multi-tiered storage system having the new storage management pool. The linear layout of the pre-existing LUN can be easily described by the new storage management metadata and any access to the pre-existing LUN is performed using the new metadata description. Once accesses start occurring through the new storage management pool using the new metadata description, advanced capabilities of the new storage management system, such as snapshots, clones, or the like, can be applied to the “old” data.

In this way, new metadata is “wrapped around” pre-existing LUNs such that new functionality of a new storage management system may be utilized with “old” data without having to implement the inefficient, complex, or impractical mechanisms described above with regard to the prior art. In addition, the use of the mechanisms of the illustrative embodiments allows new modifications to pre-existing, or “old” data, to be redirected to a different and new LUN in the new storage management pool. That is, when a new modification is to be performed on the pre-existing or “old” data, this modification may be performed to a new copy of the old data in a new LUN of the new storage management pool and then future accesses to the data may be redirected to this new copy of the data. In this way, the pre-existing or “old” data is slowly migrated onto new LUNs of the new storage management pool using the new metadata of the new storage management system.

An operating system runs on processing unit206. The operating system coordinates and provides control of various components within the data processing system200inFIG. 2. As a client, the operating system may be a commercially available operating system such as Microsoft Windows 7 (Microsoft and Windows are trademarks of Microsoft Corporation in the United States, other countries, or both). An object-oriented programming system, such as the Java programming system, may run in conjunction with the operating system and provides calls to the operating system from Java programs or applications executing on data processing system200(Java is a trademark of Oracle and/or its affiliates.).

As a server, data processing system200may be, for example, an IBM® eServer™ System P® computer system, running the Advanced Interactive Executive (AIX®) operating system or the LINUX operating system (IBM, eServer, System p, and AIX are trademarks of International Business Machines Corporation in the United States, other countries, or both, and LINUX is a registered trademark of Linus Torvalds in the United States, other countries, or both). Data processing system200may be a symmetric multiprocessor (SMP) system including a plurality of processors in processing unit206. Alternatively, a single processor system may be employed.

As mentioned above, the illustrative embodiments provide a mechanism for importing pre-existing data “in place” into a new storage management pool associated with a new storage management system. For example, with reference again toFIG. 1, assume that a computing system, such as server104and/or106has an associated multi-tiered data storage system, such as may be stored in one or more network attached storage units108or other storage devices associated with one or more of server104and/or106. In one illustrative embodiment, the multi-tiered data storage system is a clustered file system and storage pool system, as will be described hereafter.

FIG. 3is an example diagram of a multi-tiered data storage system in accordance with one illustrative embodiment. As shown inFIG. 3, the multi-tiered data storage system300is a storage system having sets of storage devices312-316,322-328, and332-338configured in tiers or groups310,320, and330, where each tier or group may have different performance capabilities and may be configured for the storage of different types of data. For example, in one illustrative embodiment, a low tier330of the multi-tiered storage system300may be associated with infrequently accessed data, a middle tier320may be associated with more frequently accessed data, a top tier310may be associated with cached data or data that is frequently accessed, or the like. Each tier310,320, and330of storage devices may be comprised of one or more physical storage devices312-316,322-328, and332-338of a same or different type. For example, the low tier330of the multi-tiered storage system300may be comprised of relatively slow hard drives used for archival purposes while the top tier310of the multi-tiered storage system may be comprised of solid state disks (SSDs) or the like due to their relative high speed access performance. Other types of multi-tiered structures300having more or less tiers and different types of storage devices may be used without departing from the spirit and scope of the illustrative embodiments.

The multi-tiered storage system300, in accordance with one illustrative embodiment, is a combination of a clustered file system and a storage pool system wherein LUNs are part of a storage pool and the aggregate content of the storage is presented as available storage space to the clustered file system. The storage pool is comprised of multiple storage tiers310,320, and330, and certain data may be placed on certain tiers depending on performance criteria, such as access usage and the like. Thus, as discussed above, data that is infrequently accessed may be placed in storage tier330and frequently accessed data may be placed in storage tier310.

In accordance with the illustrative embodiments, the multi-tiered storage system300includes a tier305of one or more storage devices for storing metadata, i.e. data that describes the data stored in the other tiers310,320, and330, of the multi-tiered storage system300. This is referred to as the system storage tier305and in fact may store the metadata as well as user data if the user has not specifically set up unique storage tiers in the particular implementation of the multi-tiered storage system300. All the other tiers310,320, and330, beyond the system storage tier305contain user data only. Thus, the descriptive meta-data for user data may appear to be, and may in actuality be, stored separately from the user data that the metadata is describing. Since the metadata for the user data is stored separately from the actual user data itself, pre-existing data in pre-existing LUNs may be imported into the storage pool “in-place” with no data movement of the originally imported LUNs.

With the illustrative embodiments, pre-existing or “old” data, when being imported into a new storage system300having a new storage management system, are brought into the new storage system300“in place” without movement of the data. That is, the physical storage devices may be brought into the system300as is without modification.

The new storage management system may utilize metadata for describing the data in the storage system300that is not compatible with metadata used by a prior storage management system used to create the pre-existing or “old” data. That is, with storage management systems, the storage virtualization solutions employed vary in their layout and manipulation capabilities for translating virtual to physical data addressing. For example, some storage management system virtualization solutions, such as the pre-existing or “old” data storage management system of the examples of the illustrative embodiments, are quite simplistic and have little virtualization capabilities other than presenting a set of physical disks as one virtual LUN that is nothing more than a concatenation of the physical disks. Others, such as the new storage management system of the examples used to describe the illustrative embodiments, may be fairly sophisticated and may have a sophisticated metadata capability that allows complex data layouts on multiple LUNs and allows physical data placement moves while still presenting an unchanged virtual layout during runtime. It should be appreciated that this is only an example and that a relative difference in complexity is not required for use of the illustrative embodiments, only that the metadata of a storage management system used to create the pre-existing or “old” data is incompatible with the metadata of the new storage management system of the storage system300.

The physical storage devices that store the pre-existing or “old” data may be incorporated into the storage system300as a unique storage tier340separate from the metadata tier305and, in some illustrative embodiments, separate from other tiers310,320, and330in the multi-tiered storage system300. In other illustrative embodiments, the pre-existing or “old” data may be part of a tier storing data configured for use with the new storage management system as well.

The LUNs of the pre-existing or “old” data are associated with an interface350in a storage management system360such that when the LUNs are read, they are accessed as a linear progression of data and offsets associated with the data. The interface350and storage management system360may be implemented as software, hardware, or any combination of software and hardware. In one illustrative embodiment, the interface350and storage management system360are implemented as software instructions executed by one or more processors of one or more data processing systems.

The interface350builds a set of metadata370detailing the linear progression of data within the LUNs and combines this metadata under a single file object370that appears within the file system. The metadata370represents a one-to-one mapping of address and offset to physical storage location on the physical storage devices, i.e. there is no virtualization assumed for the imported pre-existing data. Thus, the offsets identified in the metadata370provide metadata pointers directly pointing to the physical location of the data on the physical storage devices on which the LUNs are stored.

It should be noted that the description of the illustrative embodiments set forth herein assumes that the LUNs represent the actual physical storage devices, but the illustrative embodiments are not limited to such. Rather, the LUNs may in fact be virtual LUNs if the storage management system used to create the pre-existing data supported a storage virtualization solution. As a result, when the mapping for the imported LUNs is built, a linear layout is assumed, because this is the semantic for the accessor of a LUN served by an external storage system, and thus, the offsets may specify a virtual mapping of physical LUNs within an external storage controller.

With reference again toFIG. 3, effectively, the building of the set of metadata for the imported LUNs as a linear progression of data and offsets associated with the data gives a new access name for the LUN. When the LUN is read through the storage management system360, e.g., in response to a read request from a logical partition (LPAR) client302via a virtual input/output server (VIOS)304, metadata address translation mechanisms362of the storage management system360points to the pre-existing LUN data at the offset associated with the metadata pointer. When the LUN is written to, the storage management system360implements a redirect on write (ROW) operation. That is, when there is a write to the pre-existing LUNs, a snapshot380of the pre-existing data in the pre-existing LUNs is generated, i.e. an instant copy of the data representing the state of the data at the particular time that the copy of data is generated. A copy of the pre-existing data382is generated at a new location in a same or different tier310,320,330, or340of the multi-tiered storage system300.

The modifications to the pre-existing data are then applied to the new location of the data in the same or different tier310,320,330, or340. The metadata for the “old” data is updated with metadata386that redirects accesses to the data (“old” or modified) to the new location of the modified copy of the data382. This new metadata386is in the format and of the type used by the new storage management system360and implements any virtualization implemented by the new storage management system360, i.e. the new metadata386utilizes user space or virtual addresses within the user/virtual address space implemented by the new storage management system360that may be translated by the address translation mechanisms362into physical addresses for accessing the new location of the modified copy of the data382.

It should be noted that the storage system is capable of differentiating between LUNs that store pre-existing data that was created under a previous storage management system and copies of these LUNs that store data created under the current storage management system. This may be done in a number of different ways. One way in which this may be done is by using snapshots and generation number tags. That is, a snapshot preserves a metadata tree at a particular point in time. Whenever a block of data is “born” in the storage system of the illustrative embodiments, i.e. it is first allocated, it may be associated with a generation number, e.g., generation “323.” The generation is tagged with the metadata that points to that block that was just allocated so that the metadata essentially identifies, for example, block “10” being a location LUN “20,” offset 0x23234 and that this block “10” was allocated as part of generation “323.” When the snapshot is created, the storage system essentially is setting a condition that blocks within generation “323” or earlier are to be preserved and not deleted. When the snapshot is created, the generation number is incremented such that blocks of data allocated after the snapshot have a later generation number, e.g., “324.” Thus, the storage system can always tell what blocks of data need to be preserved, e.g., blocks on the pre-existing or “old” LUNs, and blocks that are not being preserved by a point in time snapshot and can be updated in place.

In this way, pre-existing data in pre-existing LUNs are imported into the multi-tiered storage system300“in-place” without modification to the data being required for read access to the data. Metadata for locating data on these pre-existing LUNs uses a one-to-one address mapping where the address or pointer in the metadata points to a physical location within a linear progression of data of the LUN. If the data is not modified, then the pre-existing data in the pre-existing LUN is not updated or moved and the data remains “in-place.” The data is only moved to another storage location, e.g., another tier of the multi-tiered storage system300, if the data is being modified, such as due to a write to the data being performed. In such a case, a snapshot of the data is generated and a copy of the data is created in a new location of the multi-tiered storage system300. The modifications are made to the copy of the data in the new location while the snapshot preserves the original state of the data at the time just prior to the modification being performed. The metadata associated with the data is updated to point to the new location of the copy of the data so that future accesses to the data are performed on this new data.

As a result, the pre-existing data in the pre-existing LUNs may be incrementally migrated into the multi-tiered storage system300such that the new capabilities of the new storage management system360may be executed with regard to the migrated data. For example, LUNs that, prior to the implementation of the new storage management system360, were not capable of being the subject of snapshots due to the limitations of the prior storage management system under which they were created, may now, through the mechanisms of the illustrative embodiments and the incremental migration capabilities, be the subject of such advanced capabilities. Furthermore, the VIOS304may present the migrated data or LUNs as new block devices, e.g., LUNs to the client LPAR clients302.

FIG. 4is a flowchart outlining an example operation for importing a pre-existing LUN into a multi-tiered storage system utilizing a new storage management system in accordance with one illustrative embodiment. The operation ofFIG. 4may be performed by a storage management system of a multi-tiered storage system, for example.

As shown inFIG. 4, the operation starts with the inclusion of the pre-existing LUN into a multi-tiered storage system “in-place” (step410). Metadata for the pre-existing LUN is generated in the system storage tier for the pre-existing LUN assuming a linear progression of data with the metadata pointers pointing directly to the physical storage locations of the data on the pre-existing LUN (step420). Thereafter, a determination is made as to whether an access request is received directed to the pre-existing LUN (step430). If not, then the operation returns to step430and awaits an access request to be received.

If an access request is received, then a determination is made as to whether the access requested is a read or a write (step440). If the access request is a read access, then the metadata is used to access the data from the pre-existing LUNs “in-place” without moving the data or modifying the metadata for the pre-existing LUN (step450). The operation then returns to step430for a future access request to be received.

If the access request is a write access request (step440), a copy of the data in the pre-existing LUN is created and a new copy of the data is generated in a separate storage location within the multi-tiered storage system (step460). The metadata is updated to point to the new location of the copy of the data (step470). Thereafter, the modification made by the write access request is performed on the copy of the data in the new location (step480). The operation then terminates. It should be appreciated that thereafter, read and write accesses to the same data are, via the updated metadata, directed to the new location where the copy of the data is located.

Thus, the illustrative embodiments provide mechanisms for importing pre-existing data from pre-existing LUNs, or other physical or logical groupings of data, “in-place” into a storage system that utilizes a new or different storage management system than that used to create the pre-existing data in the pre-existing LUNs. The illustrative embodiments provide mechanisms for the gradual and incremental migration of the pre-existing data from the pre-existing LUNs into new LUNs configured under the new storage management system, upon which the new capabilities of the new storage management system may be implemented. In so doing, the illustrative embodiments avoid the drawbacks discussed above with regard to known mechanisms.

It should be appreciated that while the illustrative embodiments are described in terms of a multi-tiered storage system and the data being provided as LUNs, the illustrative embodiments and the present invention are not limited to such. To the contrary, the illustrative embodiments may be utilized with any storage system architecture in which the pre-existing data may be separately stored or differentiated from the data created under the storage management system of the storage system such that the pre-existing data may be handled in accordance with the mechanisms described above. Such a storage system may be a single tier storage system or multi-tiered storage system. Moreover, other groupings or logical arrangements of data other than LUNs may be used without departing from the spirit and scope of the illustrative embodiments. For example, the groupings or logical arrangements of data may be made based on tiers of a storage system, or the like.