Method and system for migrating data

A method, system, and medium for replicating data stored on a storage array is provided. Source data, which may change during the copy process, is copied to a target device. The source data is checked to determine whether any changes were made to the data. If so, the respective data bin is copied to the target component.

CROSS-REFERENCE TO RELATED APPLICATIONS

Not applicable.

Not applicable.

TECHNICAL FIELD

This invention relates to the field of computer programming. More particularly, the present invention provides a new and useful method to migrate data from a source to a destination in a computer-networking environment.

BACKGROUND OF THE INVENTION

Storage arrays are used in various computer-networking applications to store data accessed by servers and other devices. Over time, these storage arrays can become full or technologically obsolete. When the storage array reaches its maximum capacity, or for a variety of other reasons, the data stored in the initial data-storage array is desired to be migrated onto a target storage array.

Typical storage arrays include network-storage devices, RAID-type storage devices, or other types of storage media as is known in the art. Historically, replicating data from a first source to a target source has been problematic and expensive. One prior-art solution includes attempting to copy all of the files at the file level from the source device to the target device. However, this process typically requires an initial and enduring shut down of any applications that may affect any data-stored on the data-storage devices. Such a solution is impractical because the applications that need to be shut down are often relied upon for productivity.

Another prior-art solution includes placing a host agent, or some other copying application, on each server that has data to be copied.FIG. 1Aillustrates such a topology. A source storage array is coupled to a destination array by a network switch114, which is coupled to multiple servers116,188, and120. This copying application121can attempt to mirror the various data components to copy only after it is recognized by the respective server, which requires rebooting the server. Each time one of the servers has to be rebooted, the applications running on that server are unavailable. By relegating the copying process to an application121that runs on the same server as needed applications, the server's performance is impeded. Moreover, each server (PC, node, etc.) must be equipped with the copying application121. This can translate to installing the copying application121on tens or hundreds of machines, rebooting them, and running them with diminished performance. When the copying application121is removed, each server must be rebooted again. This method requires much human interaction; is time intensive; requires multiple reboots.

In still another potential prior-art solution, an appliance is inserted between the source and destination storage arrays. Data is written through this device. But this scheme is also time and resource intensive in that all of the applications that can affect the data being copied must be brought down to insert the appliance. Moreover, after the data is copied from the source destination to the target destination, manual intervention must take place to remove the physical appliance from between the two storage arrays. The servers that store data in the storage array must then be reconfigured to point to the target storage array rather than the original source storage array.

As the amount of data storage increases, surpassing the order of terabytes, the number of applications that make use of the data increases. Thus, terminating all applications that can affect any data stored on the source data-storage device becomes problematic. Accordingly, there is a need for a new and useful method of copying data from a first storage device to a second storage device that reduces the time necessary to shut down applications that may modify the data during the entirety of the copying process. Although the data-modification applications may need to be temporarily shut down, there is a need to minimize the time associated with shutting down all of the different applications that could potentially affect the data to be migrated.

SUMMARY OF THE INVENTION

The present invention solves at least the above problems by providing a system and method for copying data without having to shut down applications that may affect the data during the entirety of the copying process. In one aspect of the invention, a system is provided that includes a computer-program product. The computer-program product includes instructions for monitoring the copying process and identifying data units that contain data modified during the copying process. The invention can first copy all data that is stored on the source data-storage device and then make a second pass to copy only the data that has been changed during the copying process. The various software applications running on various servers that might affect the data being copied need only be shut down during the subsequent pass. This secondary pass typically requires a disproportionately smaller amount of time to copy the data from a first device to a second device than the entirety of the copying process. This difference is accentuated as bandwidth is restricted between the various severs and their corresponding remote storage device. If a server is connected to a remote storage device across a network, transferring massive amounts of data can consume considerable time.

The present invention has several practical applications in the technical arts; not limited to enabling an administrator to migrate vast amounts of data from a first storage array to a second storage array, while minimizing the downtime of software applications that may affect that data. The longer the software applications have to be shut down the less productive users are. For instance, when word-processing applications or CAD applications cannot be run, output is diminished. Data can be copied at a physical layer, thereby ensuring exact data replication as well as precluding the necessity of having to match source file structures with target file structures.

DETAILED DESCRIPTION OF THE INVENTION

The present invention provides at least a new and useful method and system for migrating data from a first storage device to a second storage device in a computer-networking environment. The method of the present invention drastically reduces the time required for applications to be shut down that could possibly affect the data to be transitioned.

Acronyms and Shorthand Notations

Throughout the disclosure of the present invention, several acronyms and shorthand notations are used to aid the understanding of certain concepts pertaining to the associated system and services. These acronyms and shorthand notations are solely intended for the purpose of providing an easy methodology of communicating the ideas expressed herein and are in no way meant to limit the scope of the present invention. The following is a list of these acronyms:DIV Data-Integrity ValueDSD Data Storage DeviceLUN Logical Unit NumberPC Personal ComputerRAID Redundant Array of Inexpensive (or Independent) Disks

Further, various telecom technical terms are used throughout this disclosure. A definition of such terms can be found inNewton's Telecom Dictionaryby H. Newton, 18th Updated and Expanded Edition (2002). These definitions are intended to provide a clearer understanding of the ideas disclosed herein but are in no way intended to limit the scope of the present invention. The definitions and terms should be interpreted broadly and liberally to the extent allowed by the art and the meaning of the words offered in the above-cited reference.

As one skilled in the art will appreciate, the present invention may be embodied as, among other things: a method, system, or computer-program product. Accordingly, the present invention may take the form of a hardware embodiment, a software embodiment, or an embodiment combining software and hardware. In a preferred embodiment, the present invention takes the form of a computer-program product that includes computer-useable instructions embodied on a computer-readable medium.

Data Migration

As previously mentioned, the present invention is, among other things, a computer-program product that includes an application to migrated data from a first device to a second device. Although “server” is used herein for simplicity's sake, those skilled in the art will readily appreciate that a variety of processing components can be connected to a computer-networking switch (switch). Conventional processing components are not limited to PCs, routers, other switches, nodes, terminal devices, handheld devices etc. Multiple servers can be connected to a storage array by a switch.

FIG. 1Bdepicts a prior-art solution with a source storage array112coupled to a destination storage array122via virtual engine124. Prior to copying any data, each of the servers (116,118,120) are directed to read and write data to virtual engine124. Virtual engine124then receives or writes data to the source storage device112. This potential prior-art solution requires configuration at each of the servers. The servers are configured to read and write data to virtual engine124. Each of the servers connected to switch114are running software applications. Some of these software applications may affect data stored in source data-storage device (DSD)112. In this prior-art solution, the applications running on each of the servers (116,118, and120) must be shut down for the entirety of the copying process. When large quantities of data must be copied, terabytes and beyond, the copying process may last several hours or even days. Thus, the prior-art solution depicted inFIG. 1Bwould require the applications running on the remote servers to be shut down days, weeks, or even longer.

FIG. 2represents an exemplary operating embodiment for practicing the present invention referenced generally by the numeral210. Exemplary operating environment210includes a source DSD212, which includes a plurality of storage devices214. Exemplary storage devices could include a disk/LUN216. A LUN, or a logical unit number, can refer to a logical storage component. For example a single hard disk may be partitioned into multiple logical drives. Each logical drive would be assigned an identifier, which in this example could be a LUN.

Source DSD212is coupled to a target DSD218by way of a switch224. Switch224is conventional in nature, allowing multiple servers such as servers230,232, and234to use a common storage array such as source DSD212. Target DSD218also includes a group of data components220, which can be similar to the set of storage components214on source DSD212. A data-migration component226, which includes an application228for facilitating data copy, is coupled to switch224. Because data-migration component226is coupled to switch224, data-migration component226is exposed to the data on both source DSD212and target DSD218.

Generally, the present invention copies data stored on source DSD212to target DSD218while the applications running on server230,232and234are running. After an initial copy sequence, the applications on the servers are shut down and a final data replication process is conducted, whereby only the changed data units from source DSD212are copied to target DSD218. This process, however, can be accomplished in a variety of ways. Those skilled in the relevant art will appreciate a variety of embodiments that do not depart from the spirit and scope of the present invention. A first exemplary embodiment is illustrated inFIG. 3.

Turning now toFIG. 3, an exemplary embodiment illustrating the functionality carried out by the present invention is shown. In an exemplary embodiment application228, running on data-migration component226, carries out the process ofFIG. 3. Data-migration component226can be a conventional personal computer, server, or network of computing devices. At a step310a cluster of data is read from source DSD212. A cluster is an exemplary data bin for storing data. Alternative data bins include sectors of prescribed sizes of data. A disk cluster is but one example of a suitable data bin. As used herein, the term “data bin” refers to a portion of storage media within which data is stored. Data is typically stored in clusters on hard disks. Data embodied on alternative computer-storage media listed above can also be copied using the present invention.

At a step312, application228computes a first check parameter associated with a cluster of data read. A checksum value is one exemplary data-integrity value (DIV). A DIV is a value that is associated with each data bin, in this case cluster, of data. If optical storage media were being copied from, the corresponding data bins would be accessed. A data-integrity value is a value that is used to insure the integrity of data copied from source DSD212to target DSD218. In a preferred embodiment the DIV is computed using a cyclical-redundancy-check (CRC) algorithm.

A CRC algorithm is a conventional technique used to obtain data reliability. Those skilled in the relevant art will be familiar with computing, deriving, extracting and storing CRC values. Typically the CRC technique is used in conjunction with blocks of data. As previously mentioned, clusters can be blocks of data. Typically, a CRC program calculates a CRC value, or checksum, for the data blocks specified, which in this case could be disk clusters. The CRC program performs a calculation on portions of the file, generating a unique number for the portion of data in question. If the file or portion of data is changed at all, even a single byte, the CRC value for that portion of data would also change. Accordingly the CRC values for identical files should be identical. Those skilled in the art will recognize the many flavors of CRC checking and their equivalents, all of which are contemplated within the scope of the present invention.

At a step314, the checksum parameter associated with the cluster just read from source DSD212is stored. That cluster of data is written to target DSD218at a step316. This process continues until all source clusters of source DSD212have been read and written to target DSD218. At a step318, a check is performed to determine whether there are any additional clusters to be read from source DSD212. If not, processing continues to step320where the data on source DSD is locked down.

One method of locking down data, or preventing changes to that data, includes shutting down any application that may change the data stored in source DSD112. With the data on source DSD212locked down, application228identifies any source clusters that are not identical to their corresponding written clusters at a step322. Those skilled in the art will appreciate that this identification step can be carried out in a variety of ways. In one embodiment, a second checksum parameter can be computed for each cluster of the locked down source DSD212. Each of the second checksum parameters can be compared to the checksum parameter stored in step314to determine whether a disparity exists at a step322B. At a step322C, if the second checksum does not equal the first checksum, then a log of the respective source cluster can be created. In an alternative embodiment, that source cluster could be immediately copied. Alternatively, a log could be created that includes the source clusters that have changed.

At a step324, a determination is made as to whether any disparities exist between any of the clusters from the original source DSD212and the clusters subsequent to its lockdown. If there are disparities, then application228references at a step326a log of changed clusters created in step322C. Application228then recopies the identified source clusters to the target DSD218at a step328. Copying at the cluster level does not require application228to actually know what data was changed. Rather, the entire cluster will be recopied from source DSD212to target DSD218. This would be analogous to having a first box of articles, receive a notification that a change to one of the articles has occurred, and then instead of attempting to replace the articles, replacing the changed box with a box identical to the original. Files do not have to be copied as files. Rather, the various clusters of the various disks or LUNs on source DSD212are replicated.

At an optional step330, the first set of checksums originally created at step312can be updated with the checksums generated at step322C. This final list of checksums can then be benchmarked against a set of checksums derived from the data written to target DSD218to confirm that the data written to target DSD218is identical to the source data216. If any checksums do not match, then the respective source clusters can be respectively rewritten to the target DSD218.

FIG. 4represents an alternative embodiment of a method for practicing the present invention. At a step410, data-replication application228is initiated. At a step412, a series of events takes place that are illustrated in greater detail than inFIG. 3. As shown inFIG. 4, an exemplary set of source clusters412A is provided. This data is read and respective DIVs are created for each data bin read. These DIVs can optionally be stored in a first array412B as shown. The DIV for data bin A is Xa, stored in position DIV_0. Similarly, each other respective DIV is stored in the first array412B and associated with its corresponding data bin. The data read from source DSD212is then written to a target device, such as target DSD218.

During the writing process, application228can monitor source data stored in source DSD212and note whether data in any source clusters are modified. If data within a source data cluster is modified, those changed source clusters can be logged in a second array412C. In a step412, it can be seen that the data in clusters1and5were hypothetically modified and stored in second array412C. The source data is locked down at a step414. After having replicated a first copy of the data on source DSD212, application228can reference second array412C to determine which source clusters to re-replicate at a step416. At a step418, application228rereads the changed clusters listed in second array412C, which here are1and5. At a step420the changed clusters are copied and the respective DIVs can optionally be recomputed. In this example, a DIV for cluster1and for cluster5would be regenerated. The new DIVs, which reflect the changed source data, can be integrated into first array412B.

Thus, at an optional step422the data written to target DSD218can be confirmed as identical to the data from source DSD212. One exemplary method for carrying out this optional step is to compute DIVs for each cluster on target DSD and compare those with the DIVs stored in first array412B. The DIVs computed and compared with those in first array412B should be identical. If a disparity is found, then the respective source cluster can be recopied to the target cluster.

In still another embodiment,FIG. 5provides an alternative illustration of executing a method performed by the present invention. At a step510, application228is initiated. At a step512a series of steps are accomplished, the details of which are discussed with reference toFIG. 5. Source data is read from the clusters of source DSD212. Data-integrity values are generated for each cluster, or other data bin, read and stored in an array512A. The data read from source DSD212is then written to target DSD218. In this embodiment, source data from source DSD212is not monitored, but may be modified by applications running on servers230,232and234during the writing process. However, at a step514, after a first pass of copying data to target DSD218, the source data on source DSD212is locked down.

At a step516, application228can then identify which, if any, clusters on source DSD212changed during the writing process. Those skilled in the art will appreciate the myriad of ways of determining whether source data was modified during the writing process. An exemplary method for determining which clusters on source DSD212changed during the writing process would include rereading each source cluster, computing a second DIV, comparing this DIV with the respective DIV stored in array512A and determining whether there was a difference. If the two DIVs did not match, then the source data must have changed during the writing process. The source data cluster can immediately be rewritten to target DSD218, or included in an array for future reference. As shown, cluster representation516A depicts that clusters1and5contain data that changed during the writing process. Thus, at a step518clusters1and5would be recopied from source DSD212to target DSD218. As previously mentioned, it is not necessary to know that the data within clusters1and5, for example “B” and “F,” changed to “Y” and “Z” respectively. The present invention need only know that data within clusters1and5changed. The entire cluster is preferably recopied. If the source clusters were not individually and immediately copied, then the array storing the source cluster numbers could be referenced to rewrite the source clusters all at once to target DSD218. At an optional step520, the data on target DSD218can be confirmed as identical to data on source DSD212as of lockdown. An exemplary method for confirming that the two data sets are identical would be to compute individual checksums for each cluster of data on target DSD218and compare those checksums with the values stored in first array512A. For each source data cluster that does not match the corresponding target data cluster, that source cluster can be recopied to target DSD218.

Those skilled in the art will appreciate alternative means of implementing the described methods and processes for copying data from a source location to a target location without having to shut down applications that may modify that data during the entirety of a copying process. Those skilled in the art of computer programming appreciate that programs can be written in a variety of languages and a variety of means and in a variety of ways. The aforementioned disclosure should not be interpreted as limited to the specific order of the steps in the various process-flow diagrams.

As can be seen, the present invention and its equivalents are well-adapted to provide a new and useful method for retrieving information from network components. Many different arrangements of the various components depicted, as well as components not shown, are possible without departing from the spirit and scope of the present invention.

The present invention has been described in relation to particular embodiments, which are intended in all respects to be illustrative rather than restrictive. Alternative embodiments will become apparent to those skilled in the art that do not depart from its scope. Many alternative embodiments exist but are not included because of the nature of this invention. A skilled programmer may develop alternative means of implementing the aforementioned improvements without departing from the scope of the present invention.