Abstract:
A technique for restoring file systems by applying full and incremental backups together while streaming them from multiple devices provides a faster system restore from a full backup where incremental backups must also be applied. A first storage device streams the full backup while at least one second storage device streams at least one incremental backup. As files are detected in the incremental backup, they are written instead of the corresponding file in the full backup. Incremental backups can be pre-merged to reduce their number to one less than a number of storage devices available to stream the backups, so that the full backup can be streamed along with the pre-merged incremental backup(s) to restore the file system.

Description:
BACKGROUND OF THE INVENTION 
     1. Field of the Invention 
     The present invention is related to storage file systems within computer systems, and more specifically to a system restore methodology that restores full and incremental backups using multiple simultaneous device streams. 
     2. Description of Related Art 
     In large-scale computer systems, after a catastrophic event such as a total system failure or when a software upgrade must be backed-out due to problematic changes to the operating system or other components of the software, a full file system restore may need to be performed. When a full system backup is available, the process of performing a full system restore is typically streamlined by writing large sequential chunks of restore data to the storage containing the file system to be restored. If the backup is be partitioned, e.g., into multiple independent volumes, the restore image can be written simultaneously from several backup streams, which are generally provided from slower devices such as magnetic tape storage devices. Since the backup devices are typically the limiting bandwidth for the restore process, using a larger number of backup devices reduces the time required to restore the file system. 
     However, in a typical system backup management scheme, a full backup is performed at predetermined intervals, e.g., weekly intervals, and incremental backups are performed more frequently, e.g., at the end of every business day or on an even shorter interval. If the backup being restored requires several incremental updates, then the restore process can be slowed significantly by having to apply the full system backup and then each incremental backup. Further, a full system backup with incremental backups are not typically restored from multiple device streams, since the incremental backups take precedence over the full backup and any earlier incremental backups. 
     Therefore, it would be desirable to provide for restoring full and incremental system backups together using multiple device streams. 
     BRIEF SUMMARY OF THE INVENTION 
     The invention is embodied in a computer program product and computer system that provides for restore from full and incremental backups from multiple device streams when restoring a file system to a target storage within a computer system. 
     The computer program product and computer system copy files to restore a file system from multiple restore devices and streams incremental backups along with the full system backups by reading the full backup from a first storage device and while the full backup is being read, also reading at least one incremental backup from a corresponding at least one other storage device. Files read from the full backup are selectively not written from the full backup, but are written to the target storage from the incremental backup as the files are found in the incremental backup. 
     If insufficient storage devices are available to restore all of the incremental backups along with the full backup, then incremental backups can be pre-merged to a temporary storage to reduce the number of incremental backups that are required. The pre-merging process can be repeated until the number of merged incremental backups is one less than the number of available storage devices for reading the backups, so that the pre-merged backups and the full backup can be streamed to restore the file system. The pre-merging can be performed as part of the backup or part of the restore processing. 
     The foregoing and other objectives, features, and advantages of the invention will be apparent from the following, more particular, description of the preferred embodiment of the invention, as illustrated in the accompanying drawings. 
    
    
     
       BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWING 
       The novel features believed characteristic of the invention are set forth in the appended claims. The invention itself, however, as well as a preferred mode of use, further objectives, and advantages thereof, will best be understood by reference to the following detailed description of the invention when read in conjunction with the accompanying Figures, wherein like reference numerals indicate like components, and: 
         FIG. 1  is a block diagram illustrating a networked computer system in which techniques according to an embodiment of the present invention are practiced. 
         FIG. 2  is a flow chart of a file system restore method in accordance with an embodiment of the present invention. 
         FIG. 3  is a flow chart of a merge operation used in the file system restore algorithm depicted in  FIG. 2 . 
     
    
    
     DETAILED DESCRIPTION OF THE INVENTION 
     The present invention relates to computer backup and restore systems, and in particular to file system restore operations performed from multiple streaming devices in which a full backup and one or more incremental backups are applied in a single operation. The backup images are ordered and the incremental and full backup images are merged on the fly, so that the images can be streamed from multiple restore devices without requiring successive application of the incremental backups after the full backup has been restored, as is typically performed in file system restore operations. If the number of incremental backups combined with the full backup exceeds the number of streaming devices available, then multiple incremental backups are pre-merged to a single backup image on a temporary storage and then the merged image is further merged with the full backup and any remaining incremental backups. 
     Referring now to  FIG. 1 , a networked computer system in which an embodiment of the present invention is practiced is depicted in a block diagram. A workstation computer system  10  includes a processor CPU coupled to a memory MEM that contains program instructions for execution by CPU, including a virtual file system (VFS) interface  11 A, which provides a native file system interface to the particular operating system executed by workstation computer system  10 , for example the WINDOWS operating system. Workstation computer  10  is also depicted as including a graphical display Display and input devices Input Devices, such as mice and keyboards, for interacting with user interfaces including login screens and other user interfaces for interacting with other computers connected to the network, for example, administration screens for administering selection of the full and incremental backups to apply, and to control or edit scheduling of the backup processes and initiate the restore processes of the present invention. Workstation computer system also includes a hard disc controller HDC  14  that interfaces processor CPU to local storage device  17 A and a network interface that couples workstation computer system  10 A to network  15 , which may be fully wireless, fully wired or any type of hybrid network. VFS interface  11 A provides a uniform set of application programming interfaces (APIs) that provide access to resources, such as local storage  17 A or remote storage such as storage devices  17 B and  17 C, which are coupled to network  15  by network disc controller (NWDC)  18 . 
     A server rack computer system  12 , having at least one server memory SMEM and a server processor SCPU is also shown coupled to network  15 . A different VFS client  11 B is provided and executed within workstation computer system  10 B to provide suitable native APIs for accessing storage within server rack  12 , networked storage devices  17 B and  17 C, as well as storage device  17 A within workstation computer system  10 , if storage device  17 A is shared. A system configuration as is generally used with the present invention will include a large number of workstation computer systems such as workstation  10  and a large number of server racks such a server rack  12 . Storage within such a computer system will also generally include large arrays of storage devices. However, the techniques of the present invention are not dependent on scale and therefore can be practiced within smaller system configurations, as well. A number of backup and restore devices  19 A- 19 D, e.g., magnetic tape drives, are coupled to network  15  for generating and reading multiple backup/restore images on media MEDIA, such as backup tapes, as will be described in further detail below. The backup operations of the present invention can be controlled by either or both of VFS client  11 A and  11 B or another process executing within server rack  12  workstation computer system  10  or another system coupled to network  15  including processes executing within controllers within networked storage devices  17 B and  17 C. In general, the present invention reads images from backup storage such as media MEDIA that is installed in one or more restore devices such as devices  19 A- 19 D, and then restores a file system from the backup images onto a target storage such as networked storage devices  17 B and  17 C. 
     Network  15  may include wireless local area networks (WLANs), wired local-area networks (LANs), wide-area networks (WANs) or any other suitable interconnection that provides communication between workstation computer system  10  and server rack  12 , storage devices  17 A- 17 C, and any other systems and devices coupled to network  15 . Further, the present invention concerns backup and restore functionality that is not limited to a specific computer system or network configuration. Finally, the specification of workstation computer system  10  and server rack  12  and the location of their specific memory MEM and file system interface objects  11 A and  11 B does not imply a specific client-server relationship or hierarchical organization, as the techniques of the present invention may be employed in distributed systems in which no particular machine is identified as a server, but at least one of the machines provides an instance and functionality of a program or object that performs backup and restore operations in accordance with an embodiment of the present invention. The objects or interfaces process accesses according to methods and structures of the present invention, as described in further detail below. 
     Referring now to  FIG. 2 , a file system restore method in accordance with an embodiment of the present invention is illustrated in a flowchart. In the depicted method, first the number of incremental backups N to be applied is determined (step  30 ) and the number M of available restore devices is also determined (step  31 ). If N+1&gt;M, i.e. the number of incremental backups N incremented to include the full backup is greater than the number M of available restore devices (decision  33 ), then multiple incremental backups are pre-merged to a temporary storage location to reduce the number of incremental backups (step  34 ). Step  34  and decision  33  are repeated until the remaining incremental backups and the full backup can be simultaneously read by the available restored devices. Then, the full and incremental backups are restored by merging the full and incremental backup streams. 
     The temporary image generated by the pre-merging operations contains at most one copy of each file. The total amount of data in the temporary image cannot exceed the total amount of data to be restored. Therefore, there will be sufficient space in the storage allocated for the file system to hold the result of each pre-merge step. The pre-merge operations can be performed on any subset of the incremental backups, and there is an advantage in pre-merging the smallest incremental backups, so that the temporary image is as small as possible. When performing multiple pre-merge steps, the disk space occupied by the temporary image from the previous pre-merge step can be freed as data is being copied to the final location of the file system, so that the total of the temporary image and the restored portions of the file system are still guaranteed to fit in the allocated space. Similarly, during the final merge and restore from the full backup, the temporary image and any additional incremental backups, the disk space in the temporary file from the last pre-merge operation can be freed as its data is being read during the copy operations. Some operating systems such as AIX support a system call (e.g., fclear) that frees disk space within an existing file as it is no longer needed. In systems that do not support such operations, during restore operations where remaining storage space becomes a critical factor, the temporary image generated by the pre-merge is broken up into a sequence of smaller temporary files, so that the smaller files can be deleted as their data is no longer needed. 
     In one embodiment of the invention, the pre-merge operation(s) always merges the newest incremental backups first. Since the files in the temporary image will therefore be more recent than the files in all of the other incremental backups and the full backup, the temporary image may be stored directly in the target storage for the file system as the final data. An abbreviated summary of the temporary image, e.g., a list of files in the temporary image, may be generated to list the restored files without their data or attributes. Subsequent pre-merge operations, as well as the final merge with the full backup can then refer to the abbreviated summary to determine if more recent data has already been restored. The abbreviated summary may be updated at each pre-merge operation if multiple pre-merge operations are performed using the most recent incremental backups at each pass. 
     Referring now to  FIG. 3 , a merge operation as used in the algorithm of  FIG. 2  is shown. Once file information is streamed from the full backup image (step  40 ), if file info from the same file has been streamed from one or more incremental backup images (decision  42 ), then the file will be restored from the newest incremental backup containing the file (step  46 ) and the remainder of the file in the full backup can be skipped (step  48 ). Otherwise if the file was only present in the full backup image, then the file is restored from the full backup image (step  44 ). 
     The backup images are similar to backup images generally used by backup/restore processes, but the files are generally arranged sequentially on the backup media in a canonical order so that the incremental backup(s) can be streamed along with the full backups, to minimize the required restore time. In order to accomplish the ordering, the files are generally ordered by inode or alphabetically according to their full directory path. When reading the media, the media is read far enough ahead to determine the file information, e.g., file name or inode number, so that a comparison can be made to determine the newest incremental backup containing the file, without interrupting streaming. When using a summary of a merged stream as described above, the summary need only contain the file identifiers. 
     While the invention has been particularly shown and described with reference to the preferred embodiments thereof, it will be understood by those skilled in the art that the foregoing and other changes in form, and details may be made therein without departing from the spirit and scope of the invention.