Optimization to permit block based incremental backup across system reboot or crash

Techniques to back up data are disclosed. In various embodiments, a copy of a free block map as of a first time associated with a first backup is stored in persistent data storage. Writes made subsequent to the first backup to blocks not listed as free in the copy of the free block map as of the first time are tracked in a persistently-stored change block tracking log. A free block map as of a second time and the previously-stored copy of the free block map as of the first time are used to determine which blocks listed as free in the free block map as of the first time have been written to since the first time. At least a subset of blocks determined to have been written to since the first time are including in an incremental backup.

BACKGROUND OF THE INVENTION

In prior approaches to block based incremental backup a trade-off typically had to be made between tracking and logging in a persistently-stored change block log all blocks modified since a previous backup, which would enable incremental backups to be performed even after system reboot and/or crash but which consumed resources (e.g., time, storage) and potentially could impact performance, and not tracking changed blocks in a persistently-stored log, which resulted in a full backup operation being required in the event of a system reboot and/or crash.

DETAILED DESCRIPTION

Using a persistently stored map of blocks that were free at a time associated with a prior backup to perform a subsequent incremental block based backup is disclosed. In various embodiments, at a time a full and/or incremental backup is performed, a copy of a free block map (or other data structure) as of the time of the backup is extracted from a file system with which the backup is associated. The extracted copy of the free block map is stored persistently for future use to perform a subsequent incremental backup. In various embodiments, blocks that are subsequently modified are tracked in a persistently-stored change block log only if they were not in the previously-stored free block map. For example, all such blocks in some embodiments are pre-marked as “modified” in an in-memory change block tracking log. At the time of a subsequent incremental backup, the persistently-stored copy of the free block map as of the last backup is compared to a then-current free block map. Blocks that are determined based on the comparison to have been free as of the prior backup but no longer are listed as free as of the current backup are included in the incremental backup.

FIG. 1is a block diagram illustrating an embodiment of a backup system and environment. In the example shown, a protected system102, e.g., a file server, networked client computer system, etc., is connected via a network104, e.g., a local area network (LAN), storage area network (SAN), the Internet, etc., to a backup server106configured to store backup data in a backup media108, such as optical or other disk drives, magnetic tapes, etc. In some embodiments, a backup client or other backup agent running on protected system102coordinates the backup of data from protected system102. For example, the backup client may cooperate with backup server106to store on backup media108a backup copy of data that resides on protected system102. In various embodiments, a block-based backup may be performed. In a block-based backup, blocks of data as stored on a storage drive of a protected system, such as protected system102, may be transferred to and stored in backup media, such as backup media108, at the block level, as opposed to as files or other logical objects.

In some embodiments, an incremental block-based backup may be performed. For example, the Networker® backup solution provided by EMC® Corporation includes a change block tracking (CBT) driver which can be used to track which blocks have been written to since a last backup, such as a last full backup. When an incremental backup is to be performed, only those blocks that have been changed since the last backup are included in the incremental backup.

FIG. 2is a block diagram illustrating an embodiment of a system to perform incremental block based backup. In the example shown, the protected system102ofFIG. 1includes one or more application(s) configured to use a local file system204to store application objects, such as files, persistently on a local disk (or other non-volatile storage) drive206. A change block tracking (CBT) driver208has been integrated with the operating system210in this example to intercept writes to storage blocks on drive206. If writes relate to a protected volume, for example, in various embodiments the changed blocks may be logged in an in-memory change block log stored in memory212. In various embodiments, as described herein, changed blocks added to the in-memory log may be propagated to a corresponding persistently-stored change block log (or other persistently-stored data structure) stored on drive206.

A backup client214, e.g., a Networker® backup client/agent, is installed on the protected system102and manages backup operations on protected system102. For example, the backup client214may perform an incremental backup by transmitting to a remote backup server, via a communication interface216and associated network connection218, data stored in blocks changed since a last backup.

In various embodiments, free block maps maintained by a file system with which a protected volume or other set of stored data is associated are used to reduce the number of changed blocks that have to be tracked during file system operations that occur between backups, e.g., as explained and illustrated below with reference toFIGS. 3 through 6.

FIG. 3is a flow chart illustrating an embodiment of a process to perform a full backup. In the example shown, an indication is received to perform a full backup (302). A copy of a free block map for the protected volume, LUN, or other data set is extracted from a file system with which the protected data is associated and is stored persistently, e.g., on a local disk or other drive (304). A full backup is performed (306).

FIG. 4is a flow chart illustrating an embodiment of a process to track changed blocks. The process ofFIG. 4may begin, for example, at the conclusion of a full (or other) backup, e.g., as inFIG. 3. Blocks indicated as “free” as of the time of the immediately prior backup are pre-marked as “modified” in an in-memory change block log (402). Subsequently, for each sector/block write operation, the in-memory log is checked to determine whether the sector/block has already been marked as “modified” (404). If so, nothing further is done and processing continues with the next write operation (if any) (408). If the sector/block that has been written to has not yet been marked as “modified” (406), the in-memory log is updated to reflect that the sector/block has been modified and the update is propagated to the persistently-stored change block log (410). Note that since blocks list as “free” on the previously-stored copy of the free block map as of the time of the last backup were pre-marked as “modified” in the in-memory log (402), writes to free blocks that occur subsequently to a free block or range of blocks will not, in various embodiments, result in the in-memory log being updated with data subsequently propagated to the persistently-stored log (see, e.g.,406,410) at the time such a write operation is intercepted by the CBT driver or other entity. Instead, in various embodiments, only writes to sectors/blocks that were not “free” as of the last backup will be detected and logged in both the in-memory and persistently-stored logs.

FIG. 5is a flow chart illustrating an embodiment of a process to perform an incremental backup. An indication to perform an incremental backup is received (502). A current (i.e., as of the time of the current backup) free block map is extracted from the file system and compared to the free block map that was extracted from the file system and stored persistently at the time of the last (e.g., full) backup (504). Non-intersecting blocks, i.e., those that were listed as free in the copy of the free block maps as of the time of the last backup but are no longer listed as free in the current free block map, are added to the persistently-stored change block log (506). In this single operation, all previously “free” blocks that have been written to since the last backup become added to the persistently-stored change block log, without requiring such writes to have been reflected in the persistent log as such writes were intercepted. The persistently-stored change block log is used to perform the incremental backup (508). For example, blocks indicated in persistently-stored change block log as having been modified since the last backup, which include those blocks marked as “modified” as changes to blocks that were not previously “free” were intercepted and those blocks that were listed as “free” as of the last backup but are no longer listed as “free” as of the current backup, will be backed up in various embodiments. The in-memory and persistent change block logs are reset (510). The current free block map is used to pre-mark free blocks as “modified” in the in-memory log (512) (as in step402ofFIG. 4), and the current free block map is stored persistently (514) (as in step304ofFIG. 3) for future use, e.g., in connection with a next incremental backup.

FIG. 6is a flow chart illustrating an embodiment of a process to restore an in-memory change block log after system crash and/or reboot. In the example shown, in the event of a system crash and/or reboot (602), the copy of the free block map stored most recently in persistent storage, e.g., in connection with the most recent backup is read from the persistent storage (604) and used to pre-mark as “modified” in the in-memory change block tracking log (once again, since the same operation would have been performed in connection with the last backup, as in step402ofFIG. 4and/or step512ofFIG. 5) those blocks listed as being “free” in the map (606). Note that in various embodiments writes to blocks other than those listed as free in the persistently-stored copy of the free block map as of the last backup would have been stored in the in-memory log, which would be lost in the event of crash and/or reboot, but also in the persistent log, as in step410ofFIG. 4. As a result, using the previously-stored free block map subsequent to a system crash and/or reboot to pre-mark as “modified” those blocks listed as “free” in the map enables the techniques disclosed herein to be used to perform an incremental block-based backup reliably even after a system crash and/or reboot, without having to track in the persistent change block log at the time they are made writes to blocks that were free until that time.

Using techniques disclosed herein, block-based incremental backups may be performed, including after system crash and/or reboot, without having to incur the cost of tracking in a persistently-stored change block log as they occur all writes since a last backup.