Block level backup of virtual machines for file name level based file search and restoration

A method for backing a computing system includes generating a local history table of filesystem objects that have changed since storing a first backup of the computing system where the local history table includes attributes of the filesystem objects and a set of operations associated with changes to the filesystem objects. The method further includes transferring, from the computing system to a backup server, a second backup of the computing system, where the second backup includes a block level representation of a portion of a data storage medium associated with the computing system. The method additionally includes merging the local history table with a global history table stored on the backup server, the global history table mapping a history of filesystem objects to a set of block level backups of the computing system including the first backup of the computing system and the second backup of the computing system.

BACKGROUND

The present disclosure relates to computer data storage systems, and more specifically, to generating block level backups of virtual machines to enable file name level based searching for, and restoration of, files in the block level backups.

SUMMARY

According to embodiments of the present disclosure, a method for backing up a computing system includes generating a local history table of filesystem objects that have changed since storing a first backup of the computing system. The local history table includes attributes of the filesystem objects and a set of operations associated with changes to the filesystem objects. The method further includes transferring, from the computing system to a backup server, a second backup of the computing system, where the second backup includes a block level representation of at least a portion of a data storage medium associated with the computing system including the changed filesystem objects. The method additionally includes merging the local history table with a global history table stored on the backup server, the global history table mapping a history of filesystem objects to a set of bock level backups of the computing system including the first backup of the computing system and the second backup of the computing system.

Other embodiments are directed to a computer system and a computer program product.

DETAILED DESCRIPTION

Aspects of the present disclosure relate to computer data storage systems, and more particular aspects relate to generating block level backups of virtual machines to enable file name level based searching for, and restoration of, files in the block level backups. While the present disclosure is not necessarily limited to such applications, various aspects of the disclosure may be appreciated through a discussion of various examples using this context.

Virtualization enables computing entities to efficiently utilize the resources of powerful computer servers to provide disparate computing environments to customers. Computing entities, for example, may deploy several different operating systems over a physical or hardware layer made up of, for example, a homogenous set of computing servers. The operating systems may be executed in virtual machines managed by a virtualization manager deployed in a virtualization layer between the operating systems and the hardware layer. The virtualization manager may be configured to apportion the resources of computing hardware between the virtual machines. Each virtual machine, for example, may be allocated a datastore to serve as a virtual disk on which the operating systems executing within the virtual machines may create partition and construct filesystems. Virtual disks, like physical disks, may be backed-up periodically to enable recover from system or data integrity failures.

Backup applications executing on computing systems may manage the creation of periodic backups of storage devices allocated to the computing system. The backup applications may execute within the computing systems to identify changed filesystem objects (e.g., files and directories) or disk regions, and to periodically execute operations to copy the changes to a backup server. Computing systems configured to virtualize operating systems and other computing environments may deploy (e.g., install and execute) backup applications within virtualization layers to backup the datastores of virtual machines executing on computing systems. This architecture may reduce the overhead that may be incurred by deploying backup applications in each individual virtual machine. The backup applications may create block level backups that are agnostic of datastores partitions and filesystems. These block level backups may be based on changed blocks of disk drives or datastores. Restoring individual filesystem objects from backup servers storing multiple block level backups may require retrieving each block level backup stored on backup server, mounting each of the retrieved block level backups, reconstructing partitions and filesystems, and then searching the reconstructed partitions and filesystems to identify individual filesystem objects for restoration. Executing these operations to restore a single filesystem object may be both time and resource intensive when multiple block level backups are stored on a backup server.

Embodiments of the present disclosure are based on the recognition that restoring individual file system objects from block level backups of virtual machines may be improved by monitoring and tracking filesystem object creation and modification in a global history table available to backup applications and backup servers. Accordingly, embodiments of the present disclosure are directed towards an architecture (e.g., including systems, methods, and computer program products) for generating and maintaining filesystem object history information as part of block level backups of virtual machines. The architecture includes generating a local history table for tracking filesystem objects that have changed since a last backup of a virtual machine. The local history table may include the changed objects' attributes as well as indications of the type of changes executed on the objects (e.g., the operations executed on the object to case the change). As part of a block level backup process, the local history table may be merged with a global history table on the backup server, where the global history table is configured to track the history of all filesystem objects and their associations to the sets of block level backups of a virtual machine. According to the architecture, a user may identify a block level backup having a version of the filesystem object (e.g., a file or a directory) he/she wishes to restore without having to first mount any of the backups. The filesystem object may instead be identified by searching the global history table. The backup containing the correct version of the filesystem object can then be mounted, and the target filesystem object can be retrieved.

Characteristics are as follows:

Service Models are as follows:

Deployment Models are as follows:

Hardware and software layer60includes hardware and software components. Examples of hardware components include: mainframes; RISC (Reduced Instruction Set Computer) architecture based servers; storage devices; networks and networking components. In some embodiments, software components include network application server software.

As used here, creating a block level backup of virtual machine includes the process of identifying blocks that have changed since a previous backup of the virtual machine (e.g., a first backup) or before the current backup when the current backup is the first backup of the virtual machine. The process may further include copying and/or transmitting the identified changed blocks to a backup server for storage.

FIG. 4depicts a flowchart400of computer implemented operations for backing-up a virtual machine, according to various embodiments. In some embodiments, some or all of the computer implemented operations may be executed by a backup application deployed in the virtualization layer of a computing system such as virtualization layer62(FIG. 3) and Backup Applications816(FIG. 8), or in operating systems installed within a virtual machine. In other embodiments, some or all of the computer implemented operations may be executed by backup software applications executing on a backup server such as the backup server825(FIG. 8). Backup software applications executing may include components deployed within the virtualization layers of computing systems being backed-up to facilitate execution of the computer implemented operations. Collectively, backup software applications, whether executing within virtualization layers and on virtualized operating systems on the computing system being backed up, or on a remote backup server, in combination with supporting hardware such as computing systems and data storage volumes, will be referred to herein as a “backup system”. A backup system may execute the computer implemented operations described herein in the time before a first backup of a virtual machine, or in the time between a first and a second backup of a virtual machine. For clarity, the time period in either scenario will be referred to herein as occurring subsequent to a first backup of a virtual machine and prior to a second backup of a virtual machine.

The backup system may execute operation405by generating a local history table to track changes to filesystem objects in the period between virtual machine backups. In some embodiments, the local history table may be stored within a virtual machine executing on the computing system or cloud computing platform. The local history table may include information about filesystem objects that have changed since the last backup of a virtual machine. The information may include attributes of the filesystem objects such as the name of the object, the type of object, a unique identifier (ID) of the object, and a unique identifier of the object's parent object. The object name may be a sequence of characters (e.g., a string) assigned by, for example, a user to a filesystem object. A filesystem object name, for example, may be the name of a file or the name of a directory existing in the virtual machine. In some embodiments, the object name may not be unique. In these situations, an object may be further uniquely identified based on other attributes, including the object identifier and its parent object name and or identifier. An object identifier may be a unique sequence of characters, symbols, and/or logical relationships that uniquely identify the object in a filesystem or other computing environment. In some embodiments, an object identifier may be assigned according to an architecture of a virtual machine or other computing system. An object type may be at least one of any class of filesystem objects. Examples of object types include files and directories. A parent of an object may be a first filesystem object which hierarchically precedes a second object in a filesystem or other computing system data structure. As an example, a directory may be considered the parent of any files stored within the directory.

The information stored in the local history table additionally includes an indication of the type of operation executed on a filesystem object to case a change in its state. In certain embodiments, the type of operation may at least one of an object create, delete, change, move, and rename operation. Other operations are possible. An object create operations includes the addition of a new object to a filesystem. The type of operation may be indicated using character strings, symbols, or other identifiers. In some embodiments, the type of operation may be indicated along with a set (e.g., one or more) of parameters associated with the operation. A filesystem object rename operation, for example, may be indicated along with the old (or previous) name of the filesystem object and the new name of the filesystem object. Similarly, an object move operation may be indicated along with a unique identifier of the new parent filesystem object of the relocated object.

In some embodiments, a virtual machine executing on a computing system may generate and maintain a local history table. Filesystem journaling mechanisms associated with the virtual machine may collect information about changed filesystem objects for populating the local history table. In other embodiments, local history tables may be maintained in the virtualization layer of a computing system by a virtualization manager. In these embodiments, information about changed filesystem objects may be collected from the virtual machines via network or inter-process communications mechanisms.

According to various embodiments, information about changed filesystem objects collected by virtual machines may be stored as entries in the rows or columns of the local history table. For the remainder of this disclosure, related data stored in tables (e.g., attribute information for a given filesystem object) will be discussed as if the related data was stored in rows of the tables. The contents of the disclosure, however, equally applies to related data stored in columns of tables. The entries of the local history table may be sorted chronologically by the date and time the changes occurred. For example, when the entries are arranged in rows, a first row in the local history table may include information about the first changed filesystem object, while an Nth or last row in the local history table may include information about the Nth or last filesystem object to change. A filesystem object may have more than one entry in the local history table. For example, if a file system object is modified and then moved, the local history table may include a first entry evidencing the modification and a second entry evidencing the move.

Referring again to the operations of the flowchart400, the backup system may execute operation410to generate a block level backup of the virtual machine. Generating the block level backup may include identifying blocks of a storage device that have changed since a previous backup of the storage device. Generating the block level backup may further include copying or duplicating the identified changed blocks (e.g., locks including changed filesystem objects). The backup system may generate the block level backup using available software applications configured to identify and duplicate changed blocks of a storage device. A software application executing in hardware and software layer60(FIG. 3), or executing in virtualization layer (62) (FIG. 3) may periodically scan the storage devices associated with a computing system hosting a virtual machine identify changed blocks. The identified changed blocks may be copied or tagged in a database for future copying. Because these software applications execute outside, and independently, of any virtual machines installed on the computing system, the block level backups may be generated without consideration to the partitioning of, or the filesystems installed on, the datastores allocated to the virtual machines.

The backup system may execute operation415to transmit the block level backup and the local history table to the backup server. In some embodiments, the backup server may be a computing system remote to, or from, the computing system hosting the virtual machine being backed-up. In these embodiments, the block level backups and local history table may be transmitted from the hosting computing system to the backup server over a data communications network or via a point-to-point data communications channel. Transmitting a block level backup to a backup server, for example, may include identifying changed data blocks (e.g., data blocks previously tagged in a table or database as described in operation410) and transmitting an image of these data blocks over a network or a point-to-point communication interface. In other embodiments, the backup server may be the same computing system as the computing system hosting the virtual machine being backed up. In these embodiments, the block level backups and local history table may be transmitted to the backup server through inter-process communication channels. For example, transmitting a block level backup to a backup server in these embodiments may include transmitting a table or database containing data blocks tagged to indicate a change since a previous backup to an application configured to process or store the backups. Similarly, transmitting the local history table to a backup server may include transmitting a pointer to, or the contents of, a file or other data structure containing the local history table to an application configured to process the local history table, as described herein.

The backup system may execute operation420to merge the local history table with a global history table associating filesystem object states or a history of filesystem objects with block level backups stored on a backup server. The global history table may be stored on the backup server in a file, database or other data structure. In some embodiments, the global history table consolidates the local history tables from multiple block level backups of a virtual machine over time.

A global history table may include information about each filesystem object having a version or copy stored in one or more block level backups on the backup server. The information stored in the global history table may include a first set of fields to store attributes of the filesystem objects, such as object names, object types, object identifiers, and identifiers of parent objects. These attributes correspond to the attributes stored in local history tables provided to backup servers. The information stored in the global history table may further include a second set of fields (e.g., reference identifier fields) to store reference identifiers for each filesystem object included in the table. Each filesystem object included in the global history table may be associated with one or more reference identifier fields. Each reference identifier field may be correspond with a different backup of a virtual machine. In some embodiments, each reference identifier field may correspond to a backup time of a virtual machine backup. A reference identifier field may be populated with a reference identifier when a block level backup associated with the reference identifier field includes a version of the filesystem object. A block level backup may be associated with a reference identifier field when the field is added to the global history table in response to receiving the block level backup. Similarly, a block level backup may be associated with a reference identifier field when the reference identifier field corresponds to the time at which the block level backup was created or was added to a backup server.

Merging the local history table with a global history table may include updating the global history table with information in the local history table according to the operations specified in the type of operation fields of the local history table, as described herein.

FIG. 5depicts a flowchart500of an example set of computer implemented operations that may be executed to both change filesystem objects, and to update a global history table from a local history table, according to various embodiments. The operations of the flowchart500may be executed by a backup system having components executing on one or more computing systems to generate and store block level backups of a virtual machine, as described herein. The operations of flowchart500are discussed in with reference to a “selected filesystem object”, a term which means each filesystem object represented in a local history table considered individually or as a set. The operations of flowchart500may be executed for each filesystem object represented in a local history table as part of, or in addition to, merge operations420(FIG. 4).

The backup system may begin the operations of flowchart500at operation504by executing one or more software routines to receive, from a data communications interface of a backup server, a block level backup and a local history table from a computing system hosting a virtual machine being backed up. Operation504may further be continued by storing the received block level backup on a storage device associated with the backup server. Examples of storage devices may include hard disk drives, solid state drives, and tape drives. The backup system may further execute operation504to append a new column of reference identifier fields corresponding to the current backup to (or insert a new column into) each row of a global history table stored on the backup server. In certain embodiments, the backup system may copy the reference identifiers corresponding to the virtual machine backup stored immediately preceding the current virtual machine backup into the new column of reference identifier fields.

The backup system may continue the operations of flowchart500by executing operation505to determine whether a selected filesystem object is a filesystem file. In some embodiments, the backup system may execute one or more software routines to retrieve the object type of the selected filesystem object from a field of the local history table to determine whether the selected filesystem object is a filesystem file. The backup system may continue to operation510when the filesystem object is a filesystem file, while the backup system may end the operations of flowchart500when the filesystem object is not a filesystem file.

At operation510, the backup system may execute one or more software routines to retrieve the operation associated with a change to the selected filesystem object from a field of the local history table to determine whether an object create operation was executed on the selected filesystem object. In some embodiments, the object create operation may be executed by a filesystem to create or add a new filesystem object to a filesystem associated with a virtual machine. The object create operation, for example, may be executed to add a new file to a filesystem. The backup system may continue to operation515when the operation associated with a change to the selected filesystem object is an object create operation, while the backup system may proceed to operation530when operation associated a with a change to the selected filesystem object is not an object create operation.

The backup system may execute operation515to copy the attributes of the selected filesystem object from the local history table to a global history table. In some embodiments, the backup system may execute operation515by inserting a new row, including a set of fields to store the attributes of a selected filesystem object, into the global history table. The storage system may then copy the selected filesystem object's attributes from the local history table to the first set of fields of the new row.

At operation520, the backup system may generate a new reference identifier for the selected filesystem object. In some embodiments, reference identifiers may numbers, characters, or symbols assigned from an ordered set or according to a mathematical or logical algorithm. Generating the new reference identifier may include selecting the next number, character, or symbol available (e.g., the next unused) in the ordered set or generated from the mathematical or logical algorithm. As an example, reference identifiers may be selected from a set of ordered integer numbers. As an example, when five filesystem objects having reference identifiers ranging from 1 to 5 in the reference identifier field for the current virtual machine backup are represented in the global history, the next available reference identifier generated by the backup system for the selected filesystem object may be 6.

At operation525, the backup system may append a new reference identifier field corresponding with the current backup to (or insert a new reference identifier field into) the new row created in operation515. The backup system may then populate the new reference field with the new reference identifier generated in operation520and proceed to operation585.

At operation530, the backup system may parse or search the global history table to find a row corresponding with the selected filesystem object. A row of the global history table may correspond with the selected filesystem object when the row contains information (e.g., filesystem attributes and reference identifiers) associated with the selected filesystem object.

The backup system may then execute operation535to determine whether an object change operation was executed on, or is associated with a change to, the selected filesystem object. An object change operation may include operations to, for example, modify the contents of filesystem file. The backup system may continue to operation540when an object change operation was executed on the selected filesystem object, while the backup system may proceed to operation550when an object change operation was not executed on the filesystem object.

At operation540, the backup system may generate a new reference identifier corresponding with the current virtual machine backup, as described herein. The new reference identifier may then be written to a reference identifier field corresponding with the current virtual machine backup in the identified row, as shown in operation545. In some embodiments, a new reference identifier field may be inserted into, or appended to, the identified row before writing the new reference identifier.

At operation550, the storage system may delete the reference identifier corresponding to the current backup from the identified row.

At operation554, the storage system may determine whether an object delete operation was executed on, or is associated with a change to, the selected filesystem object. The backup system may continue to operation585when an object delete operation was executed on the selected filesystem object, while the backup system may proceed to operation555when an object delete operation was not executed on the filesystem object.

At operation555, the storage system may determine whether an object move operation was executed on, or is associated with a change to, the selected filesystem object. An object move operation may include changing the parent filesystem object of the selected filesystem object from a first parent file system object to a second parent filesystem object. A move operation, for example, may include moving a file from a first directory to a second directory. As described herein, the local history table may include the identifier of the second or new parent object associated with the selected filesystem object when a move operation was executed on the selected filesystem object. The backup system may continue to operation560when an object move operation was executed on the selected filesystem object, while the backup system may proceed to operation575when an object move operation was not executed on the filesystem object.

At operation560, the storage system may insert a new row, including a set of fields to store the attributes of the selected filesystem object, into the global history table. The storage system may then copy the selected filesystem object's attributes from the row identified in operation530to the set of fields.

At operation565, the backup system may generate a new reference identifier corresponding with the current backup, as described herein. The backup system may append a new reference identifier field corresponding with the current virtual machine backup to the new row, as shown in operation570. The backup system may then populate the new reference identifier field with the reference identifier generated in operation565and proceed to operation585.

At operation575, the storage system may determine whether an object rename operation was executed on, or is associated with a change to, the selected filesystem object. An object rename operation may include changing the name of the selected filesystem object from a first object name to a second object name. In some embodiments, the local history table may include the new name of the selected filesystem object. The backup system may continue to operation580when an object rename operation was executed on the selected filesystem object, while the backup system may proceed to operation585when an object rename operation was not executed on the selected filesystem object.

At operation580, the storage system may insert a new row, including a set of fields to store the attributes of the selected object, into the global history table. The storage system may then copy the selected filesystem object's attributes from the row identified in operation530to the set of fields. The new object name included in the local history table for the selected filesystem object may then be written to the object name field of the new row. The storage system may then continue the operations of the flowchart500at operation565.

The storage system may end the operations flowchart500at operation585.

FIG. 6depicts a flowchart600of an example set of computer implemented operations that may be executed to both change filesystem objects, and to update a global history table from a local history table, according to various embodiments. The operations of the flowchart600may be executed by a backup system having components executing on one or more computing systems to generate block level backups of a virtual machine. The operations of flowchart600are discussed in with reference to a “selected filesystem object”, a term which means each filesystem object represented in a local history table considered individually or as a set. The operations of flowchart600may be executed for each filesystem object represented in the local history table as part of, or in addition to, merge operations420(FIG. 4).

The backup system may begin the operations of flowchart600by executing operation604in accordance with the steps described for executing of operation504(FIG. 5).

The backup system may continue the operations of flowchart600by executing operation605to determine whether a selected filesystem object is a filesystem directory. In some embodiments, the backup system may execute one or more software routines to retrieve the object type of the selected filesystem object from a field of the local history table to determine whether the object is a filesystem directory. The backup system may continue to operation610when the selected filesystem object is a filesystem directory, while the backup system may end the operations of flowchart600when the selected filesystem object is not a filesystem directory.

At operation610, the backup system may execute one or more software routines to retrieve the operation associated with a change to the selected filesystem object from a field of the local history table to determine whether an object create operation was executed on the selected filesystem object. In some embodiments, the object create operation may be executed by a filesystem to create or add a new filesystem object to the filesystem. The object create operation, for example, may be executed to add a new directory to a filesystem associated with a virtual machine. The backup system may continue to operation615when the operation associated with a change to the selected filesystem object is an object create operation, while the backup system may proceed to operation630when the operation associated with a change to the selected filesystem object is not an object create operation.

The backup system may execute operation615to insert a new row into a global history table and to copy the attributes of the selected filesystem object from the local history table to global history table in accordance with the operations described for the execution of operation515(FIG. 5).

At operation620, the backup system may generate a new reference identifier for the selected filesystem object, as described herein. At operation625, the backup system may append a new reference identifier field corresponding with the current virtual machine backup to (or insert a new reference identifier field into) the new row created in operation615. The backup system may then populate the new reference identifier field with the new reference identifier generated in operation620and proceed to operation685.

At operation630, the backup system may parse or search the global history table to find a row corresponding with the selected filesystem object.

At operation635, the backup system may delete the reference identifier corresponding to the current backup from the identified row.

At operation640, the backup system may determine whether an object delete operation was executed on, or is associated with a change to, the selected filesystem object. The backup system may continue to operation685when an object delete operation was executed on the selected filesystem object, while the backup system may proceed to operation645when an object delete operation was not executed on the filesystem object.

At operations645, the backup system may recursively identify all rows of the global history table corresponding to filesystem objects having the selected filesystem object as a parent object. Recursively identifying the rows may include finding all rows descended from the selected filesystem object. The storage system may then continue to operation650to delete the reference identifiers corresponding to the current virtual machine backup from each of the identified rows. The backup system may then proceed to operation685.

The storage may execute operation655to determine whether an object move operation was executed on, or is associated with a change to, the selected filesystem object. A move operation, for example, may include moving the selected filesystem object from a first directory to a second directory. The local history table may include the identifier of the second or new parent object associated with the selected director when the change operation is a move operation. The backup system may continue to operation660when an object move operation was executed on the selected filesystem object, while the backup system may proceed to operation675when an object move operation was not executed on the selected filesystem object.

At operation660, the storage system may insert a new row, including a set of fields to store the attributes of the selected filesystem object, into the global history table. The storage system may the copy the selected directory's attributes from the row identified in operation630to the set of fields.

At operation665, the backup system may generate a new reference identifier corresponding with the current virtual machine backup, as described herein. The backup system may append a new reference identifier field corresponding with the current virtual machine backup to the new row, as shown in operation670. The backup system may then populate the new reference identifier field with the new reference identifier generated in operation665and proceed to operation685.

At operation675, the storage system may determine whether an object rename operation was executed on, or is associated with a change to, the selected filesystem object. In some embodiments, the local history table may include a new name assigned to the selected filesystem object. The backup system may continue to operation680when an object rename operation was executed on selected filesystem object, while the backup system may proceed to operation685when an object rename operation was not executed on the selected filesystem object.

At operation680, the storage system may insert a new row, including a set of fields to store the attributes of the selected filesystem object, into the global history table. The storage system may then copy the selected filesystem object's attributes from the row identified in operation630to the set of fields. The new object name included in the local history table may then be written to the object name field of the new row. The storage system may then continue the operations of the flowchart600at operation665.

The storage system may end the operations flowchart600at operation685.

FIG. 7depicts a flowchart700of computer implemented operations for retrieving an individual filesystem object from block level backups of a virtual machine, according to various embodiments. Some or all of the operations of flowchart700may be executed by a backup system, as described herein.

The backup system may begin the operations of flowchart700by executing operation705to receive a request to access an individual filesystem object stored in a block level backup of a virtual machine. In some embodiments, the request may be received from a user, a computing device, or software and hardware component of a computing system. The request may be received over a data communication network, via inter-process communication mechanisms, and through point to point connections between computing devices. In various embodiments, the request may include a search pattern or expression (e.g., a regular expression) for identifying the requested filesystem object. The search pattern may include characters, symbols, and logical expressions which may be interpreted by a processor associated with the backup system to match one or more attributes of the requested filesystem object.

At operation710, the backup system may apply the search pattern to a global history table mapping a history of filesystem objects to block level backups of a virtual machine to identify one or more matching filesystem objects. Identifying matching filesystem objects may include identifying rows of the global history table and associated object identifiers that correspond to filesystem objects having an object identifier that logically or canonically match the search pattern.

At operation715, the backup system may identify a set of one or more block level backup having a given copy or version of the requested filesystem object. Based on the object identifier matched in operation710, the backup system may identify a row of the global history table corresponding to the requested filesystem object. The identified row may include one or more reference identifiers corresponding to one or more virtual machine backup times or block level backups. Each block level backup may include distinct copies or versions of the requested filesystem object. The backup system may provide, to a user, a list including each of backups having a version of the requested filesystem object. The list may additionally include a complete logical path to the each matching version of the requested object, the matching object's identifier, and the date and/or time each backup was created.

At operation720, the backup system may select a block level backup from the virtual machine backups identified in operation715. In some embodiments, the storage system may receive a user selection for the block level backup based on the list provided in operation715. The user selection may be received through any of the communication mechanisms described herein.

At operation725, the backup system may mount the selected block level backup to access the backed-up virtual machine's filesystem. The block level backup may be mounted according to known processes for mounting storage devices, disk volumes, and image files. Mounting the block level backup may further include reconstructing any partitions and filesystems included in the backup.

At operation730, the backup system may retrieve the requested filesystem object from the mounted block level backup. Retrieving the requested filesystem object may include searching reconstructed filesystems based on, for example, the object's identifier and/or a full filesystem path to the requested filesystem object. The full filesystem path may include the object's name or identifier, parent name or identifier, and the object name or identifiers of all objects hierarchically preceding the requested filesystem object. The retrieved filesystem object may then be provided to a user via any of the communication mechanisms described herein.

FIG. 8depicts a block diagram of an example computing environment800including systems for creating block level backups of a virtual machine and for retrieving individual filesystem objects from the backups, according to various embodiments. The example computing environment800depicts components of the backup system described herein. The computing environment may include virtual environment805, proxy server840and backup server825. Components of the computing environment may communicate using any of the data communication mechanisms described herein.

The virtual environment805may include virtual machine810A and810B, virtualization manager815and local storage820. In some embodiments, the virtual environment805corresponds with, or is an embodiment of, the cloud computing environment abstraction shown inFIG. 3. Virtual machines810A and810B may include local history table811A and811B along with virtualized hardware such as a datastore for constructing partitions and filesystems, as described herein. Virtual machine810A and810B may further include operating systems, software application, and application programmer interface mechanisms (APIs) configured to identify and track changed filesystem objects, and to store information about these objects to in local history table811A and811B. The software applications and APIs may be further configured to provide the information about the changed objects and/or the local history table to a backup storage system as described herein.

Virtualization manager815may include software and/or hardware for managing the virtualization and allocation of resources to virtual machines810A and810B. Virtualization manager815may include backup application816. In some embodiments, the backup application816may be deployed in virtual machines810A and810B. The combination of backup application816, local history table811A and811B, and the object change tracking components associated with virtual machines810A and810B may form an object change tracking module configured to extract information about changed filesystem objects that are eligible for backup. In some embodiments, virtualization manager815may maintain include local history tables811A and811B.

Storage820serve as the storage volume for the data stores allocated to virtual machines810A and810B. Changed blocks from storage820may be periodically identified for backup by a backup system as described herein.

Backup server825may be a computing system for storing block level backups of the datastores allocated to virtual machines810A and810B. In some embodiments, backup server825may be a computing system different from, or remote to, the computing system hosting virtual environment805. In other embodiments, the backup server may be a component of the computing system hosting virtual environment805. Backup server825may include a server manager830, global history table832, and a storage835. The server manager830may be a software application configured to maintain the global history table832, as described herein. The storage850may be any data storage volume configured to store block level backups of virtual machines810A and810B, as described herein. The server manager830may receive local history table811A and811B along with one or more change blocks from virtual environment805. The server manager may then store the changed blocks in storage835, and merge the local history table811A and811B with the global history table832, as described herein. In some embodiments, the server manager830may process requests to access individual filesystem objects, as described herein.

Proxy server840may include backup manager845and virtual storage850. The backup manager845may serve as an intermediary between virtual environment805and backup server825. The backup manager845may initiate periodic backup operations. In some embodiments, the backup manager845may additionally serve as a proxy for processing filesystem object access requests by receiving block level backups from backup server825and mounting them to virtual storage850. In some embodiments the components of proxy server840may be included in backup server825.

FIG. 9illustrates an example local history table for tacking a history of filesystem objects, according to various embodiments. The example local history table includes an “Object” column to store an object name, a “type” column to store an object's type, an “Obj-ID” column to store an object's identifier, a “Parent Object ID” column to store the object identifier of an object's parent object, a “Change operation” column to store an indication of the type associated with, or executed on, an object to cause a change in the object, and “New object” column to store a name of a new object.

After the local history table is sent to the backup server, the rows in the local history table are erased. Consequentially, after each backup the local history table is empty. This ensures that only new changes are sent to the backup server during each virtual machine backup. The example local history table, however, shows entries from five different backups (T1-T5).

FIG. 10illustrates an example global history table for tracking a history of filesystem object changes, according to various embodiments. The example global history table includes an “Object” column to store an object name, a “Type” column to store an object's type, “Ref-ID” columns (e.g., reference identifier fields) to store reference identifiers associated with backup times, an “Obj-ID” column to store an object's identifier, a “Parent Object ID” column to store the object identifier of an object's parent object. The “Ref-ID” further include a set of sub-columns for storing corresponding to backup times of each backup stored on a backup server.

Each time a backup of a virtual machine is executed, a new column is added to “Ref-ID” field at backup time Tx. The initial content of this column is copied from the previous backup column (e.g., column Tx−1). The “Object”, “Type”, “Obj-ID” and “Parent Obj-ID” columns entries in the of the global history table are the same as in the local history table, while the “Ref-ID” value is a number that is provided to a filesystem object in a sequential order depending on the change operation executed on the Object. Other alternative approaches of assigning the “Ref-ID” values are possible.

The following are brief examples of updates to the example global history table based on the example local history table depicted inFIG. 9andFIG. 10. As an example of an object change operations, at backup time T2, objects D2, F3 and F4 are added to new rows in the example global history table and assigned reference identifiers 4, 5, and 6, respectively. As an example of an object delete operation, at backup time T2, the reference identifier for object F1 is delete from the T2 column of the global history table. As an example of an object or file change operation, at backup time T3, a new reference identifier 10 is added to the T3 column in row of the global history table associated with changed object F2. As an example of an object move operation at backup time T5, filesystem object D1 is moved from parent object 0001 (e.g., the root directory) to parent object 0005 (e.g., directory D2). The move is illustrated at backup time T5 where the reference identifier associated with D1 is deleted, and a new row is added to the bottom of the global history table when the same object name D1, “Type” Dir, “Obj-ID” 002. A new “Parent Obj-ID” 0005 and a new “Ref-ID” 16 is added to the new row. As an example of an object rename operation, at backup time T5, the name of filesystem object D4 was changed to D5. The object rename is illustrated at backup time T5 where the reference identifier for backup time T5 is deleted from the row associated with filesystem object D4. A new row is then added to the global history table with the same “Type” Dir, “Obj-ID” 0011, “Parent Obj-ID” 0005 as object D4. In the example, the new row is the second from the last row in the global history table. The object name in the new row is set to the new object name D5, and a new reference identifier 15 is added to the new row in the “REF-ID” column for the current backup.

FIG. 11depicts a flowchart1100of an example set of computer implemented operations that may be executed to synchronize virtual machine backups stored on a backup server with a global history table after the expiration of a bock level virtual machine backup. The operations of the flowchart1100may be executed by a backup system having components executing on one or more computing systems to generate and store block level backups of a virtual machine, as described herein.

The backup system may begin the operations of flowchart1100at operation1105by executing one or more software routines to determine whether a block level backup of a virtual machine has expired. In some embodiments, the backup system may be configured to store a threshold number of block level backups of a virtual machine on a backup server. In these embodiments, a block level backup may expire when the backup system, having stored the threshold number of block level backups on the backup server, receives an additional block level backup to store. For example, a backup system may be configured to store three block level backups of a virtual machine. The backup system, after storing the threshold 3 block level backups, may receive a fourth block level backup to store. In response to receiving the fourth block level backup, the backup system may determine that one of the three previously saved backups have expired.

The backup system may execute operation1110to identify an expired block level backup. In some embodiments, the expired block level backup may be the oldest backup stored on a backup server. In other embodiments, the expired block level backup may be determined according to other identification or selection criteria. Identifying the expired block level backup may further include identifying a reference identifier column of the global history table associated with the expired block level backup.

The backup system may execute operation1115to delete the reference identifier column for the expired block level backup from the global history table. In some embodiments, the storage system may execute one or more software routines to delete the expired block level backup from the backup server.

The backup system may then execute operation1120to identify and delete rows of the global history table that have no more reference identifiers.