Abstract:
Techniques disclosed herein provide for global management of a set of data management services being executed by at least one of one or more physical data management appliances and one or more virtual data management appliances. First instructions are transmitted to a first copy data management appliance that cause the first copy data management appliance to send first data associated with protecting a first application entity to the management server. Second instructions are transmitted to a second copy data management appliance that cause the second copy data management appliance to send second data associated with protecting a second application entity to the management server. A plurality of data management policies associated with the first copy data management appliance and the second copy data management appliance are scheduled using a third service level template, which includes a merged set of schedules for the first and second copy data management appliances.

Description:
CROSS REFERENCE TO RELATED APPLICATIONS 
       [0001]    This application claims the benefit under 35 U.S.C. §119(e) of Provisional Application No. 62/051,138, filed Sep. 16, 2014, entitled “Apparatus and Computerized Methods for Copy Data Management;” Provisional Application No. 62/063,180, filed Oct. 13, 2014, entitled “Copy Data Techniques;” Provisional Application No. 62/106,306, filed Jan. 22, 2015, entitled “Apparatus and Computerized Methods for Copy Data Management;” and Provisional Application No. 62/160,153, filed May 12, 2015, entitled “Methods and Apparatus for Managing a Large-Scale Environment of Copy Data Management Appliances,” the entire contents of which are incorporated by reference here in their entirety. 
     
    
     TECHNICAL FIELD 
       [0002]    The techniques described herein generally relate to managing a large-scale environment of copy data management techniques, and in particular to managing multiple physical and virtual copy data management appliances. 
       BACKGROUND OF THE INVENTION 
       [0003]    Current management of copy data management appliances in an enterprise environment takes an approach of element manager, in which each element manager, e.g. a desktop GUI application, manages a single appliance. To manage a pair of copy data management appliances for disaster recovery purposes normally two element manages are needed, one on the local site and the other on the remote site. 
       SUMMARY 
       [0004]    In larger enterprise environments where multiple instances of copy data management appliances are used to manage data across a broad set of infrastructure, a holistic and global management solution can coordinate and orchestrate copy data management across all of the separately deployed appliances. Otherwise each appliance needs to be managed and configured separately, and information from one copy data management appliance cannot be used by other copy data management appliances. 
         [0005]    A Global Manager, or AGM in short, is such a computerized solution, which provides features not available previously because information from multiple physical and/or virtual copy data management appliances could not be leveraged by a single device. For example, AGM can provide features such as a timeline view of tasks and a management layer to control solutions such as Snapshots, Dedupe Pools, Live Clones, Remote Replications, and movements via Dedupe Async. Tasks can be managed along the lines of applications, hosts, VMs or groups of the same. In some exemplary embodiments, the physical copy data management appliance is an Actifio CDS and the virtual copy data management appliance is an Actifio SKY instance. 
         [0006]      FIG. 1A  shows a deployment diagram of an exemplary installment of an AGM server instance  100  to manage two copy data management appliances, a CDS appliance  108  and a SKY appliance  109 . Each appliance accesses a set of production data for copy data management. For example, CDS appliance  107  can access Virtual Machine application  112 , Oracle Database  113 , SqlServer Database  114 , Exchange Server  115  and ERP application  116 . SKY appliance can access ROBO (Remote Office Branch Office) application  117 , Cloud application  118  and Virtual Machine application  119 . The deployment diagram also shows other numbered components that are not immediately related with the techniques described herein and there are descriptions that are omitted.  FIG. 1B  shows a screen capture of AGM UI&#39;s Domain Manager, where two copy data management appliances,  120  and  121 , have been added under the management of an exemplary AGM server appliance. 
         [0007]    AGM can also manage a few hundred appliances for a large configuration with each appliance having managed entities up to a few hundred thousand. 
         [0008]    In summary the techniques described herein provide mechanisms and techniques for managing a large-scale copy data management environment. The management server instance, AGM in particular, uses an efficient data capture and change tracking technique to replicate management metadata in near real-time, which also maintains referential integrity at presence of replication failures. A policy-driven approach can be applied for assigning target copy data appliance for application protection. The protected application can be reassigned at a later time and the entire backup history is maintained by the management server instance through its protection life cycle. The management server instance also maintains a set of service level agreement policies, which can be applied across multiple appliances for application protection. 
         [0009]    The disclosed subject matter includes a computerized method of providing global management of a set of data management services being executed by at least one of one or more physical data management appliances and one or more virtual data management appliances. The method is executed by a management server comprising a processor in communication with memory storing instructions that are configured to cause the processor to transmit first instructions, from the management server, to a first copy data management appliance that cause the first copy data management appliance to send first data associated with protecting a first application entity protected by the first copy data management appliance to the management server, the first data including first service level agreement data associated with the first application entity, the first service level agreement data including a first service level template and a first service level profile, the first service level template including a plurality of first service level policies indicating a first set of schedules associated with backing up the first data, the first service level profile indicating at least one of a plurality of source storage pools and at least one of a plurality of destination storage pools associated with creating a copy of first content data associated with the first application entity. The memory stores instructions that are configured to cause the processor to transmit second instructions, from a management server, to a second copy data management appliance that cause the second copy data management appliance to send second data associated with protecting a second application entity protected by the second copy data management appliance to the management server, the second data including second service level agreement data associated with the second source entity, the second service level agreement data including a second service level template and a second service level profile, the second service level template including a plurality of second service level policies indicating a second set of schedules associated with backing up the second data, the second service level profile indicating at least one of a plurality of source storage pools and at least one of a plurality of destination storage pool associated with creating a copy of second content associated with the second application entity. The memory stores instructions that are configured to cause the processor to receive, at the management server, third data comprising instructions to create a third service level template, the third service level template including a link to the first service level template and to the second service level template such that the first set of schedules and the second set of schedule are merged into a third set of schedules. The memory stores instructions that are configured to cause the processor to create, at the management server, a list of backup images associated with the first data and the second data, and the list of backup images associated with: one of the first copy data management appliance and the second copy data management appliance, and one of the first source entity and the second source entity. The memory stores instructions that are configured to cause the processor to schedule, at the management server, a plurality of data management policies associated with the first copy data management appliance and the second copy data management appliance using the third service level template, the plurality of data management policies comprising at least one of: creating and retaining a first point in time image of first application data; creating, incrementally refreshing, and retaining a full copy of a second point in image associated with second application data; creating and retaining a first deduplicated point in time image of third application data; creating and retaining a second deduplicated backup of fourth application data at a remote site; and creating and retaining a third deduplicated backup of fifth application data using asynchronous data replication, thereby providing global management of a set of data management services for a plurality of physical and virtual appliances. 
         [0010]    The disclosed subject matter includes a computing system for providing global management of a set of data management services being executed by at least one of one or more physical data management appliances and one or more virtual data management appliances. The computing system includes a processor and a memory coupled to the processor. The memory includes instructions that, when executed by the processor, cause the processor to transmit first instructions to a first copy data management appliance that cause the first copy data management appliance to send first data associated with protecting a first application entity protected by the first copy data management appliance to the management server, the first data including first service level agreement data associated with the first application entity, the first service level agreement data including a first service level template and a first service level profile, the first service level template including a plurality of first service level policies indicating a first set of schedules associated with backing up the first data, the first service level profile indicating at least one of a plurality of source storage pools and at least one of a plurality of destination storage pools associated with creating a copy of first content data associated with the first application entity. The memory includes instructions that, when executed by the processor, cause the processor to transmit second instructions to a second copy data management appliance that cause the second copy data management appliance to send second data associated with protecting a second application entity protected by the second copy data management appliance to the management server, the second data including second service level agreement data associated with the second source entity, the second service level agreement data including a second service level template and a second service level profile, the second service level template including a plurality of second service level policies indicating a second set of schedules associated with backing up the second data, the second service level profile indicating at least one of a plurality of source storage pools and at least one of a plurality of destination storage pool associated with creating a copy of second content associated with the second application entity. The memory includes instructions that, when executed by the processor, cause the processor to receive third data comprising instructions to create a third service level template, the third service level template including a link to the first service level template and to the second service level template such that the first set of schedules and the second set of schedule are merged into a third set of schedules. The memory includes instructions that, when executed by the processor, cause the processor to create a list of backup images associated with the first data and the second data, and the list of backup images associated with: one of the first copy data management appliance and the second copy data management appliance, and one of the first source entity and the second source entity. The memory includes instructions that, when executed by the processor, cause the processor to schedule a plurality of data management policies associated with the first copy data management appliance and the second copy data management appliance using the third service level template, the plurality of data management policies comprising at least one of: creating and retaining a first point in time image of first application data; creating, incrementally refreshing, and retaining a full copy of a second point in image associated with second application data; creating and retaining a first deduplicated point in time image of third application data; creating and retaining a second deduplicated backup of fourth application data at a remote site; and creating and retaining a third deduplicated backup of fifth application data using asynchronous data replication, thereby providing global management of a set of data management services for a plurality of physical and virtual appliances. 
         [0011]    The disclosed subject matter includes a non-transitory computer readable medium having executable instructions operable to cause an apparatus to transmit first instructions to a first copy data management appliance that cause the first copy data management appliance to send first data associated with protecting a first application entity protected by the first copy data management appliance to the management server, the first data including first service level agreement data associated with the first application entity, the first service level agreement data including a first service level template and a first service level profile, the first service level template including a plurality of first service level policies indicating a first set of schedules associated with backing up the first data, the first service level profile indicating at least one of a plurality of source storage pools and at least one of a plurality of destination storage pools associated with creating a copy of first content data associated with the first application entity. The non-transitory computer readable medium having executable instructions operable to cause an apparatus to transmit second instructions to a second copy data management appliance that cause the second copy data management appliance to send second data associated with protecting a second application entity protected by the second copy data management appliance to the management server, the second data including second service level agreement data associated with the second source entity, the second service level agreement data including a second service level template and a second service level profile, the second service level template including a plurality of second service level policies indicating a second set of schedules associated with backing up the second data, the second service level profile indicating at least one of a plurality of source storage pools and at least one of a plurality of destination storage pool associated with creating a copy of second content associated with the second application entity. non-transitory computer readable medium having executable instructions operable to cause an apparatus to receive third data comprising instructions to create a third service level template, the third service level template including a link to the first service level template and to the second service level template such that the first set of schedules and the second set of schedule are merged into a third set of schedules. non-transitory computer readable medium having executable instructions operable to cause an apparatus to create a list of backup images associated with the first data and the second data, and the list of backup images associated with: one of the first copy data management appliance and the second copy data management appliance, and one of the first source entity and the second source entity. non-transitory computer readable medium having executable instructions operable to cause an apparatus to schedule a plurality of data management policies associated with the first copy data management appliance and the second copy data management appliance using the third service level template, the plurality of data management policies comprising at least one of: creating and retaining a first point in time image of first application data; creating, incrementally refreshing, and retaining a full copy of a second point in image associated with second application data; creating and retaining a first deduplicated point in time image of third application data; creating and retaining a second deduplicated backup of fourth application data at a remote site; and creating and retaining a third deduplicated backup of fifth application data using asynchronous data replication, thereby providing global management of a set of data management services for a plurality of physical and virtual appliances. 
         [0012]    The disclosed subject matter includes a computerized method of providing global management of a set of data management services being executed by one or more physical data management appliances and one or more virtual data management appliances. The method is executed by a processor in communication with memory storing instructions that are configured to cause the processor to transmit first instructions to a first copy data management appliance that cause the first copy data management appliance to replicate first data associated with protecting a file or application from a first source entity protected by the first copy data management appliance to form a first inventory object, the first data including first host data, first application data, and first backup data, wherein each of the first host data, the first application data, and the first backup data include a first composite key identifying the associated first copy data management appliance and the first source entity. memory storing instructions that are configured to cause the processor to transmit second instructions to a second copy data management appliance that cause the second data management appliance to replicate second data associated with protecting a file or application from a second source entity protected by the second copy data management appliance, the second data including host data, application data, and backup data, wherein each of the second host data, the second application data, and the second backup data include a second composite key identifying the associated second copy data management appliance and the second source entity, wherein replicating from the second copy data management appliance, comprises: associating the second data with the first inventory object when the second composite key matches the first composite key; and forming a second inventory object from the second data when the second composite key does not match the first composite key. 
         [0013]    In some embodiments, the first copy data management appliance is one of a physical copy data management appliance and a virtual copy data management appliance, further wherein the second copy data management appliance is one of a physical copy data management appliance and a virtual copy data management appliance. 
         [0014]    In some embodiments, the processor in communication with memory is configured to transmit third instructions, from a management server, to a first copy data management appliance that cause the first copy data management appliance to send first data associated with protecting a file or application from a first source entity protected by the first copy data management appliance to the management server, the first data including first service level agreement data associated with the first source entity, the first service level agreement data including a first service level template and a first service level profile, the first service level template including a plurality of first service level policies indicating a first set of schedules associated with backing up the first data, the first service level profile indicating at least one of a plurality of source storage pools and at least one of a plurality of destination storage pools associated with creating a copy of first content data associated with the first source entity. Fourth instructions can be transmitted from a management server, to a second copy data management appliance that cause the second copy data management appliance to send second data associated with protecting a file or application from a second source entity protected by the second copy data management appliance to the management server, the second data including second service level agreement data associated with the second source entity, the second service level agreement data including a second service level template and a second service level profile, the second service level template including a plurality of second service level policies indicating a second set of schedules associated with backing up the second data, the second service level profile indicating at least one of a plurality of source storage pools and at least one of a plurality of destination storage pool associated with creating a copy of second content associated with the second source entity. Third data can be received, including instructions to create a third service level template, the third service level template including a link to the first service level template and to the second service level template such that the first set of schedules and the second set of schedule are merged into a third set of schedules. A list of backup images associated with the first data and the second data can be created, the list of backup images associated with: one of the first copy data management appliance and the second copy data management appliance, and one of the first source entity and the second source entity. A plurality of data management policies associated with the first copy data management appliance and the second copy data management appliance can be scheduled using the third service level template, the plurality of data management policies comprising at least one of: creating and retaining a first point in time image of first application data; creating, incrementally refreshing, and retaining a full copy of a second point in image associated with second application data; creating and retaining a first deduplicated point in time image of third application data; creating and retaining a second deduplicated backup of fourth application data at a remote site; and creating and retaining a third deduplicated backup of fifth application data using asynchronous data replication, thereby providing global management of a set of data management services for a plurality of physical and virtual appliances. 
         [0015]    In some embodiments, replicating from a first copy data management appliance includes receiving a dependency mapping corresponding to an order for replication, wherein the dependency mapping includes a plurality of replication entities, the replication entities associated with the first host data, the first application data, the first backup data, the first service level template, and the first service level profile, and the first service level agreement, replicating a first replication entity of the plurality of replication entities when the first replication entity has not been replicated and does not have a depending entity that has not been replicated, finding a first depending replication entity of the plurality of replication entities when the first replication entity has not been replicated and has a depending replication entity, and replicating the first depending entity when the first depending entity has not been replicated and does not have a second depending entity that has not been replicated. 
         [0016]    In some examples, techniques described in U.S. Patent Application Publication No. 2013/0339297, entitled “System and Method for Efficient Database Record Replication Using Different Replication Strategies based on the Database Records,” filed on Jun. 18, 2013, can be leveraged by the copy data management system, which is hereby incorporated by reference herein in its entirety. For example, a Service Level Agreement (SLA) can capture the detailed business requirements with respect to secondary copies of the application data. A Service Level Template (SLT) can contain a plurality of Service Level Policies with each Service Level Policy representing one single statement in the business requirements for the protection of application data. The statement concerns the type of backups to be generated, the interval to generate the backups and the retain period of generated backups to name a few. The set of Service Level Policies applied together composes one Service Level Template (SLT). A Service Level Profile (SLP) can include all of the Service Level Policies with a particular combination of source and destination storage pool and location, say for example, source Primary Storage pool and destination local Snapshot pool, when taken together, specify the business requirements for creating copies into that particular destination pool. The resource pool requirement is captured as Service Level Profile (SLP). 
         [0017]    These and other capabilities of the disclosed subject matter will be more fully understood after a review of the following figures, detailed description, and claims. It is to be understood that the phraseology and terminology employed herein are for the purpose of description and should not be regarded as limiting. 
     
    
     
       BRIEF DESCRIPTION OF FIGURES 
         [0018]    Various objectives, features, and advantages of the disclosed subject matter can be more fully appreciated with reference to the following detailed description of the disclosed subject matter when considered in connection with the following drawings, in which like reference numerals identify like elements. 
           [0019]      FIG. 1A  is an exemplary deployment diagram of an exemplary installment of an AGM server appliance to manage two technology enabled platforms, one is a CDS instance and the other is a SKY instance, according to some embodiments. 
           [0020]      FIG. 1B  is an exemplary screen capture of AGM UI&#39;s Domain Manager, which shows two copy data management appliances have been added under the management of an exemplary AGM server instance, according to some embodiments. 
           [0021]      FIG. 2A  is an exemplary screen capture of AGM UI&#39;s Application Manager, which shows AGM&#39;s inventory of all applications of various types of all the appliances managed by the exemplary AGM server appliance, according to some embodiments. 
           [0022]      FIG. 2B  is an exemplary screen capture of AGM UI&#39;s Application Manager, which shows backup images of an exemplary application on all AGM&#39;s member appliances, according to some embodiments. 
           [0023]      FIG. 2C  is an exemplary diagram that shows AGM&#39;s different model components, which include replication model, inventory model and protection model, according to some embodiments. 
           [0024]      FIG. 2D  is an exemplary flow chart that illustrates the algorithm of smart management data capture and replication, according to some embodiments. 
           [0025]      FIG. 2E  is an exemplary xml snippet that describes the dependency relationship of inventory model used by the replication algorithm of  FIG. 2D , according to some embodiments. 
           [0026]      FIG. 2F  is an exemplary screen capture of AGM UI&#39;s Application Manager, which shows backup images of an exemplary application on all AGM&#39;s member appliances, according to some embodiments. 
           [0027]      FIG. 3A  is an exemplary screen capture of AGM UI&#39;s SLA Architect, which shows AGM&#39;s inventory of all SLA templates, according to some embodiments. 
           [0028]      FIG. 3B  is an exemplary screen capture of AGM UI&#39;s SLA Architect, which shows the content of an exemplary SLA template, according to some embodiments. 
           [0029]      FIG. 3C  is an exemplary screen capture of AGM UI&#39;s Application Manager, which shows the protection of an exemplary application by a centrally managed AGM SLA template, according to some embodiments. 
           [0030]      FIG. 3D  is an exemplary diagram which shows the policy-driven approach that AGM adopts to centrally manage the inventory of SLA templates which could be distributed among all member appliances, according to some embodiments. 
           [0031]      FIG. 4A  is an exemplary screen capture of AGM UI&#39;s Application Manager, which shows the unique feature enabled by AGM to migrate the protection of a virtual machine application from one member appliance to another, according to some embodiments. 
           [0032]      FIG. 4B  is an exemplary diagram which shows the pre-migration inventory objects related with a particular virtual machine application among AGM server appliance, the source appliance and the target appliance, according to some embodiments. 
           [0033]      FIG. 4C  is an exemplary diagram which shows the post-migration inventory objects related with a particular virtual machine application among AGM server appliance, the source appliance and the target appliance, according to some embodiments. 
           [0034]      FIG. 5  illustrates an exemplary apparatus configured to provide global management for copy data appliances, according to some embodiments. 
       
    
    
     DETAILED DESCRIPTION 
       [0035]    In the following description, numerous specific details are set forth regarding the systems and methods of the disclosed subject matter and the environment in which such systems and methods may operate, etc., in order to provide a thorough understanding of the disclosed subject matter. It will be apparent to one skilled in the art, however, that the disclosed subject matter may be practiced without such specific details, and that certain features, which are well known in the art, are not described in detail in order to avoid unnecessary complication of the disclosed subject matter. In addition, it will be understood that the embodiments provided below are exemplary, and that it is contemplated that there are other systems and methods that are within the scope of the disclosed subject matter. 
       Smart Management Data Capture and Replication 
       [0036]      FIG. 2A  is an exemplary screen capture of AGM UI&#39;s Application Manager, which shows AGM&#39;s inventory of all applications of various types of all the appliances managed by an exemplary AGM server appliance. The list of applications is shown in a tabular format with each line representing a discovered application, e.g., virtual machine  230  and  231 . The attributes of an application including its name  232 , host  233 , type  235  and appliance  234  it resides are shown as table columns. The exemplar screen shows the filtered result of virtual machine only and only the first of seven pages all virtual machines is in display. 
         [0037]      FIG. 2B  is an exemplary screen capture of AGM UI&#39;s Application Manager, which is a graphical display designed to show all the backup images of an exemplary application selected from Application Manager as embodied in  FIG. 2A . In the center all the backup images of the exemplar application are shown in a ramp format. Different types of backups, Snapshot  240 , Dedup  241 , Liveclone  242 , Remote-Dedup  243  and Dedup-Async  244 , are each shown in its own lane. Snapshot is a backup image, which can be generated in an efficient way in regards to both time and disk space. A snapshot is a point in time image of data. Dedup is a backup image, which can be generated from snapshot backup and compressed using deduplication technology. Dedup images can be retained for longer period of time because they take up less space than non-deduplicated images. Remote-Dedup is a type of backup image, which can be generated from a Dedup image and sent to store at the remote appliance for disaster recovery purposes. Dedup-Async is a backup image, which can be generated from a snapshot image but sent to the remote appliance and kept up to date by periodic refresh and stands ready for restore at the remote site. Liveclone is a type of backup image, which is a full copy from the snapshot image but can be refreshed incrementally from the snapshot image. The levels of each lane represent the time series of all backups produced for that particular type. The accordion picture below the ramp shows the same information. In some embodiments, different portions of the ramp can be shown (e.g., scrolled through in terms of time). The accordion  246  can be similarly adjusted. In some embodiments, adjusting the ramp view can affect the accordion view and adjusting the accordion view can affect the ramp view.  FIG. 2F  shows a ramp review after moving the focus slider  247  of the accordion  246  to the end. The tab  245  of the upper corner shows the detailed information of the backup image and operations can be launched for that particular image can be launched from there. 
         [0038]    In some embodiments, an icon associated with a backup image will change appearance (e.g., have highlighting, change color) when selected or hovered over by a cursor. Other backup images associated in time with the selected backup image can also change appearance indicating the association. 
         [0039]    Management entity, for example, application and backup entity as embodied in  FIGS. 2A and 2B  is continuously monitored and replicated from member appliances to the AGM server instance through the life cycle of managing appliance. 
         [0040]      FIG. 2C  is a diagram which illustrates  3  models used by AGM for the life cycle of managing an appliance. The details of each model are described below. 
         [0041]    Replication Model. Source entities embodied as HostSource  208 , ApplicationSource  209  and BackupSource  210  are replicated to AGM server instance, respectively, as HostRepl  205 , ApplicationRepl  206  and BackupRepl  207 . Similar protection related entities embodied as SLASource  217 , SLTSource  218  and SLPSource  219  are replicated respectively as SLARepl  214 , SLTRepl  215  and SLPRepl  216 . For each source entity object there is a corresponding replication entity within AGM that the replication process periodically synchronizes with. Replication entities from all member appliances form the replication model of the AGM instance. To uniquely identify the link between the source entity and the replication entity a unique composite key is defined on the replication entity which consists of a pair of ids, (clusterid, srcid), where clusterid is a global unique id assigned to each appliance and srcid is the id of the source entity which is globally unique within that appliance. 
         [0042]    Inventory Model. Once replication model is formed a correlation and identification step as part of the replication process is conducted. For each replicate entity, for example:
       If an inventory object of the same type with a matching business key, for example, uuid for a vCenter server, exists, the replicate entity will be associated with the existing inventory object.   If no previous inventory entity object with a matching business key exists, a new one will be created and associated with incoming replicate entity.       
 
         [0045]    As embodied in  FIG. 2C  inventory entities such as Host  202 , Application  203  and Backup  204  are identified from replicate entities, respectively, HostRepl  205 , ApplicationRepl  206  and BackupRepl  207 . Similarly protection related inventory entities embodied as SLA  211 , SLT  212  and SLP  213  are identified from replicate entities, respectively, SLARepl  214 , SLTRepl  215  and SLPRepl  216 . The inventory entities correlated and identified from replicate entities form the inventory model of the AGM instance. Some exemplary use cases of the inventory model are described below:
       If an Application, for example, a VMware virtual machine, is discovered or protected on multiple appliances, the correlation and identification step will identify all source entities that represent the same Application as a single AGM entity. This enables AGM to provide a consolidated view of all the Backups for that Application that otherwise can only be seen partially from a single appliance. The AGM inventory model also enables the techniques described below in “Move SLA” section.   The management functionalities and interfaces are defined using AGM inventory model, which abstracts out underlying differences represented by the replication model constructed out of source entities of different types of appliances.   When a provisioning operation, like mounting a snapshot image, to a target application, is conducted against an AGM inventory entity, the corresponding replicate entity is located which provides the link back to the source entity of the appliance. The provisioning operation is then carried out on the source appliance.       
 
         [0049]    Protection Model. Following inventory model construction the relationships for application protection embodied as SLASource_ApplicationSource  220 , SLASource_SLTSource  221  and SLASource_SLPSource  222  are fully replicated and reconstructed at AGM using inventory entities, which is embodied as SLA_Application  223 , SLA_SLT  224  and SLA_SLP  225 . 
         [0050]    AGM can be designed to manage multiple appliances ranging from a few dozen for a small to medium-sized configuration to a few hundred for a large configuration. Each appliance can have a large number of managed entities, ranging from a few thousand to a few hundred thousand, to replicate to AGM. The data capture and replication techniques can be based in part upon the section of “Smart Data Synchronization” in U.S. Patent Application Publication No. 2013/0339297. 
         [0051]    U.S. Patent Application Publication No. 2013/0339297 describes a set of techniques of replicating management entities. Management entities of individual types are replicated incrementally but different types of entities are replicated without consideration of their interdependency. As embodied in  FIG. 2C  ApplicationRepl  206  can be replicated before HostRepl  205 . For AGM this causes a violation of referential integrity, as an application entity cannot exist without its containing host entity. A host, modeled as a host entity, is a physical or a virtual server, which runs an operating system, e.g., a Windows server or a Linux server. In an enterprise environment each host can run one or multiple applications for business solutions. The application can be file systems, database servers, email servers, content servers and virtual machines. Each application, modeled as an application entity, consumes disk spaces to store data critical to business processes. The data of each application needs to be backed up periodically to prevent data loss and for auditing. Each backup of all disk spaces consumed by the application is modeled as one backup entity and application entities can contain many such backup entities. 
         [0052]    Unlike previous technologies, the techniques described herein preserve referential integrity of AGM inventory. Source entities of different types of appliances are replicated in a topological order calculated from a predefined dependency graph, which describes parent-child relationship among related entities. In doing so the referential integrity among all related entities are preserved. The replication algorithm tolerates the restart of either AGM or the target appliance when replication is on the fly. The algorithm also tolerates failure of replication of any single type of entities without compromising the referential integrity of the AGM inventory. 
         [0053]      FIG. 2D  is an exemplary flow chart that illustrates this algorithm of smart management data capture and replication. The algorithm traverses a predefined dependency graph embodied by the xml snippets in  FIG. 2E . The algorithm as such is recursive in nature. 
         [0054]    When the algorithm starts to replicate an entity, it first checks if the entity has been replicated or not by condition check  251 . If the condition check passes then the recursion step of replicating the entity is done and control moves to stop  255 . If the entity has not been replicated, control moves to condition check  252 . In step  252  if there are any depending entities that need to be replicated before current entity, control moves to step  256  and finds the first depending entity the algorithm needs to replicate. The function implementing the algorithm then recurses by calling itself again to replicate the depending entity, which moves control to start  250 . If all depending entities have been replicated by passing condition check  252 , control moves to step  253  to replicate the current entity. Step  253  is composed of a series of small steps from step  258  to step  265 , which can be based in part on “Smart DataSynchronization” described in U.S. Patent Application Publication No. 2013/0339297, except step  264 , which is described further below. Once step is finished control moves to condition check  254 . If there are any dependent entities that need to be replicated after the current entity, control moves step  257  and finds the first dependent entity the algorithm needs to replicate. The function implementing the algorithm then recurses by calling itself again to replicate the dependent entity, which moves control to start  250 . If all dependent entities have been replicated by passing condition check  254 , control moves to stop  255  finishing up replicating the entity. 
         [0055]    The algorithm described above requires finding both the parent and child entities of current replicating entity. The parent-child relationship is embodied by the xml snippet of  FIG. 2E . Element  270  defines the dependency schema. Element  271  states that Application depends on Host. Element  272  states that Host has no dependencies. Element  273  states that SLA depends on a list of entities of Application, SLT and SLP. Element  275  states that SLT depends on Policy. Element  274  states that SLP has no dependencies and Element  276  states that Policy has no dependencies. 
         [0056]    Step  258  to step  265 , which can be based in part on “Smart DataSynchronization” of U.S. Patent Application Publication No. 2013/0339297, can attempt to guarantee the fault tolerance of replication failure of a single entity type as below, 
         [0057]    No partial replication. An entity is either fully replicated or not at all. 
         [0058]    Restart of replication. Replication can always be restarted at where it was left off. 
         [0059]    Step  250  to step  257  provides the following additional guarantee when replication of a particular entity type fails,
       No referential constraint violation. Successfully replicated entities have their entire parent entities successfully replicated as well.   Restart of replication. Replication of different entity types always follows the same predefined order and can be restarted at any point of the dependency graph where it was left off.       
 
         [0062]    These techniques can provide efficiency since as the replication of individual entity types is incremental. Only entities changed since last replication are replicated. 
         [0063]    Regarding step  264  of  FIG. 2D  previous techniques replicated deleted objects by sending all known IDS from a requesting side to a target appliance. This can cause one or more of the following performance issues:
       The AGM inventory can hold entity objects per type for multiple appliances, each of which can have a large number of objects. Generating a complete ID list per object type per appliance can be expensive.   Filtering out deleted IDs based on AGM&#39;s sent known IDs is also an expensive operation, which further reduces response time.       
 
         [0066]    Step  264  implements a newly introduced mechanism of keeping track of deleted entities on the target appliances using, for example, a “tombstone” table: 
         [0000]    
       
         
               
               
             
               
               
               
             
               
               
             
           
               
                   
                   
               
             
             
               
                   
                 CREATE TABLE tombstone ( 
               
             
          
           
               
                   
                  id 
                 BIGINT NOT NULL, 
               
               
                   
                  typename  
                 VARCHAR(64) NOT NULL, 
               
               
                   
                  enddate 
                 TIMESTAMP NOT NULL, 
               
             
          
           
               
                   
                  PRIMARY KEY(id, typename) 
               
               
                   
                 ); 
               
               
                   
                   
               
             
          
         
       
     
         [0067]    Upon deletion of any tracked entity type within the target appliance a record is entered into the tombstone table to record its deletion. When retrieving deleted records from target appliance instead of sending all known IDs AGM just sends (MaxKnownID, type). Any IDs smaller than MaxKnownID for a particular entity type that exists in the “tombstone” table are turned to the AGM side and its AGM inventory object would be deleted accordingly. Periodically the tombstone table is purged based on the replication history of individual entity types. 
         [0068]    Combined the techniques described above can be utilized so that AGM can manage a large-scale data protection environment leveraging the appliances. 
       Distributed Policy-Driven Data Protection Using Centrally Managed Service Level Templates 
       [0069]    Upon construction of AGM entity inventory the service level templates and their associated policies go through a correlation and conflict resolution process. A protection model is then created on top of newly identified AGM entities, which consists of a set of centrally managed set of service level templates with identified and correlated replicas distributed across appliances managed by the AGM instance. 
         [0070]      FIG. 3A  is an exemplary screen capture of AGM UI&#39;s SLA Architect  330 , which shows AGM&#39;s inventory of all SLA templates. All the service level templates are listed in a tabular format with each line representing a managed template, e.g. service template  331  and  332 . The attributes of a template including its name  333 , description  334  and whether it can be overridden,  335 , are shown as table columns. The exemplar screen shows the first of two pages of all templates.  FIG. 3B  is an exemplary screen capture of AGM UI&#39;s SLA Architect, which shows the content of an exemplary SLA template  340 . The screen shows the template&#39;s name  341 , description  342  and control of the override flag at the top  343 . In the center is a graphical representation  345  of various service level policies the template is composed of. The tab  346  on the right shows the service level policies in a list format. Clicking on the “+” sign  347  of the graph opens up its corresponding editing pane within the right side tab.  FIG. 3C  is an exemplary screen capture of AGM UI&#39;s Application Manager  350 , which shows the protection of an exemplary application using a centrally managed AGM SLA template. The service level template  351  and the service level profile  352  used for protecting the exemplar application are shown at the top. The blue buttons on the right controls the advanced settings  353  of the protection and enabling and disabling of the protection. In the center is the graph  354  of the service level polices the service template is composed of. The difference of  FIG. 3C  and  FIG. 3B  is that  FIG. 3C  shows all defined policies while  FIG. 3B  shows all possible policies that can be defined. 
         [0071]      FIG. 3D  is an exemplary diagram which the policy-driven approach that AGM adopts to centrally manage the inventory of SLA templates which could be distributed among all member appliances. Within AGM inventory  300  an application embodied as Application  305  has been replicated from an appliance  301  and hence backed by a replica of Application  312 . It is protected by a SLA template  306  backed by a replica of SLA template  313  and a SLP backed by a replica of SLP  314 . Similarly an application embodied as Application  308  is protected a SLA template  306  and also a SLP  310 , which are backed respectively by the SLA template replica  317  and SLP replica  318  on appliance  318 . It should be pointed out two different SLAs, SLA  303  and SLA  304 , are sharing a common service level template SLT  306 , which are backed respectively by two identical replicas with each on a different appliance, which is embodied as a service level template  313  on appliance  301  and a service level template  317  on appliance  302 . Upon construction of the protection model AGM will tag all correlated and identified service level templates to be owned by the AGM instance preventing them from further modification from the appliance side, which is termed as promotion. Changes of promoted service level templates can only be pushed from AGM side. As embodied in  FIG. 3D  the content of service level template replicas, SLT  313  on appliance  301  and SLT  317  on appliance  302 , are semantically identical to that of SLT  306  within AGM  300 . Changes may only be made to SLT  306  on AGM side and pushed from SLT  306  to SLT replica  313  and SLT replica  317  in that direction. 
         [0072]    AGM can adopt a “lazy” push policy. When a service level template is initially created on AGM, it is not pushed to all appliances immediately. When protecting an application on a target appliance, all service level templates visible to the application on the target appliance are presented. The visible set of service level templates includes all templates originated from the appliance and also all newly created AGM service level templates that have not been pushed to the appliance. If any of the AGM service level templates that does not have a replica on the target appliance is chosen, the template will be pushed down from AGM first before a SLA is created on the target appliance using the newly pushed service level template. This lazy push policy makes sure that only the appliance that actively uses the service template will receive the replica, which saves bandwidth and also reduces the chance of a partial failure when pushing service level template changes. 
       Move SLA 
       [0073]    The capability of correlation and identification of the same application discovered on multiple appliances with AGM inventory is a unique feature in AGM.  FIG. 4A  is a screen capture of AGM UI&#39;s Application Manager  420 , which showcases the feature of migrating the protection of a virtual machine application from one member appliance to another. The left pane  430  of the UI shows the current protection status of the application on the source appliance  424 . The exemplar application  421  is protected by a service level template  422  listed under “current template name” and a service level profile  423  listed under “current resource profile”. The right pane  431  of the UI shows the required parameters for re-protecting the application on the target appliance. For example, the parameters include: the target appliance  425 , the target service level template  426  and the target service level profile  427 . 
         [0074]      FIG. 4B  is a diagram which shows the pre-migration inventory objects related with a particular virtual machine application among AGM server appliance, the source appliance and the target appliance. As embodied inside Appliance  401  Application  410  is protected by SLA 1   409 , which refers to SLT  412  and SLP 1   411 . The protection has created a Backup 1   413 . 
         [0075]    AGM inventory has the source entities from appliance  401  fully replicated. The embodiments of the diagram show that inside AGM instance  400  Application  403  are protected by SLA 1   404 , which refers to SLT  406  and SLP 1   407 . The Application has created a Backup  405 . All the above inventory entities are replicated from source appliances, respectively, Application  410 , SLA 1   409 , SLT  412  and SLP 1   411 . And AGM inventory&#39;s Backup 1   405  is replicated from appliance&#39;s Backup 1   413 . The diagram also shows the required parameters for conducting the “Move SLA” operation, among which SLT  406  has been pushed down to appliance  402  and appliance&#39;s SLP  415  has been replicated to AGM as SLP 2   408 . 
         [0076]      FIG. 4C  is a diagram which shows the post-migration inventory objects related with a particular virtual machine application among AGM server appliance, the source appliance and the target appliance. The diagram shows that the target appliance  402  has Application  417  rediscovered and protected with a new SLA 2   416 , which refers to SLT  414  and SLP 2   415 . A new Backup 2   418  has been produced in appliance  402 . The newly created entities in target appliance  402  are fully replicated to AGM inventory, respectively, as Application  402 , SLA 2   419 , SLT  406 , SLP 2   408  and backup 2   419 . 
         [0077]    It is worth noting the Application  410  has been unprotected on source appliance  402  and has been removed from that appliance. However Backup 1   413  still exists on source appliance  401  subject to expiration. 
         [0078]    Application  403  preserves its identity during the entire process of Move SLA in AGM server  400  even the protection of the application has moved from SLAT of appliance  401  to SLA 2  of appliance  402 . Also Application  403  keeps track of all backup images during the life cycle of the protection of the application, respectively embodied as Backup 1   405  from appliance  401  and Backup 2   419  from appliance  402 . 
         [0079]      FIG. 5  illustrates an exemplary apparatus configured to provide global management for copy data appliances, according to some embodiments. 
         [0080]      FIG. 5  illustrates an exemplary computerized apparatus  5000  configured to provide a global management solution, according to some embodiments. Computerized apparatus  5000  includes a processor  5001  in communication with temporary and/or permanent memory  5002 . Memory  5002  stores instructions and is configured to cause the processor  5001  to perform the subject matter described herein, including the apparatuses (e.g., servers, data appliances) and methods disclosed in the figures above. As described further herein, the memory  5002  can be flash memory, a magnetic disk drive, an optical drive, a programmable read-only memory (PROM), a read-only memory (ROM), or any other memory or combination of memories. The processor  5001  can be a general purpose processor and/or can also be implemented using an application specific integrated circuit (ASIC), programmable logic array (PLA), field programmable gate array (FPGA), and/or any other integrated circuit. The computerized apparatus  5000  can include a database that may also be flash memory, a magnetic disk drive, an optical drive, a programmable read-only memory (PROM), a read-only memory (ROM), or any other memory or combination of memories. The computerized apparatus  5000  can execute an operating system that can be any operating system, including a typical operating system such as Windows, Windows XP, Windows 7, Windows 8, Windows Mobile, Windows Phone, Windows RT, Mac OS X, Linux, VXWorks, Android, Blackberry OS, iOS, Symbian, or other OSs. 
         [0081]    The subject matter described herein can be implemented in digital electronic circuitry, or in computer software, firmware, or hardware, including the structural means disclosed in this specification and structural equivalents thereof, or in combinations of them. The subject matter described herein can be implemented as one or more computer program products, such as one or more computer programs tangibly embodied in an information carrier (e.g., in a machine readable storage device), or embodied in a propagated signal, for execution by, or to control the operation of, data processing apparatus (e.g., a programmable processor, a computer, or multiple computers). A computer program (also known as a program, software, software application, or code) can be written in any form of programming language, including compiled or interpreted languages, and it can be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. A computer program does not necessarily correspond to a file. A program can be stored in a portion of a file that holds other programs or data, in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files that store one or more modules, sub programs, or portions of code). A computer program can be deployed to be executed on one computer or on multiple computers at one site or distributed across multiple sites and interconnected by a communication network. 
         [0082]    The processes and logic flows described in this specification, including the method steps of the subject matter described herein, can be performed by one or more programmable processors executing one or more computer programs to perform functions of the subject matter described herein by operating on input data and generating output. The processes and logic flows can also be performed by, and apparatus of the subject matter described herein can be implemented as, special purpose logic circuitry, e.g., an FPGA (field programmable gate array) or an ASIC (application specific integrated circuit). 
         [0083]    Processors suitable for the execution of a computer program include, by way of example, both general and special purpose microprocessors, and any one or more processor of any kind of digital computer. Generally, a processor will receive instructions and data from a read only memory or a random access memory or both. The essential elements of a computer are a processor for executing instructions and one or more memory devices for storing instructions and data. Generally, a computer will also include, or be operatively coupled to receive data from or transfer data to, or both, one or more mass storage devices for storing data, e.g., magnetic, magneto optical disks, or optical disks. Information carriers suitable for embodying computer program instructions and data include all forms of nonvolatile memory, including by way of example semiconductor memory devices, (e.g., EPROM, EEPROM, and flash memory devices); magnetic disks, (e.g., internal hard disks or removable disks); magneto optical disks; and optical disks (e.g., CD and DVD disks). The processor and the memory can be supplemented by, or incorporated in, special purpose logic circuitry. 
         [0084]    To provide for interaction with a user, the subject matter described herein can be implemented on a computer having a display device, e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor, for displaying information to the user and a keyboard and a pointing device, (e.g., a mouse or a trackball), by which the user can provide input to the computer. Other kinds of devices can be used to provide for interaction with a user as well. For example, feedback provided to the user can be any form of sensory feedback, (e.g., visual feedback, auditory feedback, or tactile feedback), and input from the user can be received in any form, including acoustic, speech, or tactile input. 
         [0085]    The subject matter described herein can be implemented in a computing system that includes a back end component (e.g., a data server), a middleware component (e.g., an application server), or a front end component (e.g., a client computer having a graphical user interface or a web browser through which a user can interact with an implementation of the subject matter described herein), or any combination of such back end, middleware, and front end components. The components of the system can be interconnected by any form or medium of digital data communication, e.g., a communication network. Examples of communication networks include a local area network (“LAN”) and a wide area network (“WAN”), e.g., the Internet. 
         [0086]    It is to be understood that the disclosed subject matter is not limited in its application to the details of construction and to the arrangements of the components set forth in the following description or illustrated in the drawings. The disclosed subject matter is capable of other embodiments and of being practiced and carried out in various ways. Also, it is to be understood that the phraseology and terminology employed herein are for the purpose of description and should not be regarded as limiting. 
         [0087]    As such, those skilled in the art will appreciate that the conception, upon which this disclosure is based, may readily be utilized as a basis for the designing of other structures, methods, and systems for carrying out the several purposes of the disclosed subject matter. It is important, therefore, that the claims be regarded as including such equivalent constructions insofar as they do not depart from the spirit and scope of the disclosed subject matter. 
         [0088]    Although the disclosed subject matter has been described and illustrated in the foregoing exemplary embodiments, it is understood that the present disclosure has been made only by way of example, and that numerous changes in the details of implementation of the disclosed subject matter may be made without departing from the spirit and scope of the disclosed subject matter.