Abstract:
Techniques are disclosed for preventing a user from configuring a service level agreement from creating a data management schedule that creates a set of data backups that exceeds data resource limits available for storing the set of data backups. A first amount of pool resources associated with the backup of each of the at least one application is determined according to a received schedule. A first amount of data volumes associated with the backup of each of the at least one application is determined according to the received schedule. A resource shortage warning is transmitted when the aggregate amount of pool resources exceeds an available amount of pool resources or the aggregate amount of data volume resource exceeds an available amount of data volume resource, thereby preventing a user from configuring a service level agreement that exceeds data resource limits.

Description:
CROSS REFERENCE TO RELATED APPLICATIONS 
       [0001]    This application claims the benefit under 35 U.S.C. §119(e) of Provisional Application No. 62/051,138, filed Sep. 16, 2014, entitled “Apparatus and Computerized Methods for Copy Data Management;” and Provisional Application No. 62/063,180, filed Oct. 13, 2014, entitled “Copy Data Techniques,” the entire contents of which are incorporated by reference here in their entirety. 
     
    
     TECHNICAL FIELD 
       [0002]    The techniques described herein generally relate to data resource management, and in particular to preventing a user from exceeding data resource limits. 
       BACKGROUND 
       [0003]    A Copy Data Storage system can protect lots of applications with various different complex Service Level Agreements (SLAs). In such an environment, it is easy to set up policies that use too much limited resources, resulting in missing SLAs, or putting system at risk, as the system does not have enough resource to handle the load specified by the SLAs, either due to too many applications, operate the backups too frequently, or keeping the backups for too long. The critical system resource could be storage pool, volumes, or any other limited resources. Traditionally, users wait for the system to break, and look for the cause. 
       SUMMARY 
       [0004]    The disclosed subject matter includes a computerized method of preventing a user from configuring a service level agreement from creating a data management schedule that creates a set of data backups that exceeds data resource limits available for storing the set of data backups. The method is executed by a processor in communication with memory storing instructions configured to cause the processor to receive first data indicative of a schedule to perform a backup of at least one application. The memory stores instructions configured to cause the processor to determine a first amount of pool resources associated with the backup of each of the at least one application according to the received schedule. Determining the first amount of pool resources includes calculating a number of copies of an application associated with the received schedule, determining a change rate parameter comprising at least one of an application specific change rate associated with historical backup data corresponding to each of the at least one application, a system-wide change rate corresponding to change rates associated with applications similar to each of the at least one application, and a generic application change rate, and multiplying the change rate parameter for each the at least one application with a size of the application, and with a number of copies of the application associated with each of the at least one application. The memory stores instructions configured to cause the processor to add the first amount of pool resources for each of the at least one application to form an aggregate amount of pool resources. The memory stores instructions configured to cause the processor to determine a first amount of data volumes associated with the backup of each of the at least one application according to the received schedule. Determining the first amount of data volumes includes determining a second amount of data volumes associated with each copy of the at least one application, and multiplying the second amount of data volumes with the number of copies of the application associated with the received schedule. The memory stores instructions configured to cause the processor to add the first amount of data volumes for each of the at least one application to form an aggregate amount of data volume resource. The memory stores instructions configured to cause the processor to transmit a resource shortage warning when the aggregate amount of pool resources exceeds an available amount of pool resources or the aggregate amount of data volume resource exceeds an available amount of data volume resource, thereby preventing a user from configuring a service level agreement that exceeds data resource limits. 
         [0005]    The disclosed subject matter includes a computing system for preventing a user from configuring a service level agreement from creating a data management schedule that creates a set of data backups that exceeds data resource limits available for storing the set of data backups. The computing system includes a processor and a memory coupled to the processor. The memory includes instructions that, when executed by the processor, cause the processor to receive first data indicative of a schedule to perform a backup of at least one application. The memory includes instructions that, when executed by the processor, cause the processor to determine a first amount of pool resources associated with the backup of each of the at least one application according to the received schedule. Determining the first amount of pool resources includes calculating a number of copies of an application associated with the received schedule, determining a change rate parameter comprising at least one of: an application specific change rate associated with historical backup data corresponding to each of the at least one application, a system-wide change rate corresponding to change rates associated with applications similar to each of the at least one application, and a generic application change rate, and multiplying the change rate parameter for each the at least one application with a size of the application, and with a number of copies of the application associated with each of the at least one application. The memory includes instructions that, when executed by the processor, cause the processor to add the first amount of pool resources for each of the at least one application to form an aggregate amount of pool resources. The memory includes instructions that, when executed by the processor, cause the processor to determine a first amount of data volumes associated with the backup of each of the at least one application according to the received schedule. Determining the first amount of data volumes includes determining a second amount of data volumes associated with each copy of the at least one application, and multiplying the second amount of data volumes with the number of copies of the application associated with the received schedule. The memory includes instructions that, when executed by the processor, cause the processor to add the first amount of data volumes for each of the at least one application to form an aggregate amount of data volume resource. The memory includes instructions that, when executed by the processor, cause the processor to transmit a resource shortage warning when the aggregate amount of pool resources exceeds an available amount of pool resources or the aggregate amount of data volume resource exceeds an available amount of data volume resource, thereby preventing a user from configuring a service level agreement that exceeds data resource limits. 
         [0006]    The disclosed subject matter includes a non-transitory computer readable medium having executable instructions operable to cause an apparatus to receive first data indicative of a schedule to perform a backup of at least one application. The non-transitory computer readable medium has executable instructions operable to cause an apparatus to determine a first amount of pool resources associated with the backup of each of the at least one application according to the received schedule. Determining the first amount of pool resources includes calculating a number of copies of an application associated with the received schedule, determining a change rate parameter including at least one of: an application specific change rate associated with historical backup data corresponding to each of the at least one application, a system-wide change rate corresponding to change rates associated with applications similar to each of the at least one application, and a generic application change rate, and multiplying the change rate parameter for each the at least one application with a size of the application, and with a number of copies of the application associated with each of the at least one application. The non-transitory computer readable medium has executable instructions operable to cause an apparatus to add the first amount of pool resources for each of the at least one application to form an aggregate amount of pool resources. The non-transitory computer readable medium has executable instructions operable to cause an apparatus to determine a first amount of data volumes associated with the backup of each of the at least one application according to the received schedule. Determining the first amount of data volumes includes: determining a second amount of data volumes associated with each copy of the at least one application; and multiplying the second amount of data volumes with the number of copies of the application associated with the received schedule. The non-transitory computer readable medium has executable instructions operable to cause an apparatus to add the first amount of data volumes for each of the at least one application to form an aggregate amount of data volume resource. The non-transitory computer readable medium has executable instructions operable to cause an apparatus to transmit a resource shortage warning when the aggregate amount of pool resources exceeds an available amount of pool resources or the aggregate amount of data volume resource exceeds an available amount of data volume resource, thereby preventing a user from configuring a service level agreement that exceeds data resource limits. 
         [0007]    These and other capabilities of the disclosed subject matter will be more fully understood after a review of the following figures, detailed description, and claims. It is to be understood that the phraseology and terminology employed herein are for the purpose of description and should not be regarded as limiting. 
     
    
     
       BRIEF DESCRIPTION OF FIGURES 
         [0008]    Various objectives, features, and advantages of the disclosed subject matter can be more fully appreciated with reference to the following detailed description of the disclosed subject matter when considered in connection with the following drawings, in which like reference numerals identify like elements. 
           [0009]      FIG. 1  is a system diagram showing a backup application, according to some embodiments. 
           [0010]      FIG. 2  is a system diagram showing a backup application with guardrails  200 , according to some embodiments. 
           [0011]      FIG. 3  is a flowchart illustrating a computerized method for performing application profile updates through the application profile update scheduler, according to some embodiments. 
           [0012]      FIG. 4  is a flowchart illustrating the computerized process of potentially warning user about exceeding resource limit when protecting applications, changing application protection with different SLA and/or profile, or changing SLAs, according to some embodiments. 
           [0013]      FIG. 5  is a flowchart illustrating the computerized process of warning a user on a different type of resources, pools, when protecting applications, or changing application protection with different SLA and/or profile, according to some embodiments. 
           [0014]      FIG. 6  illustrates an exemplary apparatus configured to provide guardrails for copy data storage, according to some embodiments. 
       
    
    
     DETAILED DESCRIPTION 
       [0015]    A better approach is to analyze the SLAs and applications that they protect and to warn the user if the overall protection will exceed the system limit (e.g., data storage space allowable for the overall protection configured by the SLAs), before the user actually exceeds the system limit. This allows user to plan ahead for future expansion and guard against over allocating resources on the system. 
         [0016]    In the following description, numerous specific details are set forth regarding the systems and methods of the disclosed subject matter and the environment in which such systems and methods may operate, etc., in order to provide a thorough understanding of the disclosed subject matter. It will be apparent to one skilled in the art, however, that the disclosed subject matter may be practiced without such specific details, and that certain features, which are well known in the art, are not described in detail in order to avoid unnecessary complication of the disclosed subject matter. In addition, it will be understood that the embodiments provided below are exemplary, and that it is contemplated that there are other systems and methods that are within the scope of the disclosed subject matter. 
         [0017]      FIG. 1  is a system diagram showing a backup application  100 , according to some embodiments. Backup application  100  includes an application specific module  102 , SLA store  104 , resource profile store  106 , protection catalog store  108 , history store  110 , service policy engine  112 , service policy scheduler  114 , data movement requestor/monitor  116 , and object manager data and mover  118 . 
         [0018]    Service policy engine,  112 , with the service policy scheduler  114 , determines when to perform a backup, based on information in protection catalog store  108  that protects the application. Then using policy information from SLA store  104 . At which time, it communicates with application specific module, to place the application in a consistency state, if necessary, then send the changed data to service policy engine. Service policy engine stores the changed data at appropriate pool, which is specified in resource profile store  106 , using object manager data and mover  118 . At the successful completion of the job, information about the job is stored in history store  110 . Additional details regarding an exemplary backup application can be found in U.S. application Ser. No. 13/920,950, filed on Jun. 18, 2013, entitled “System and Method for Intelligent Database Backup,” the disclosure of which is herein incorporated by reference in its entirety. 
         [0019]      FIG. 2  is a system diagram showing a backup application with guardrails  200 , according to some embodiments. Backup application with guardrails  200  includes an application specific module  102 , SLA store  104 , resource profile store  106 , protection catalog store  108 , history store  110 , service policy engine  112 , application discovery service  202 , application profile update module  204 , application profile update scheduler  206 , and application profile store  208 . 
         [0020]    An application is discovered through application discovery service  202 , which communicates with application specific module  102 , to discover information pertinent to the application, such as the size of the application, how application data can be retrieved during backup, the disk volumes that the applications reside, type of application, etc. This information is communicated back to application discovery service  202 , which then employs application profile update module  204 , to store the information in application profile store  208 . The application can then be protected by applying SLA from SLA store  104 , and resource profile store  106 , saving the protection details in protection store  108 . As the backup scheduler runs, information regarding the backup job is stored in history store  110 , which includes time of the backup, amount of data copied, communication mechanism, etc. 
         [0021]    In some embodiments, the application profile update scheduler  206  can be configured to run regularly, such as once a day. As it runs, it examines the information of the successful backups, and compares the information with previously collected application details. There can be a difference between information of the successful backups and information from previously collected application details, as details may not be available at the time of discovery, or the application details may have changed, such as the size may have changed significantly. The details in application profile store  208  are then updated with the latest information through application profile update module  204 . 
         [0022]      FIG. 3  is a flowchart illustrating a computerized method for performing application profile updates through the application profile update scheduler  206 , according to some embodiments. The schedule typically runs once a day, but can be run on a different frequency. When it starts, it examines each of the latest successful backups of an application  302 , which will have details of the backup operation. In step  304 , the details of the backup operation (e.g., backup job statistics) are collected. Application profiles are updated from the actual backup operation  306 . Backup operation details are then compared with the saved information of the application  308 . If they are the same, the scheduler  206  checks if there are any further backup job histories  312 . If backup operation details are not the same, application information is updated with the detail from latest backup  310 . Application information is related to static information about the application, such as the details about the volumes that makes up the application, including the size and universally unique identifier (UUID) of the volume. Also included are number of volumes, and how the data is captured, etc. There can be other information regarding the backup that is also useful (e.g., how long the backup takes, CPU resources needed to complete a task).). In addition, number of bytes copied is also collected for each backup. Number of bytes copied can be related to data changed in the application. This metric can be used to calculate the change rate. As more backups are collected, the average of these backups informs the change rate of the application, which can be saved as part of the application details. Other information like the dedup efficiency (data reduced through deduplication), and the compression efficiency (data reduced through compression), can also be gathered if the operation is a dedup backup operation. This information can be helpful in determining how much storage is needed for future backup jobs. Next, a check is performed to determine if there are any more backup job histories  312 . If yes, the process continues until all protected applications are examined. If not, application profile update scheduler  206  sleeps until the next schedule  314 . 
         [0023]    Calculating the resource required depends on the type of resources in the Copy Data Storage system. First we need to calculate number of copies per volume for a given SLA. Each SLA is made up of multiple policies, which has a Recovery Point Objective (RPO), and a retention period. It may also contain repeat interval and exclusion interval, and window of operation. With retention, and RPO, we can calculate number of copies required per day. For example, an RPO of 3 hours, will produce 8 copies if window of operation is 24 hours, which is easily calculated by (window-of-operation−1)/RPO+1, which is (24−1)/3+1=8. But if the window of operation is between 9 am to 5 pm, which is (8−1)/3+1=3 the number will be 3 copies per day. Retention time is then multiplied to produce the total copies. To apply selection (such as only weekday), and/or exclusion, we can simply create a pseudo-calendar, and mark off the selection, and exclusion to produce the projected copies required maintaining the SLA. 
         [0024]      FIG. 4  is a flowchart illustrating the computerized process of potentially warning user about exceeding resource limit when protecting applications, changing application protection with different SLA and/or profile, or changing SLAs, according to some embodiments. This is achieved by examining each application  402 . For each application, use data from the protection detail (if protection is not changed), or use the new protection detail (if changed), which provides the resource profile and SLA used to protect this application,  406 . The resource required for protecting this application can then be calculated,  408 . The required resource for this application is added to the sum,  410 . This is repeated for each protected application,  412 . Once all resources are added together, it is compared with system resource limit,  414 . If this resource limit exceeded user-defined threshold, a warning is sent to user, warning about the potential resource shortage,  416 . 
         [0025]    One of the critical resources of a Copy Data Storage system is the number of volumes available in the system, or it can be a licensed limit. For each application, number of volumes for the application can be discovered during discovery, in addition, a base copy may be required if there is no storage port available. The number of volumes required to protect this application is number of volumes multiplied by the number of copies calculated from the SLA, and add the number of volumes if base copy is required. This is repeated for all protected applications. The sum of which is the projected volume count required during steady state on the system. This can then be compared with the system limit. If it exceeds a preset limit, a warning can then be issued to customers, before the change takes effect. In some embodiments, the resource limit is a hard limit and cannot be exceeded (e.g., storage associated with a physical device). In some embodiments, there is no physical resource limit (e.g., virtual machine), and the limit can be set by an administrator. Types of resources include storage pools, volumes, CPU resources, etc. 
         [0026]      FIG. 5  is a flowchart illustrating the computerized process of warning a user on a different type of resources, pools, when protecting applications, or changing application protection with different SLA and/or profile, according to some embodiments. The device (e.g., the computer illustrated in  FIG. 6 ) retrieves the application information,  502 . The device looks up the SLA used to protect the application,  504 . The device uses data from the protection detail  504 , to calculate an estimated number of copies for this SLA,  506 . For example, the device can calculate the estimated number of copies over a certain period of time based on the specified retention period(s) specified in the SLA. The device looks up the historical data for change rate of this application,  508 . If the system has built up enough historical data (e.g., 10 or more successful backups, or some other number above a pre-determined threshold), the system can use the application specific change rate,  510 . For example, if the change rate is approximately 3% per month for several months, the application specific change rate is assumed to be and can be set to 3%. 
         [0027]    If there is not enough historical data (or none available) at step  508 , the computerized method examines the similar application types for a system-wide change rate,  512 . For example, Oracle database and a SQL database can be considered similar application types. If there is enough historical data for a statistically significant threshold (e.g., 10 or more successful backups, or some other pre-configurable threshold) for this application type, use the change rate for similar application type,  514 . If there is not enough data, the system proceeds to use a generic change rate, which is collected over time from many customers with a similar application,  516 . For example, in situations where the specific change rate cannot be accurately abstracted (e.g., where there is not enough historical data), generalized trends can be used to set a change rate. For example, in general, some applications change more frequently than other applications (e.g., databases as compared to file systems). 
         [0028]    The system multiplies the change rate (e.g., from either step  508 ,  512  or  516 ) by a number of copies from the SLA calculation to calculate a pool resource requirement  518 . In cases where an entire copy of the application is needed (e.g., for a first copy in an out of band copy operation), it is necessary to add a base copy to the application size. If there are no more protected applications  520 , the method looks up the resource profile that is used to protect the application; this is the steady state pool resource required to protect this application. For example, assume that an application is 100 GB. Assume further that the change rate is 3%, each copy requires 3 GB. If the service level template requires 10 copies for steady state, then the steady state resource equals 30 GB. When a base copy is also needed, the steady state resource would equal the base copy size plus the copy size (i.e., 100 GB+30 GB), which is 130 GB. By summing up the total steady state pool requirements for all of the protected application(s), the method can compare the total steady state pool requirements with the configured pool size, to see if the required storage exceeds the warning level for the configured pool  522 . For example, the warning level can be a pre-configured amount of storage remaining in the pool (e.g., less than 10% space available, less than 20% space available, etc.) A warning can then be issued to users, warning them about the pool resource shortage if the policy change is left in place  524 . 
         [0029]      FIG. 6  illustrates an exemplary apparatus configured to provide guardrails for copy data storage, according to some embodiments. 
         [0030]      FIG. 6  illustrates an exemplary computerized apparatus  6000  configured to provide Guardrails for a CDS, according to some embodiments. Computerized apparatus  6000  includes a processor  6001  in communication with temporary and/or permanent memory  6002 . Memory  6002  stores instructions and is configured to cause the processor  6001  to perform the subject matter described herein, including the apparatus disclosed in  FIGS. 2 and 3 , and the computerized methods disclosed in  FIGS. 4 and 5 . As described further herein, the memory  6002  can be flash memory, a magnetic disk drive, an optical drive, a programmable read-only memory (PROM), a read-only memory (ROM), or any other memory or combination of memories. The processor  6001  can be a general purpose processor and/or can also be implemented using an application specific integrated circuit (ASIC), programmable logic array (PLA), field programmable gate array (FPGA), and/or any other integrated circuit. The computerized apparatus  6000  can include a database that may also be flash memory, a magnetic disk drive, an optical drive, a programmable read-only memory (PROM), a read-only memory (ROM), or any other memory or combination of memories. The computerized apparatus  6000  can execute an operating system that can be any operating system, including a typical operating system such as Windows, Windows XP, Windows 7, Windows 8, Windows Mobile, Windows Phone, Windows RT, Mac OS X, Linux, VXWorks, Android, Blackberry OS, iOS, Symbian, or other OSs. 
         [0031]    The same process can be used to warn user about potential over allocation of a dedup pool. The same calculation is applied. As deduplication can further reduce the amount of space required to protect an application of the changes. Further reduction of the change rate can be applied with historical deduplication efficiency, to be used for change rate. The same approach is employed; first check the application specific dedup rate, next application type dedup rate, and lastly, system wide application type dedup rate. The same can be applied if compression is used by further applying the compression ratio. 
         [0032]    The subject matter described herein can be implemented in digital electronic circuitry, or in computer software, firmware, or hardware, including the structural means disclosed in this specification and structural equivalents thereof, or in combinations of them. The subject matter described herein can be implemented as one or more computer program products, such as one or more computer programs tangibly embodied in an information carrier (e.g., in a machine readable storage device), or embodied in a propagated signal, for execution by, or to control the operation of, data processing apparatus (e.g., a programmable processor, a computer, or multiple computers). A computer program (also known as a program, software, software application, or code) can be written in any form of programming language, including compiled or interpreted languages, and it can be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. A computer program does not necessarily correspond to a file. A program can be stored in a portion of a file that holds other programs or data, in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files that store one or more modules, sub programs, or portions of code). A computer program can be deployed to be executed on one computer or on multiple computers at one site or distributed across multiple sites and interconnected by a communication network. 
         [0033]    The processes and logic flows described in this specification, including the method steps of the subject matter described herein, can be performed by one or more programmable processors executing one or more computer programs to perform functions of the subject matter described herein by operating on input data and generating output. The processes and logic flows can also be performed by, and apparatus of the subject matter described herein can be implemented as, special purpose logic circuitry, e.g., an FPGA (field programmable gate array) or an ASIC (application specific integrated circuit). 
         [0034]    Processors suitable for the execution of a computer program include, by way of example, both general and special purpose microprocessors, and any one or more processor of any kind of digital computer. Generally, a processor will receive instructions and data from a read only memory or a random access memory or both. The essential elements of a computer are a processor for executing instructions and one or more memory devices for storing instructions and data. Generally, a computer will also include, or be operatively coupled to receive data from or transfer data to, or both, one or more mass storage devices for storing data, e.g., magnetic, magneto optical disks, or optical disks. Information carriers suitable for embodying computer program instructions and data include all forms of nonvolatile memory, including by way of example semiconductor memory devices, (e.g., EPROM, EEPROM, and flash memory devices); magnetic disks, (e.g., internal hard disks or removable disks); magneto optical disks; and optical disks (e.g., CD and DVD disks). The processor and the memory can be supplemented by, or incorporated in, special purpose logic circuitry. 
         [0035]    To provide for interaction with a user, the subject matter described herein can be implemented on a computer having a display device, e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor, for displaying information to the user and a keyboard and a pointing device, (e.g., a mouse or a trackball), by which the user can provide input to the computer. Other kinds of devices can be used to provide for interaction with a user as well. For example, feedback provided to the user can be any form of sensory feedback, (e.g., visual feedback, auditory feedback, or tactile feedback), and input from the user can be received in any form, including acoustic, speech, or tactile input. 
         [0036]    The subject matter described herein can be implemented in a computing system that includes a back end component (e.g., a data server), a middleware component (e.g., an application server), or a front end component (e.g., a client computer having a graphical user interface or a web browser through which a user can interact with an implementation of the subject matter described herein), or any combination of such back end, middleware, and front end components. The components of the system can be interconnected by any form or medium of digital data communication, e.g., a communication network. Examples of communication networks include a local area network (“LAN”) and a wide area network (“WAN”), e.g., the Internet. 
         [0037]    It is to be understood that the disclosed subject matter is not limited in its application to the details of construction and to the arrangements of the components set forth in the following description or illustrated in the drawings. The disclosed subject matter is capable of other embodiments and of being practiced and carried out in various ways. Also, it is to be understood that the phraseology and terminology employed herein are for the purpose of description and should not be regarded as limiting. 
         [0038]    As such, those skilled in the art will appreciate that the conception, upon which this disclosure is based, may readily be utilized as a basis for the designing of other structures, methods, and systems for carrying out the several purposes of the disclosed subject matter. 
         [0039]    Although the disclosed subject matter has been described and illustrated in the foregoing exemplary embodiments, it is understood that the present disclosure has been made only by way of example, and that numerous changes in the details of implementation of the disclosed subject matter may be made without departing from the spirit and scope of the disclosed subject matter.