Method, device and computer program product for managing data replication

Embodiments of the present disclosure provide a method, a device and a computer program product for managing data replication. According to example implementations of the present disclosure, a replication policy model associated with data replication of a source device can be obtained, which is determined based on historical status information of the source device and a historical replication policy corresponding to the historical status information; current status information of the source device is determined, wherein the current status information indicates status information associated with pending data replication of the source device; and a target replication policy is determined based on the replication policy model and the current status information, which indicates a replication policy to be applied for performing the pending data replication. Therefore, the replication policy can be adjusted automatically based on the status of the source device, enabling a more efficient and intelligent data replication of the source device.

CROSS-REFERENCE TO RELATED APPLICATION

This patent application claims priority, under 35 U.S.C. § 119, of Chinese Patent Application No. 201811251871.5, filed Oct. 25, 2018, entitled “METHOD, DEVICE AND COMPUTER PROGRAM PRODUCT FOR MANAGING DATA REPLICATION.”

FIELD

Embodiments of the present disclosure generally relate to the field of data storage, and more specifically, to a method, a device and a computer program product for managing data replication.

BACKGROUND

Data backup refers to a procedure of replicating all or a part of a dataset from a storage system of an application host to other storage media, to prevent data loss resulting from operation errors or system failure. Typically, data protection is provided through data backup. With growing demands on data protection, the size of the dataset to be replicated is increased accordingly.

A current backup/recovery solution manages a replication policy at a user end through the following steps: a user manually sets a schedule for replicating and length of time for retaining replicated data at a server end based on a replication policy formulated according to needs; and the user updates the replication policy manually when necessary. It can be seen that the current solution of managing data replication depends on a manual input and therefore the efficiency thereof is relatively low. In addition, the flexibility and adaptability of the current solution are low. For example, the replication policy cannot be updated automatically when importance of data at the user end is changed, resulting in failing to protect significant data in time.

SUMMARY

Embodiments of the present disclosure provide a method, a device and a computer program product for managing data replication.

In a first aspect of the present disclosure, there is provided a method of managing data replication, comprising: obtaining a replication policy model associated with data replication of a source device, the replication policy model being determined based on historical status information of the source device and a historical replication policy corresponding to the historical status information; determining current status information of the source device, wherein the current status information indicates status information associated with pending data replication of the source device; and determining a target replication policy based on the replication policy model and the current status information, the target replication policy indicating a replication policy to be applied for performing the pending data replication.

In a second aspect of the present disclosure, there is provided a device for managing data replication, comprising at least one processing unit and at least one memory. The at least one memory is coupled to the at least one processing unit and stores instructions executable by the at least one processing unit. The instructions, when executed by the at least one processing unit, cause the device to perform acts comprises: obtaining a replication policy model associated with data replication of a source device, the replication policy model being determined based on historical status information of the source device and a historical replication policy corresponding to the historical status information; determining current status information of the source device, where the current status information indicates status information associated with pending data replication of the source device; and determining a target replication policy, based on the replication policy model and the current status information, the target replication policy indicating a replication policy to be applied for performing the pending data replication.

In a third aspect of the present disclosure, there is provided a computer program product. The computer program product is tangibly stored on a non-transient computer readable medium and includes a machine executable instruction, and the machine executable instructions which, when executed, causes a machine to implement any step of the method according to the first aspect of the present disclosure.

DETAILED DESCRIPTION OF EMBODIMENTS

Preferred embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings. Although the drawings illustrate preferred embodiments of the present disclosure, it would be appreciated that the present disclosure may be implemented in various manners but should not be construed as being limited by the embodiments illustrated herein. Rather, these embodiments are provided to disclose the present disclosure more thoroughly and completely, and to convey the scope of the present disclosure fully to those skilled in the art.

As used herein, the term “includes” and its variants are to be read as open-ended terms that mean “includes, but is not limited to.” The term “or” is to be read as “and/or” unless the context clearly indicates otherwise. The term “based on” is to be read as “based at least in part on.” The term “an example embodiment” and “an embodiment” are to be read as “at least one example embodiment.” The term “another embodiment” is to be read as “at least another embodiment.” The terms “first,” “second,” and the like may refer to different or the same objects. Other definitions, either explicit or implicit, may be included below.

FIG. 1illustrates a schematic diagram of architecture100that may implement embodiments of the present disclosure. It would be appreciated that the structure and functionality of the architecture100as shown inFIG. 1are provided merely as an example, without implying any limitation to the scope of the present disclosure. Embodiments of the present disclosure may be implemented in different structures and/or architectures.

As shown inFIG. 1, the architecture100includes a source device110and a target device130. The source device110may also be referred to as user end, and the target device130may also be referred to as server end for storing data from the source device110. In some embodiments, the source device110and the target device130may communicate via a network. Data115to be replicated in the source device110is replicated periodically to the target device130in a unit of data block, to accomplish the purpose of data protection. The size of the data block, for example, is 10 GB, and it would be appreciated that it is provided only as an example, without limitation, and the size of the data block may be of other numerical value.

As described above, in the current backup/recovery solution, a user of the source device110predetermines a replication policy based on the need, the replication policy specifies occurrence time of replication and a length of time that replicated data is retained in the target device130; and the policy is stored in the target device130. It can be seen that, the replication policy is fixed. When a user's demand is changed, it takes the user a lot of time to configure a new replication policy. Moreover, when the status of the source device110is changed (for example, data are updated frequently), replication cannot be started automatically in advance according to the fixed replication policy, resulting in failure to protect data in time.

A solution of managing data replication is provided by embodiments of the present disclosure. The solution takes historical demands of a user and current status information of the source device110into account, and adjusts the replication policy automatically based on the information. In this way, it not only saves a lot of configuration time for a user, but also provides more intelligent, more efficient protection for user data. According to embodiments of the present disclosure, a computing device120is provided in the architecture100, and the functionality/operation of the computing device120will be described below in detail with reference toFIG. 2. Preferably, the computing device120is implemented outside of the source device110and the target device130.

FIG. 2illustrates a flowchart of a method200of managing data replication according to embodiments of the present disclosure. For example, the method200may be performed by the computing device120as shown inFIG. 1. Acts involved in the method200will be described below with reference toFIG. 1. It would be appreciated that the method200may also include additional acts not shown and/or may skip the acts shown, and the scope of the present disclosure is not limited in this aspect.

At block210, the computing device120may obtain a replication policy model associated with a data replication of the source device110, and the replication policy model may be determined based on historical status information of the source device110and a historical replication policy corresponding to the historical status information, which is substantially a model of a relation between the status information and the replication policy. Only as an example with no limitation, the replication policy model may be a machine learning model, for example, a decision tree model, an artificial neural network model etc.

According to some embodiments, the computing device120may obtain, from the source device110, historical status information and a historical replication policy corresponding thereto, which are collectively referred to as historical data105, as shown inFIG. 1. The historical status information indicates status information associated with the data replication of the source device110performed previously, while the historical replication policy indicates a replication policy adopted for data replication performed previously, given the historical status information. Alternatively, the computing device120preprocesses the historical data105. The computing device120trains the replication policy model by taking the preprocessed historical status information and historical replication policy as sample status information and a sample replication policy, respectively. The trained replication policy model can reflect a user's historical needs precisely, i.e., a user formulate historical replication policies for different statuses of the source device100. Alternatively, the trained replication policy model is stored and may be accessed by the computing device120. This implement is more efficient.

At block220, the computing device120may determine the current status information of the source device110; the current status information indicates status information associated with pending data replication of the source device110. Preferably, the current status information and the historical status information include the same information items (which are also referred to as attributes), e.g., a data size of the pending data replication, a resource utilization rate of the source device110, time consumption for historical data replication etc.

According to some embodiments, the computing device120collects status information125of the source device110within a predetermined period of time before the pending data replication, as shown inFIG. 1. For example, after the last data replication starts, the computing device collects the status information125of the source device110a plurality of times in a certain frequency, for example, collecting once every day. Alternatively, the computing device120preprocesses the collected status information125. The computing device120may determine the current status information based on the preprocessed status information. For example, the computing device120may compute a mean of values for the status information, e.g., geometric mean, harmonic mean, weight mean, quadratic mean and the like. As such, the status of the source device110within the period of time may be considered more thoroughly, so that the current status of the source device110can be determined more accurately.

According to some embodiments, the computing device120may determine the data size of the pending data replication of the source device110, as the current status information. Only as an example with no limitation, the data size, for example, is a size of data to be replicated at the source device110, a newly increased data size as compared to the previous replication, an average size of files and the like. The data size, for example, may be represented in unit of bytes. By determining an average size of the files, the computing device120can determine a number of files to be replicated. The more the number of files, the more the time consumption for the replication.

According to another embodiment, the computing device120may determine the resource utilization rate of the source device110as the current status information, including, for example, computing resources, storage resources and network resources. The computing resource utilization rate may be related to the CPU utilization rate of the source device110. The storage resource utilization rate may be related to a memory utilization rate of the source device110or I/O utilization. The network resource utilization rate may be related to the network bandwidth of the source device110and round-trip time (RTT). A high resource utilization rate of the source device110may indicate that data of the source device100is probably updated frequently, and may be required to perform data replication more frequently so as to protect data in time.

According to a further embodiment, the computing device120may determine time consumption for historical data replication before the pending data replication, as the current status information. For example, the computing device120may determine the time consumption for the last data replication, and thus estimate time consumption for the pending data replication.

At block230, the computing device120determines a target replication policy135based on the replication policy model obtained at block210and the current status information obtained at block220, and the target replication policy135indicates a replication policy adopted for the pending data replication. Only served as an example with no limitation, the computing device120takes the current status information as an input of the replication policy model, and takes an output obtained from the replication policy model as the target replication policy135.

According to some embodiments, the computing device120may determine a start time for the pending data replication. For example, when determining that the resource utilization rate of the source device110is high, the computing device120may advance the start time automatically according to the replication policy model, so as to make the data replication in time. As another example, when determining that the pending data replication of the source device110will take a comparatively long time, the computing device120may also advance the start time automatically according to the replication policy model, to avoid too much time consumption for replication due to too large data, and to prevent user experience from being affected.

According to another embodiment, the computing device120may determine a length of time that the data115of the pending data replication retains in the target device130. For example, if need of a user is directed to retaining the data updated frequently for a shorter time duration, and the need is reflected in historical data (for example, the user which is an instant messaging application company requires to retain the data updated frequently (for example, during holidays) for a short time), then the computing device120, when determining that the resource utilization rate of the source device110is high, may automatically set the length of time for retaining the data115in the target device relatively short according to the replication policy model.

According to some embodiments, the computing device120may update the replication policy model with the current status information and the target replication policy135.FIG. 3illustrates a schematic diagram of a procedure300for generating the target replication policy135according to embodiments of the present disclosure. As shown, the computing device120receives data315from the source device110, where the data includes the historical data105and the status information125shown inFIG. 1. Alternatively, at block310, the computing device120preprocesses the data315. At block320, the computing device120trains the replication policy model with the preprocessed historical data325as sample status information and a sample replication policy.

At block330, the computing device120applies the trained replication policy model to the determined current status data335, so as to obtain the target replication policy135. In some embodiments, the computing device120adds the current status information335and the target replication policy135to the sample status information and the sample replication policy, and retrains the replication policy model, i.e., returns to block320, with the updated sample status information and sample replication policy. By retraining using expanded samples, the replication policy model can become more robust, enabling the computing device120to formulate the target replication policy135more in line with the need of user, based on the current status information. In another embodiment, the replication policy model is not retrained entirely, but tuned finely with newly added sample data, so that updating of the model becomes more efficient.

According to some embodiments, a user may be provided with options of using the technical solution according to the present invention and the legacy solution. If the user selects the legacy solution, the status information of the user is not collected, nor is its replication policy updated automatically.

The replication policy model may be a Hoeffding tree based very fast decision tree (VFDT) model, which will be introduced below in more details. It would be appreciated that, the VFDT model is only an example, and the replication policy model may also be based on ID3, ID4, ID5R, C4.5 and SLIQ learning algorithms, and the like.

The Hoeffding tree algorithm may be applied to classification of data stream. The effect of the Hoeffding tree is that: split attributes of nodes in the tree may be determined with fewer data samples, and how many data samples are required is determined by a Hoeffiding bound ϵ:

where, R is a range of a real number random variable r, and n is the number of independent variables of the observed variable r. Assumed thatris a mean value computed from n independent variables, the Hoeffiding bound ϵ specifies that, with a probability 1-δ, the true mean value of the variables is at leastr-ϵ. The Hoeffding bound is able to give the same results regardless of the probability distribution of the generated observations. However, the number of observations required to reach certain values of δ and ϵ are different across probability distributions. Pseudo code of the Hoeffding tree is provided in Table 1.

TABLE 1Pseudo code of Hoeffding TreeLet HT be a tree with a single leaf (the root)For all training samples doSort samples into leaf ι using HTUpdate sufficient statistics in ιIncrement nι, the number of examples seen at ιIf nlmod nmin=0 and examples seen at ι not all of same class thenComputeGι(Xι) for each attributeLet Xabe attribute with highestGιLet Xbbe attribute with second-highestGιCompute⁢⁢Hoeffding⁢⁢bound⁢⁢ϵ=R2⁢ln⁡(1/δ)2⁢⁢nlIf Xa≠XΦand (Gι(Xa) −Gι(Xb) > ϵ or ϵ< τ) thenReplace ι with an internal node that splits on XaFor all branches of the split doAdd a new leaf with initialized sufficient statisticsEnd forEnd ifEnd ifEnd for

The Hoeffding tree determines a minimum number of samples required by node split with the Hoeffding bound ϵ. The Hoeffding tree is generated by constantly replacing leaf nodes with branch nodes, i.e., each decision node retains an important statistical quantity, and split is made when the statistical quantity of the node reaches the Hoeffding bound. The Hoeffding tree can reach an accuracy rate of a general decision tree, and along with arriving of data in the data stream, the Hoeffding tree can continue building the tree when performing classification. The very fast decision tree (VFD) is a Hoeffding tree based improved algorithm, which discards useless nodes and attributes thereof, thereby having better storage utilization rate.

An example of the historical data325is provided in Table 2.FIG. 4illustrates a schematic diagram of a Hoeffding tree400built based on the historical data325. Various attributes in composite status information, e.g., an updated data size410-1, an average file size410-2and time410-1consumption for the last replication, are synthesized to obtain respective leaf nodes of the tree, i.e., various replication policies420-1,420-2,420-3and420-4. In some embodiments, daily average data of a user end is imported to the computing device120for incremental training, so as to enrich the current decision tree. For example, when the size of updated data of the user end is increased or I/O use is increased within a period of time, the computing device120may determine a new replication policy, thereby protecting the user data more safely and efficiently.

As can be seen from the above description, embodiments of the present disclosure can implement a dynamic, intelligent and automatic adjustment to a replication policy. Several factors of a user end may determine importance of data to a user, and the present technical solution combines these factors based on historical needs, to select an optimum replication policy for data backup of the user end. In this way, the user data can be protected more intelligently and more efficiently.

FIG. 5illustrates a schematic block diagram of an example device500that can be used to implement embodiments of the present disclosure. As shown, the device500includes a central processing unit (CPU)510which performs various appropriate acts and processing, according to a computer program instruction stored in a read-only memory (ROM)520or a computer program instruction loaded from a storage unit580to a random access memory (RAM)530. The RAM530stores therein various programs and data required for operations of the device500. The CPU510, the ROM520and the RAM530are connected via a bus540with one another. An input/output (I/O) interface550is also connected to the bus540.

A plurality of components in the device500are connected to the I/O interface550: an input unit560e.g., a keyboard, a mouse, etc.; an output unit570e.g., various kinds of displays and a loudspeaker, etc.; a storage unit580e.g., a magnetic disk, an optical disk, etc.; and a communication unit590e.g., a network card, a modem, and a wireless communication transceiver, etc. The communication unit590allows the device500to exchange information/data with other devices through a computer network such as an Internet and/or various kinds of telecommunications networks.

Various processes and processing described above, e.g., the method200or procedure300, may be executed by the processing unit510. For example, in some embodiments, the method200or procedure300may be implemented as a computer software program that is tangibly included in a machine readable medium, e.g., the storage unit580. In some embodiments, part or all of the computer programs may be loaded and/or mounted onto the device500via the ROM520and/or communication unit590. When the computer program is loaded to the RAM530and executed by the CPU510, one or more steps of the method200or procedure300as described above may be performed.

The present disclosure may be a method, an apparatus, a system, and/or a computer program product. The computer program product may include a computer readable storage medium having computer readable program instructions thereon for causing a processor to carry out aspects of the present disclosure.

Computer readable program instructions for carrying out operations of the present disclosure may be assembler instructions, instruction-set-architecture (ISA) instructions, machine instructions, machine related instructions, microcode, firmware instructions, state-setting data, or either source code or object code written in any combination of one or more programming languages, including an object oriented programming language such as JAVA, Smalltalk, C++ or the like, and conventional procedural programming languages, such as the “C” programming language or similar programming languages. The computer readable program instructions may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In ta scenario related to a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection may be made to an external computer (for example, through the Internet with an Internet Service Provider). In some embodiments, electronic circuitry including, for example, programmable logic circuitry, field-programmable gate arrays (FPGA), or programmable logic arrays (PLA) may execute the computer readable program instructions by utilizing state information of the computer readable program instructions to personalize the electronic circuitry, in order to perform aspects of the present disclosure.

These computer readable program instructions may be provided to a processor unit of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, when executed via the processing unit of the computer or other programmable data processing device, generate apparatus implementing the functions/acts specified in one or more blocks in the flowchart and/or block diagrams. These computer readable program instructions may also be stored in a computer readable storage medium that may direct a computer, a programmable data processing apparatus, and/or other devices to function in a particular manner, such that the computer readable storage medium having instructions stored therein includes an article of manufacture including instructions which implement aspects of the function/act specified in one or more blocks of the flowchart and/or block diagram.

The computer readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus or other device to produce a computer implemented process, such that the instructions which are executed on the computer, other programmable data processing apparatus, or other devices implement the functions/acts specified in one or more block of the flowchart and/or block diagram.

The flowchart and block diagrams illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to a plurality of embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, snippet, or portion of code, which includes one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in a reversed order, depending upon functionalities involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, may be implemented by special purpose hardware-based systems that perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.