CONCURRENT DATA TRANSMISSION

For concurrent data transmission, an example method includes acquiring a processor utilization rate related to transmission of historical data packet(s) and a bandwidth utilization rate of a data link within a target time period. A time consumption for transmitting a target data packet, a splitting processing resource related to a splitting operation, and a packaging processing resource related to a packaging operation are determined. Based on the processor utilization rate, the bandwidth utilization rate, the time consumption, the splitting processing resource, and the packaging processing resource, a split number is determined for the target data packet to concurrently transmit the target data packet. In this way, a data transmission speed can be improved, time consumed for transmitting the target data packet can be reduced, and meanwhile, it can also alleviate problems such as excessive processing resource consumption and overloaded operation of the processor caused by concurrent transmission.

RELATED APPLICATION

The present application claims the benefit of priority to Chinese Patent Application No. 202310907985.5, filed on Jul. 21, 2023, which application is hereby incorporated into the present application by reference herein in its entirety.

TECHNICAL FIELD

The present disclosure relates to the field of data storage, and more specifically, relates to concurrent data transmission.

BACKGROUND

A Recovery Point Objective (RPO) refers to a degree of data loss that an organization or system can tolerate in the event of a failure. The RPO is usually measured in time units, representing the time of the last valid data backup before a failure occurred. A lower RPO indicates that data is replicated and backed up at a higher frequency, and in the event of a failure, the organization or system will suffer data loss in a shorter period of time. A higher RPO indicates that the organization or system can tolerate data loss for a longer period of time.

Replication snapshot is a method of data backup and recovery in a computer system. Replication snapshot is to take a snapshot of data at a certain time point, and then replicate the snapshot to another location or storage system to provide data redundancy and fault recovery capability.

SUMMARY

Embodiments of the present disclosure propose a method, a device, and a computer program product for concurrent data transmission. In the embodiments of the present disclosure, statistical data such as processor utilization rate related to transmission of a historical data packet and bandwidth utilization rate of a data link within a past target time period can be acquired. Moreover, time consumption for transmitting a target data packet can be calculated, and a processing resource to be consumed for splitting a data packet in a source storage system and a processing resource to be consumed for packaging multiple data packets in a destination storage system can be determined. Then, an optimal split number can be calculated based on these data. Thus, when the target data packet is transmitted, the source storage system can split the target data packet into multiple data packets based on the calculated split number and concurrently transmit them to the destination storage system, and then the multiple data packets are packaged into the target data packet at the destination storage system. In this way, a target data packet can be concurrently transmitted, thereby improving a data transmission speed and reducing time consumed in transmitting the target data packet. At the same time, it can also alleviate the problems of excessive processing resource consumption and overloaded operation of the processor caused by concurrent transmission, so as to ensure an acceptable processor performance for the source storage system and the destination storage system while improving the data transmission speed.

In a first aspect of the embodiments of the present disclosure, a method for concurrent data transmission is provided. The method includes acquiring a processor utilization rate related to transmission of one or more historical data packets and a bandwidth utilization rate of a data link within a target time period. The method further includes determining time consumption for transmitting a target data packet, a splitting processing resource related to a splitting operation, and a packaging processing resource related to a packaging operation. The method further includes determining, based on the processor utilization rate, the bandwidth utilization rate, the time consumption, the splitting processing resource, and the packaging processing resource, a split number for the target data packet to concurrently transmit the target data packet.

In a second aspect of the embodiments of the present disclosure, an electronic device is provided. The electronic device includes one or more processors; and a storage apparatus used to store one or more programs, and when the one or more programs are executed by the one or more processors, enable the one or more processors to implement a method for concurrent data transmission. The method includes acquiring a processor utilization rate related to transmission of one or more historical data packets and a bandwidth utilization rate of a data link within a target time period. The method further includes determining time consumption for transmitting a target data packet, a splitting processing resource related to a splitting operation, and a packaging processing resource related to a packaging operation. The method further includes determining, based on the processor utilization rate, the bandwidth utilization rate, the time consumption, the splitting processing resource, and the packaging processing resource, a split number for the target data packet to concurrently transmit the target data packet.

In a third aspect of the embodiments of the present disclosure, a computer readable storage medium is provided, and stores a computer program thereon. The program, when being executed by a processor, implements a method for concurrent data transmission. The method includes acquiring a processor utilization rate related to transmission of one or more historical data packets and a bandwidth utilization rate of a data link within a target time period. The method further includes determining time consumption for transmitting a target data packet, a splitting processing resource related to a splitting operation, and a packaging processing resource related to a packaging operation. The method further includes determining, based on the processor utilization rate, the bandwidth utilization rate, the time consumption, the splitting processing resource, and the packaging processing resource, a split number for the target data packet to concurrently transmit the target data packet.

It should be understood that the content described in the Summary of the Invention part is neither intended to limit key or essential features of the embodiments of the present disclosure, nor intended to limit the scope of the present disclosure. Other features of the present disclosure will become readily understood from the following description.

DETAILED DESCRIPTION

The following will describe the embodiments of the present disclosure in more detail with reference to the accompanying drawings. Although the accompanying drawings show some embodiments of the present disclosure, it should be understood that the present disclosure may be implemented in various forms, and should not be explained as being limited to the embodiments stated herein. Rather, these embodiments are provided for understanding the present disclosure more thoroughly and completely. It should be understood that the accompanying drawings and embodiments of the present disclosure are for exemplary purposes only, and are not intended to limit the protection scope of the present disclosure.

In the description of the embodiments of the present disclosure, the term “include” and similar terms thereof should be understood as open-ended inclusion, that is, “including but not limited to.” The term “based on” should be understood as “based at least in part on.” The term “an embodiment” or “the embodiment” should be understood as “at least one embodiment.” The terms “first,” “second,” and the like may refer to different or identical objects. Other explicit and implicit definitions may also be included below.

Data replication is a process of creating copies or backups of data from a source storage system in a destination storage system, and data replication includes synchronous and asynchronous replication. Synchronous replication can create data copies in real time, but creating data copies in real time will have an adverse impact on host writes, and can only be achieved in scenarios with short transmission distances. Asynchronous replication is designed for data replication in scenarios with long transmission distances. Asynchronous replication can perform periodic replication in the background without negatively affecting the performance of a host. However, since the replication process is asynchronous instead of in real time, there may be some degree of data loss. A Recovery Point Objective (RPO) refers to a maximum acceptable amount of data loss in the event of a catastrophic event or data loss, and the RPO is represented by a period of time, such as 30 minutes, 1 hour, 2 hours, etc. If the RPO of a system is 1 hour, it means that an interval between the time of the disaster and the last data replication by the system does not exceed 1 hour.

In order to reduce the amount of data loss, users (such as system administrators) can choose a small RPO value (such as 5 minutes) and hope that in the event of a failure, only a small amount of data will be lost. However, when an I/O task is heavy and the source storage system has a large number of storage objects involving asynchronous replication sessions, such a situation may occur—when approaching the second RPO, transmission of the incremental data to be replicated by the first RPO from the source storage system to the destination storage system is not completed. In this case, if a disaster occurs during the second RPO, data from both RPOs may be lost. Therefore, it is necessary to improve the data transmission performance to ensure that transmission of all incremental data is completed within a time window required by the RPO.

Hence, an embodiment of the present disclosure proposes a solution for concurrently transmitting data. In the embodiment of the present disclosure, statistical data such as a processor utilization rate for transmitting a historical replication snapshot (which is also referred to as a historical data packet) and a bandwidth utilization rate of a data link within a past RPO (which is also referred to as target time period) can be acquired. Moreover, time consumption for transmitting a target data packet can be calculated, and a processing resource to be consumed for splitting a data packet in a source storage system and a processing resource to be consumed for packaging multiple data packets in a destination storage system can be determined. Then, an optimal split number can be calculated based on these data. Thus, when a target replication snapshot (which is also referred to as the target data packet) is transmitted, the source storage system can split the target replication snapshot into multiple data packets based on the calculated split number and concurrently transmit them to the destination storage system, and then the multiple data packets can be packaged into the target replication snapshot at the destination storage system.

In this way, a target replication snapshot can be concurrently transmitted, thereby improving a data transmission speed and improving bandwidth utilization rate of a data link, reducing a case of not completing replication snapshot transmission within RPO time. Meanwhile, it can also alleviate the problems of excessive processing resource consumption and overloaded operation of the processor caused by concurrent transmission, so as to ensure an acceptable processor performance for the source storage system and the destination storage system while improving the data transmission speed.

FIG.1shows a schematic diagram of an example environment100in which multiple embodiments of the present disclosure can be implemented therein. As shown inFIG.1, the environment100includes a source storage system102and a destination storage system112. The source storage system102includes storage objects104-1,104-2, . . . , and104-N (which are all referred to as a storage object104). The storage objects are a way of organizing and managing data in a storage system, which can be data blocks of any size, and can be files, database records, images, videos, and other types of data. In the environment100, the storage object104is replicated or backed up by means of asynchronous replication sessions. As shown inFIG.1, each storage object104has a related replication snapshot, i.e., replication snapshots106-1,106-2, . . . , and106-N (which are all referred to as a replication snapshot106).

As shown inFIG.1, in the environment100, a data link120can be used to transmit data between the source storage system102and the destination storage system112. For example, the data link120can be used to transmit the replication snapshot106from the source storage system102to the destination storage system112, so as to create replication snapshots116-1,116-2, . . . , and116-N (which are all referred to as a replication snapshot116) at the destination storage system as a transmission result. The data link120has a theoretical bandwidth, and the theoretical bandwidth refers to a maximum transmission speed supported by the data link120, indicates a maximum data amount that can be transmitted by the data link120in an ideal condition, and is often measured by the number of bits transmitted per second (bps). For example, if the theoretical bandwidth of the data link120is 100 Mbps, it indicates that it can transmit 100 megabits of data per second under ideal conditions. However, in fact, due to other factors (such as signal interference, bandwidth sharing, transmission delay, etc.) that may exist in the network, an actual transmission speed of 100 Mbps may not be achieved.

As shown inFIG.1, in the environment100, the source storage system102includes a processor108, and the destination storage system112includes a processor118. The processors108and118may execute operations related to storage management, such as data access, data compression and decompression, data replication, and some computation tasks. The capabilities and performance of the processors108and118can directly affect indexes such as the throughput and response time of the source storage system102and the destination storage system112.

FIG.2shows a schematic diagram of a process200of replicating incremental data based on RPO asynchronous transmission according to some embodiments of the present disclosure. As shown inFIG.2, in the process200, data to be replicated (that is, the replication snapshot212) in a source storage system (for example, the source storage system102inFIG.1) is replicated to a destination storage system (for example, the destination storage system112inFIG.1) at a time point1for the first time, and the first replication is also referred to as initial replication. After the initial replication is completed, incremental data will be replicated according to an RPO-based plan in the process200. Changes in these incremental data since the last replication will be monitored to achieve a faster replication speed and lower storage requirements. When a system disaster occurs, administrators can choose to restore the system to the time point of the last incremental replication to minimize data loss.

As shown inFIG.2, in the process200, replication incremental data will be transmitted at a time point2from the source storage system102to the destination storage system112at an interval of the RPO time. For example, at the time point2, a replication snapshot214is transmitted from the source storage system102to the destination storage system112in the process200. After the RPO time, at a time point3, a replication snapshot216is transmitted from the source storage system102to the destination storage system112in the process200. After another RPO time, at a time point4, a replication snapshot218is transmitted from the source storage system102to the destination storage system112in the process200. It should be noted that the replication snapshots214,216, and218may include one or more replication snapshots, and the number of the replication snapshots depends on newly added or modified data since the last replication.

FIG.3shows a flowchart of a method300for concurrent data transmission according to some embodiments of the present disclosure. As shown inFIG.3, in a block302, a method300includes acquiring a processor utilization rate related to transmission of one or more historical data packets and a bandwidth utilization rate of a data link within a target time period. For example, in the process200shown inFIG.2, the method300may include acquiring a processor utilization rate (for example, a processor utilization rate of the processor108of the source memory102and a processor utilization rate of the processor118of the destination storage system112inFIG.1) related to transmission of the replication snapshot214and a bandwidth utilization rate of a data link (for example, the data link120inFIG.1) at the time point3within the RPO time between the time point2to the time point3.

At a block304, the method300includes determining time consumption for transmitting a target data packet, a splitting processing resource related to a splitting operation, and a packaging processing resource related to a packaging operation. For example, in the environment100shown inFIG.1, time consumption for transmitting the replication snapshot106can be calculated based on a data size of the replication snapshot106and an average transmission speed of the data link120. In addition, the processor108of the source storage system102may perform the splitting operation to split a data packet into multiple data packets. In this text, an operation of splitting one data packet into two data packets is referred to as one splitting operation. The method300may include determining processor consumption required for performing one splitting operation by the processor108of the source storage system102. Besides, the processor118of the destination storage system112may perform the packaging operation to package multiple data packets into one data packet. In this text, an operation of packaging two data packets into one data packet is referred to as one packaging operation. The method300may include determining processor consumption required for performing one packaging operation by the processor118of the destination storage system112.

At a block306, the method300includes determining, based on the processor utilization rate, the time consumption, the bandwidth utilization rate, the splitting processing resource, and the packaging processing resource, a split number for the target data packet to concurrently transmit the target data packet. For example, in the process200shown inFIG.2, the method300may include determining the split number for the replication snapshot216based on the processor utilization rate, the time consumption, and the bandwidth utilization rate for transmitting the replication snapshot214within the RPO time between the time point2to the time point3, a processor resource required for performing one splitting operation by the processor108of the source storage system102, and a processor resource required for performing one packaging operation by the processor118of the destination storage system112. After the split number is determined, the source storage system102can split the replication snapshot216into a split number of sub data packets before transmitting the replication snapshot216, and then concurrently transmit these data packets. Then the destination storage system112can package the split number of sub data packets into a complete replication snapshot based on identifiers in the sub data packets for storage.

In this way, in the method300, a replication snapshot216can be concurrently transmitted so as to improve a data transmission speed and improve a bandwidth utilization rate of the data link120, and reduce the possibility of failing to complete the transmission of the replication snapshot216within the RPO time between the time point3and the time point4shown inFIG.2. Meanwhile, it can also alleviate the problems of excessive resource consumption of processors108and118of the source storage system102and the destination storage system112and overloaded operation of the processors caused by concurrent transmission, so as to ensure acceptable performance of the processors108and118for the source storage system102and the destination storage system112while improving the data transmission speed.

In some embodiments, to speed up the transmission speed, a to-be-transmitted target data packet can be split into multiple sub data packets at the source storage system, and the data link is used to concurrently transmit the multiple sub data packets to the destination storage system. After receiving the multiple sub data packets, the destination storage system can package the multiple sub data packets to obtain the recovered target data packet, so as to complete the process of replicating the target data packet from the source storage system to the destination storage system.

FIG.4AtoFIG.4Bshow a schematic diagram of a process400of splitting a replication snapshot at a source storage system according to some embodiments of the present disclosure, and a schematic diagram of a process420of packaging multiple split data packets at a destination storage system according to some embodiments of the present disclosure. As shown inFIG.4A, the source storage system402(for example, the source storage system102inFIG.1) stores a replication snapshot404. A processor408of the source storage system402can split the replication snapshot404into multiple data packets, i.e., data packets406-1,406-2, . . . , and406-N (which are all referred to as a data packet406). Then, a data link410is used to concurrently transmit multiple data packets406to a destination storage system412. It should be noted that when the processor408is used to split the replication snapshot into multiple data packets406, corresponding processor resources for splitting data packets need to be consumed, and the consumed processor resources depend on the number of times the data is split; the more times the split occurs, the more split data packets, and the higher the efficiency of concurrent transmission, the more the corresponding consumption of processor resources.

As shown inFIG.4B, after the destination storage system412receives multiple data packets406from the source storage system402via the data link410, the processor418of the destination storage system412can package the multiple data packets406so as to obtain a recovered replication snapshot414, so as to complete a process of replicating the replication snapshot404from the source storage system402to the destination storage system412. It should be noted that when the processor418is used to package multiple data packets406, corresponding processor resources for packaging data packets need to be consumed, and the consumed processor resources depend on the number of split sub data packets after the replication snapshot404is split into sub data packets at the source storage system402. The higher the number of split sub data packets is, the more times they need to be packaged, and correspondingly, the more processor resources are consumed.

In this way, the replication snapshot404can be split into multiple data packets406at the source storage system402, and then the multiple data packets406are transmitted concurrently via the data link410, so as to improve the bandwidth utilization rate of the data link410, accelerate the speed of replicating the replication snapshot404from the source storage system402to the destination storage system412, reduce the transmission time length, and improve the possibility of completing the transmission of replication incremental data within the RPO time.

The higher the number of split data packets, the higher the efficiency of the concurrent transmission, and the higher the speed of the transmission. However, as stated above, the more splitting times, the more processor resources need to be consumed for the splitting operation and the packaging operation. When the number of split data packets is too large, it will make the processor resources a bottleneck and even have a negative impact on the processor's execution of other tasks. Therefore, in order to complete the transmission of replication incremental data within the RPO time and limit the consumed processor resources to an acceptable range, it is necessary to determine the optimal split number for concurrent transmission of the replication incremental data.

In some embodiments, before determining an optimal split number, it can be determined whether to concurrently transmit the target data packet (for example, the target replication snapshot). In some embodiments, an average transmission speed of the data link can be determined, and the ratio of the average transmission speed to a theoretical bandwidth of the data link can be determined. Then, the ratio can be compared with a predetermined threshold, and the target data packet is directly transmitted without splitting in response to determining that the ratio is greater than or equal to the predetermined threshold.

In some embodiments, before determining the optimal split number, the average transmission speed of the data link can be determined, and a data size of the target data packet can be determined. Then, the transmission time length for transmitting the target data packet can be determined in a case where the target data packet is not split based on the average transmission speed and the data size of the target data packet. In response to determining that the transmission time length is less than or equal to the target time period, the target data packet is directly transmitted without splitting.

FIG.5shows a flowchart of a process500of determining whether to concurrently transmit a replication snapshot and determining a split number according to some embodiments of the present disclosure. As shown inFIG.5, in a block502, information is collected in the process500. For example, the collected information may include a real average transmission speed and a theoretical bandwidth of a data link (for example, the data link120inFIG.1), and may further include an RPO value of a replication session and time required for transmitting the target replication snapshot (for example, the replication snapshot216inFIG.2). In some embodiments, the processor utilization rate for transmitting a historical replication snapshot within certain RPO time in the past and the average transmission speed and the bandwidth utilization rate of the data link can be determined. In some embodiments, the processor utilization rate can be determined by determining a mean value of multiple processor utilization rates of transmitting multiple historical replication snapshots within multiple previous RPO time, and the average transmission speed and the bandwidth utilization rate of the data link can be determined by determining a mean value of multiple bandwidth utilization rates for transmitting multiple historical replication snapshots.

At a block504, in the process500, it can be determined based on the average transmission speed and the theoretical bandwidth of the data link whether there is still room for improvement in the transmission speed of the data link. If the bandwidth of the data link has been fully utilized, even splitting the replication snapshot for concurrent transmission cannot further improve the transmission speed. In some embodiments, a ratio of the average transmission speed of the data link to the theoretical bandwidth can be determined. If the ratio is greater than or equal to a predetermined threshold, it indicates that the bandwidth of the data link has been fully utilized and has no improvement room. Then, the process500proceeds to a block508. At the block508, a replication snapshot is directly transmitted from a source storage system to a destination storage system during the process500without splitting.

Return to the block504. If the ratio of the average transmission speed of the data link to the theoretical bandwidth is less than the predetermined threshold, it indicates that the transmission speed still has room to be improved. The process500proceeds to a block506. At the block506, it is determined in the process500whether the target replication snapshot needs to be split based on the RPO time and the transmission time length required to transmit the target replication snapshot. In some embodiments, the transmission time length for transmitting a replication snapshot can be determined based on the average transmission speed of the data link and a total data size of the replication snapshot to be transmitted. If the transmission time length is less than or equal to the RPO time, it indicates that the replication snapshot can be transmitted within the RPO time, and then the process500proceeds to the block508. If the transmission time length is greater than the RPO time, it indicates that the transmission of the replication snapshot cannot be completed within the RPO time, and it is necessary to split the replication snapshot to concurrently transmit multiple split data packets so as to improve the transmission speed and enable the replication snapshot to complete transmission within the RPO time.

At a block510, a performance gain score of not splitting the replication snapshot is calculated during the process500. The performance gain score indicates an overall performance of a system under the current splitting method (such as not splitting, splitting into two data packets, splitting into three data packets, etc.). A higher performance gain score indicates that the overall performance of the system is better or more in line with user requirements. In some embodiments, the performance gain score can be determined by comprehensively considering the processor consumption, transmission time consumption, and bandwidth utilization rate under the current splitting method. In some embodiments, a processor consumption score, a transmission time consumption score, and a bandwidth utilization score for transmitting the target data packet in a specific splitting method can be determined based on a processor utilization rate related to the transmission of the historical replication snapshot and a bandwidth utilization rate of the data link, the time consumption related to the transmission of the target replication snapshot, the split processor resource related to the splitting operation, and the packaging processor resource related to the packaging operation. Then, the performance gain score can be determined based on the processor consumption score, the transmission time consumption score, and the bandwidth utilization score.

The composition of a performance gain score according to some embodiments of the present disclosure is described below with reference toFIG.6.FIG.6shows a schematic diagram of a process600of determining a performance gain score according to some embodiments of the present disclosure. As shown inFIG.6, a performance gain score602includes a processor consumption score604, a transmission time consumption score606, and a bandwidth consumption score608. The processor consumption score604can be calculated based on a processor utilization rate610, a splitting processor resource612, and a packaging processor resource614. In some embodiments, the processor utilization rate610can be calculated by counting a processor utilization rate of a source storage system and a processor utilization rate of a target storage system when a historical replication snapshot is transmitted. In some embodiments, the transmission time consumption score606can be calculated based on the RPO time616(i.e., a RPO value of a replication session) and the time consumption618of transmitting the replication snapshot from the source storage system to the destination storage system, where measurement units for the RPO time616and the transmission time consumption618can be seconds. For example, the RPO time616can be 600 seconds, and the transmission time consumption618can be 240 seconds. In some embodiments, an average transmission speed622of a data link can be calculated during the transmission of the historical replication snapshot, and a bandwidth utilization score608is calculated based on the average transmission speed622and a theoretical bandwidth620of the data link, where measurement units of the theoretical bandwidth620and the average transmission speed622can be Megabits per second (Mbps).

In some embodiments, different weights can be assigned to the processor consumption score604, the transmission time consumption score606, and the bandwidth utilization score608based on user requirements, and then the performance gain score602is determined by weighting these scores. For example, if the processor configuration of the storage system is low, users may be sensitive to processor consumption, which can increase the weight of the processor consumption score604, making changes in processor resource consumption more significantly affect the performance gain score602. On the contrary, if the processor configuration of the storage system is high, users may not be sensitive to processor resource consumption and hope that the source storage system can split the replication snapshot into more data packets to improve the speed of concurrent transmission, thereby reducing the weight of the processor consumption score604. In some embodiments, the sum of the weight of the processor consumption score604, the weight of the transmission time consumption score606, and the weight of the bandwidth utilization score608is 1.

Return toFIG.5. At the block512, multiple candidate split numbers can be determined, and a candidate split number to maximize the performance gain score can be calculated in the process500. For example, in the process500, multiple performance gain scores for transmitting the target replication snapshot based on different candidate split numbers can be calculated, and then the candidate split number to maximize the performance gain scores can be determined by comparing multiple performance gain scores. At the block514, in the process500, it is determined whether the maximum performance gain score is greater than the performance gain score without splitting. If the performance gain score without splitting is greater than or equal to the maximum performance gain score, the process500proceeds to the block508. If the maximum performance gain score is greater than the performance gain score without splitting, the candidate split number is determined as a target split number in the process500, and the process500proceeds to the block516.

A process of determining an optimal split number according to some embodiments of the present disclosure is described below with reference toFIG.7.FIG.7shows a schematic diagram of an example700of determining an optimal split number according to some embodiments of the present disclosure. As shown inFIG.7, a replication snapshot702is a target replication snapshot to be transmitted from a source storage system to a destination storage system. If the replication snapshot702is not split, scores related to transmission of the replication snapshot702include a processor consumption score703, a transmission time consumption score704, and a bandwidth utilization score705. In the example700, since the replication snapshot702is not split, there is no need to consume split processor resources and package processor resources. Therefore, the processor consumption score703is relatively high, while the transmission time consumption score704and the bandwidth utilization score705are relatively low. After weighted summation is performed on the processor consumption score703, the transmission time consumption score704, and the bandwidth utilization score705, a performance gain score706is obtained.

In the example700, if the replication snapshot702is split into two data packets (i.e., the split number is 2) for concurrent transmission, scores related to the transmission of the replication snapshot702include a processor consumption score713, a transmission time consumption score714, and a bandwidth utilization score715. As shown inFIG.7, since concurrent transmission is used, the transmission time consumption score714is higher than the transmission time consumption score704without splitting, and the bandwidth utilization score715is higher than the bandwidth utilization score705without splitting. However, due to the split processor resources and packaging processor resources being consumed for splitting the replication snapshot702, the processor consumption score713is lower than the processor consumption score703without splitting. After weighted summation is performed on the processor consumption score713, the transmission time consumption score714, and the bandwidth utilization score715, a performance gain score716is obtained; in the example700, the performance gain score716is higher than the performance gain score706without splitting.

In the example700, if the replication snapshot702is split into three data packets (i.e., the split number is 3) for concurrent transmission, scores related to transmission of the replication snapshot702includes a processor consumption score723, a transmission time consumption score724, and a bandwidth utilization score725. As shown inFIG.7, since the replication snapshot702is split into more data packets for concurrent transmission, the transmission time consumption score724is higher than the transmission time consumption score704without splitting and the transmission time consumption score714when the split number is 2, and the bandwidth utilization score725is higher than the bandwidth utilization score705without splitting and the bandwidth utilization score705when the split number is 2. However, due to the need to perform more times of data packet splitting and packaging operations, the processor consumption score723is lower than the processor consumption score703without splitting and the processor consumption score713when the split number is 2. After weighted summation is performed on the processor consumption score723, the transmission time consumption score724, and the bandwidth utilization score725, a performance gain score726is obtained. In the example700, the performance gain score726is higher than the performance gain score706without splitting but lower than the performance gain score716when the split number is 2. In the candidate split numbers shown inFIG.7, when the split number is 2, the performance gain score can be maximum, and the performance gain score716is greater than the performance gain score706without splitting; therefore, it can be determined that the target split number is 2.

Return toFIG.5. At the block516, in the process500, the target replication snapshot is split into multiple data packets, and the number of the multiple data packets is the target split number. Then, these data packets are concurrently transmitted from the source storage system to the destination storage system, and these data packets are packaged in the destination storage system to obtain a recovered replication snapshot. In some embodiments, the process500can return to the block502from the block508or the block516to recollect information so as to redetermine an optimal split number according to updated data.

In this way, the split number that maximizes the performance gain score can be calculated based on the degree to which users value various factors (a processor factor, a time factor, and a bandwidth factor). Therefore, it is possible to improve the bandwidth utilization rate and reduce the transmission time length while ensuring that the processor resource consumption is within an acceptable range for users. Thus, it can avoid splitting the target replication snapshot into too many data packets due to only considering reducing the transmission time length that results in processor resources becoming bottlenecks and reduces the overall performance of the system.

In some embodiments, several formulas introduced in the following text can be used to implement the process500shown inFIG.5. In this text, As is used to represent a source storage system (for example, the source storage system102inFIG.1), and Ad is used to represent a destination storage system (for example, the destination storage system112inFIG.1). The source storage system includes a storage object oi(for example, the storage object104inFIG.1) with an asynchronous replication session, and/represents the number of the storage objects. In addition, Riis used to represent a replication snapshot corresponding to the storage object oi, and Diis used to represent a data size of the replication snapshot Ri; a total data size of all the replication snapshots Dtis calculated by the following equation (1):

In this text, fB_ris used to represent an average transmission speed of the data link, fB_ris used to represent a theoretical bandwidth of the data link. Due to some factors in the network (such as signal interference, bandwidth sharing, transmission delay, etc.), the actual bandwidth of the data link cannot reach the theoretical bandwidth. Therefore, a threshold ratio w can be set. When fB_r≥w*fB_t, it indicates that the data link is fully utilized and does not have improvement room. When fB_r<w*fB_1, it indicates that the current the data transmission does not reach the theoretical bandwidth of the data link, the data link still has room for utilization.

In this text, TRPOis used to represent a RPO value of a replication session, and time consumption Tdof transmitting all the replication snapshot Dtfrom the source storage system to the destination storage system by the following equation (2):

If Td>TRPO, it indicates that the transmission of the replication snapshot cannot be completed within the RPO time, and the transmission speed of the replication snapshot needs to be improved. Therefore, in a case where fB_r<w*fB_tand Td>TRPO, the replication snapshot can be split into multiple data packets, and these data packets can be concurrently transmitted to improve the transmission speed and reduce the transmission time length.

A method for calculating the performance gain score of transmitting a replication snapshot from the source storage system to the destination storage system without splitting it according to some embodiments is introduced below. Suppose the processor utilization rate of the source storage system is Cs, and the processor utilization rate of the destination storage system is Cd, the overall processor utilization rate Cwoof the system is calculated by the following equation (3), and there is no measurement unit for the processor utilization rate Cwo:

Since the lower Cwo, the higher the processor consumption score, the reciprocal of Cwocan be used to represent the processor consumption score. In addition, the time consumption Twoof transmitting the replication snapshot from the source storage system to the destination storage system can be calculated by the following equation (4):

Since the measurement unit of the time consumption Twois time (for example, seconds), it needs to be standardized. For example, Twocan be compared with TRPOthrough the following equation (5) so as to obtain a time consumption score RTwo:

In addition, the bandwidth utilization rate Bwo(i.e., the bandwidth utilization score) of the data link is calculated through the following equation (6):

In some embodiments, according to the user's requirements, a weight we can be allocated to the processor consumption score, a weight wTcan be allocated to the time consumption score, and a weight wBcan be allocated to the bandwidth utilization score, where 7>wC+wT+wB=1, so as to calculate a performance gain score Swowithout splitting through the following equation (7).

A method for calculating a performance gain score of concurrently transmitting multiple data packets to a destination storage system in a case where a replication snapshot is split into multiple data packets according to some embodiments is introduced below. In this text, Ne is used to represent a maximum split number that the replication snapshot can be split into, and Ne can be calculated through the following equation (8):

where int indicates a rounding function.

If Nc=1, it indicates that the bandwidth of the data link is fully utilized; at this time, the replication snapshot is not split. If Nc>1, it indicates that the bandwidth of the data link still has room for utilization; correspondingly, the data transmission speed also has room for improvement. Suppose the replication snapshot is split into Nkdata packets, Cuis used to represent processor resource consumption of performing one splitting operation by the source storage system, Cpis used to represent processor resource consumption of performing one packaging operation by the destination storage system, and the overall processor utilization rate Cwof the system can be calculated through the following equation (9):

Since the lower Cw, the higher the processor consumption score, a reciprocal of Cwcan be used to represent the processor consumption score. In addition, since the replication snapshot is split into Nkdata packets for concurrent transmission, the data transmission speed can be improved to fB_r*Nk, where fB_r*Nk/fB_tand 1≤Nk≤Nc. Then, the time consumption Twof transmitting all the replication snapshots Dt from the source storage system to the destination storage system can be calculated through the following equation (10):

Then, Twcan be compared with TRPOthrough the following equation (11) so as to obtain a time consumption score RTw:

In addition, a bandwidth utilization score Bwcan be calculated through the following equation (12):

A performance gain score Swof splitting the replication snapshot into Nkdata packets for concurrent transmission can be calculated through the following equation (13) in combination with the above equations (9), (11), and (12):

The greater the performance gain score Sw, the better the performance of the data transmission. Since the performance gain score Swdepends on the value of Nk, Nkthat maximizes Swcan be calculated, and Nkmust meet the conditions of the following equation (14):

Then, a performance gain score Swowhen not splitting can be compared with a maximum performance gain score max max(Sw) when splitting; if Swo≥max(Sw), the replication snapshot can be directly transmitted without splitting. If Swo<max(Sw), the replication snapshot is split into Nkdata packets for concurrent transmission so as to improve the performance of data transmission.

In this way, it is possible to quantify the performance gain of splitting a replication snapshot into different numbers of data packets for concurrent transmission. Therefore, an optimal split number can be found by comparing performance gain scores corresponding to different split numbers, improving the performance of the process of transmitting the replication snapshot from a source storage system to a destination storage system. Therefore, it avoids splitting the target replication snapshot into too many data packets due to only considering reducing the transmission time length that results in processor resources becoming bottlenecks and reduces the overall performance of the system.

FIG.8shows a schematic block diagram of an example device800that may be used to implement an embodiment of the present disclosure. As shown in the figure, a device800includes a computing unit801, which may execute various appropriate actions and processing according to computer program instructions stored in a read-only memory (ROM)802or computer program instructions loaded from a storage unit808onto a random access memory (RAM)803. Various programs and data required for the operation of the device800may also be stored in the RAM803. The computing unit801, the ROM802, and the RAM803are connected to each other through a bus804. An input/Output (I/O) interface805is also connected to the bus804.

A plurality of components in the device800are connected to the I/O interface805, including: an input unit806, such as a keyboard and a mouse; an output unit807, such as various types of displays and speakers; a storage unit808, such as a magnetic disk and an optical disc; and a communication unit809, such as a network card, a modem, and a wireless communication transceiver. The communication unit809allows the device800to exchange information/data with other devices via a computer network, such as the Internet, and/or various telecommunication networks.

The computing unit801may be various general-purpose and/or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit801include, but are not limited to, central processing units (CPUs), graphics processing units (GPUs), various specialized artificial intelligence (AI) computing chips, various computing units for running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit801performs various methods and processes described above, such as the method300. For example, in some embodiments, the method300may be implemented as a computer software program that is tangibly included in a machine-readable medium such as the storage unit808. In some embodiments, part or all of the computer program may be loaded and/or installed onto the device800via the ROM802and/or the communication unit809. When the computer program is loaded to the RAM803and executed by the computing unit801, one or more steps of the method300described above may be performed. Alternatively, in other embodiments, the computing unit801may be configured to implement the method300in any other suitable manners (such as by means of firmware).

Program code for implementing the method of the present disclosure may be written by using one programming language or any combination of a plurality of programming languages. The program code may be provided to a processor or controller of a general purpose computer, a special purpose computer, or another programmable data processing apparatus, such that the program code, when executed by the processor or controller, implements the functions/operations specified in the flowcharts and/or block diagrams. The program code may be executed completely on a machine, executed partially on a machine, executed partially on a machine and partially on a remote machine as a stand-alone software package, or executed completely on a remote machine or server.

In the context of the present disclosure, a machine-readable medium may be a tangible medium that may include or store a program for use by an instruction execution system, apparatus, or device or in connection with the instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. The machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the above content. More specific examples of the machine-readable storage medium may include a one or more wires-based electrical connection, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combinations thereof. Additionally, although operations are depicted in a particular order, this should be understood that such operations are required to be performed in the particular order shown or in a sequential order, or that all illustrated operations should be performed to achieve desirable results. Under certain environments, multitasking and parallel processing may be advantageous. Likewise, although the above discussion contains several specific implementation details, these should not be construed as limitations to the scope of the present disclosure. Certain features that are described in the context of separate embodiments may also be implemented in combination in a single implementation. Conversely, various features that are described in the context of a single implementation may also be implemented in a plurality of implementations separately or in any suitable sub-combination.

Although the present subject matter has been described using a language specific to structural features and/or method logical actions, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the particular features or actions described above. Rather, the specific features and actions described above are merely example forms of implementing the claims.