# Multipath Transmission Control Protocol for Live Virtual Machine Migration in the Cloud Environment

Authors: Kadiyala Ramana; Rajanikanth Aluvalu; Vinit Kumar Gunjan; Ninni Singh; M. Nageswara Prasadhu
Journal: Wireless Communications and Mobile Computing
 (2022)

---

## Abstract

For mobile cloud computing (MCC), a local virtual machine- (VM-) based cloudlet is proposed to improve the performance of real-time resource-intensive mobile applications. When a mobile device (MD) discovers a cloudlet nearby, it takes some time to build up a virtual machine (VM) inside the cloudlet before data offloading from the MD to the VM can begin. Live virtual machine migration refers to the process of transferring a running Virtual Machine (VM) from one host to another without interrupting its state. Theoretically, live migration process must not render the instance being migrated unavailable during its execution. However, in practice, there is always a service downtime associated with the process. This paper focuses on addressing the need to reduce the service downtime in case of live VM migration in cloud and providing a solution by implementing and optimizing the multipath transmission control protocol (MPTCP) ability within an Infrastructure as a service (IaaS) cloud to increase the efficiency of live migration. We have also introduced an algorithm, theα-best fit algorithm, to optimize the usage of bandwidth and to effectively streamline the MPTCP performance.


---

## Body

## 1. Introduction

A virtual machine (VM) refers to an entity that does not exist physically, but is hosted on a hypervisor, and yet has all the capabilities of an equivalent physical machine [1]. This virtualization gives an added advantage in the form of the ability to migrate a VM from one host machine to another. Virtual machine migration may be done in the cloud for several reasons including load balancing and energy usage optimization. When a VM is used to handle data traffic within a cloud, a relatively freer host (destination host) can be chosen to migrate it to, as an attempt to relieve the load on the source host. Similarly, energy optimization within the cloud can be done by shutting off the nodes which do not host any occupied VMs [2].In datacenter virtualization, live migration is a crucial technology and feature. With live migration, VMs can be transferred from one physical host to another with little to no impact on the availability of running applications. This ensures that running applications are not harmed by any physical server failures, significantly improving service availability. The TCP/IP protocol is used to send live migration communications via the Ethernet network that connects the cluster servers. The content that needs be migrated is mostly the CPU cache, memory, and buffers; however, the memory content takes up the majority of the time. The content of the CPU cache and buffers is almost insignificant in comparison to the RAM content, which is what most authors assume in live migration modeling.In the early stages of its development, VM migration was static in nature. This is to say that the general practice was to suspend the VM state before the actual migration took place. This caused the instance to be unavailable throughout the migration process. The functioning of the VM was resumed after the entire VM data, that is, the memory pages and the processor state, was copied to the destination host. It is easily imaginable how much inconvenience this sort of a process can cause when implemented in cloud environment [3]. The users pay for the services they receive, and they expect those services to be available all the time. In such scenario, where a VM handling some or all of those services is rendered unavailable till the completion of the migration process, the user experience surely tends to degrade [4].The solution was live VM migration. In this case, the VM is not unavailable for the entire duration of the migration but only a smaller fraction of it, which is termed as the service downtime. Theoretically, the machine continues to function as if nothing had occurred at the back end during the migration process. However, as shown by Sv¨ard et al. [5], running processes on the VM are interrupted even during its live migration. This is attributed to the service downtime. Therefore, even if the service downtime is small, to ensure good end user experience, there will always be a need to reduce it as much as possible.When we talk about the functioning of an Infrastructure as a Service (IaaS) cloud, we automatically step into the realm of software-defined networking (SDN). SDN refers to the use of programmable software components that can be installed in a networking framework and still provide the same functionality as their physical hardware counterparts [6]. Modern clouds depend heavily on SDN to function. Though physical resources are required to host the cloud instances, the networking in the cloud combines elements which are both physical and software defined in nature. Without SDN, it would be nearly impossible to cope with the growing pressure on data networks as the number of mobile networking devices increase [7]. IaaS cloud platforms such as “OpenStack” use SDN to connect the instances to the physical networking resources. With SDN, networking becomes more manageable and less cumbersome. Moreover, we can add or remove elements from a software defined network according to our needs, which would otherwise require the installation and removal of physical networking devices.In this paper, we have implemented multipath TCP, which is a recent project by the Internet Engineering Task Force (IETF) to facilitate faster data transfer within the cloud environment [8]. The VM data can be transferred with a higher speed between the host nodes in case of data centers if the effective bandwidth of the connection between them can be increased. This ability is provided by MPTCP by allowing us to use multiple paths for TCP data transfer at the same time. By the creation of TCP subflows, MPTCP makes use of multiple interfaces on the connected hosts for the transfer of data. Hence, theoretically, the aggregate bandwidth of each of the individual paths is available for data transfer. However, in reality, this is not the case. MPTCP requires some optimization techniques to be implemented effectively in the cloud. We introduce an algorithm, the α-best fit algorithm, for the optimization of MPTCP usage, and compare it to the subflow optimization offered by Joshi and Kataoka [9]. This algorithm provides a much better optimization scheme than the one already proposed [9] in terms of bandwidth usage. The generation of optimum number of subflows in [9] leads to wastage of bandwidth by generating suboptimal results, while the α-best fit algorithm minimizes this wastage, which is crucial in the cloud scenario where bandwidth resources to be dealt with are massive in scale.There are three forms of live VM migration: precopy, postcopy, and hybrid-copy. The most resilient migration type is precopy live migration, which we focus on in this study. The source host memory content copy begins and continues to transfer until it reaches a halting circumstance in precopy migration. The VM will then stop on the source host and start on the target host. This is the most popular migration method used by commercial and open source hypervisors since it is the most dependable.The remaining part of this paper is organized as follows: Section2 explains the conventional algorithm that is in use for conducting live migrations, Section 3 explains MPTCP, and its effects on live migration of cloud instances, Section 4 explains the need for optimization of MPTCP implementation in cloud environments and puts forward our proposed approach to do so, Section 5 contains the details of implementation of MPTCP in the cloud environment, Section 6 embodies the results obtained, and Section 7 includes the discussion on the results and the conclusion.


## 2. Related Work

Multipath TCP is a TCP enhancement that allows for simultaneous transmission of data through many pathways from one end to the other. For example, multipath TCP allows many applications to transport and receive data over multiple interfaces, such as cellular and Wi-Fi, by creating a single TCP sub flow for each interface [10]. This subflow is used by the MPTCP scheduler to transmit and receive data. MPTCP’s design has various advantages, including resource utilization, throughput, and a mild response to faults, as well as good route performance.If numerous sub flows are available, a scheduler [11] selects the subflow with the least round-trip time (RTT) to transfer the data. During the transmission of the segment, the scheduler chooses the way with the shortest round-trip time among all subflows whose congestion window is not yet full. If there are multiple such paths, the scheduler creates a bias towards one of them. It also continues to deliver information on the particular subflow until the congestion window for that sub flow is full. In prior research, MPTCP was examined in a mobile situation. The authors investigated the impact of mobility on MPTCP in [12], and in [13], they offered various MPTCP options for use by cell phones for Wi-Fi handover. However, neither of the works investigates the heterogenetic nature of the path in lossy subflows. The authors of [14] gave a comparison of single TCP vs. multipath TCP. The authors of [15] used various scheduler algorithms to calculate the impact of scheduler architecture on performance. In [16, 17], different congestion mechanism techniques are compared.The authors proposed scheduler algorithms in [18], which select subflows based on an estimate of the amount of traffic they can handle before becoming congested. By considering large network transfers and a restricted buffering quantity, the methodology demonstrates. Another unique scheduling policy offered by the authors in [19] was to keep a strategic distance from out-of-request parts. The authors do not explain why a segment is removed from the TCP buffer after it is transmitted by another subflow. The authors presented a delay-aware packet scheduler technique on the ns-2 simulator platform in [20]. In both delay and stable settings, the proposed technique analyses path heterogeneity. The performance of MPTCP over a mobile network was examined by the authors in [21]. The analysis took into account a variety of subflows as well as precise statistics such as round-trip time (RTT) and out-of-order delivery, but the authors did not account for lossy subflows. Lim et al. proposed a new scheduler method that sends buffers by monitoring available bandwidth on each subflow, but it does not take use of the data loss rate on each sub flow.


## 3. The Precopy Algorithm

This is the conventional method for carrying out live VM migrations. In this method, the memory pages of the VM are sent before its processor state to the destination node. The transfer of the memory pages occurs in several iterations. These iterations continue till a set number is reached, or a small enough writable working set (WWS) is obtained [22]. In each iteration, there are certain memory pages which are modified during the transmission, and these are called dirty pages. During each iteration, there is a possibility of a memory page being dirtied, and these dirty pages are transferred to the target node in the next iteration.The duration of service downtime starts when either a WWS is obtained or the cap on the number of iterations is reached [23]. If either of the above conditions are satisfied, the VM at the source node is suspended, and all the remaining memory pages along with the dirty pages from the previous iteration are moved to the destination node. After this transfer, the VM is resumed at the destination node, and its source node copy is destroyed. Hence, it is obvious that the service downtime depends upon the number of memory pages to be sent, which also includes the dirty pages from the previous iteration [24].The time taken to transfer the entire RAM is given by [22]
(1)t0=VB,whereV is the total memory size of the VM, and B is the available bandwidth.We must also consider the page dirtying rate to get a measure of the time required for their transfer in each iteration during precopy. This time required for the transfer of the remaining memory pages, which also includes the pages dirtied in each iteration, is given by [22]
(2)t1=R×t0B,t2=R×t1B,tk=R×tk−1B,whereR is the memory dirtying rate, and k is the total number of iterations.The precopy process gets suspended when thek-th iteration occurs, and the stop conditions are satisfied. Following this, the VM at the source is suspended, and the remaining dirty pages, along with the processor state of the VM, are transferred to the destination node [22]. This takes about tk=R×tk−1/B amount of time.Hence, the total time for the precopy and stop and copy is calculated as(3)T0=t0+t1+t2+⋯+tk=VB×1−R/Bk+11−R/B.Therefore, the total migration time is given by(4)Tmig=Pre−migrationoverhead+To+Post‐migrationoverhead,where the premigration overhead refers to the time required for the resource reservation, and the postmigration overhead consists of the commitment and activation stage of the precopy [22]. While these overheads are largely unavoidable, the value of To can be varied.


## 4. Multipath TCP (MPTCP)

MPTCP is a recent project by the Internet Engineering Task Force (IETF). This project enables data transfer via multiple TCP subflows that are sent over different TCP paths [8]. This capability is of great value if we consider its implementation in data centers. Multiple paths to send data imply more bandwidth for data transfer. In principle, this bandwidth should be equal to the aggregate of the bandwidth capacities of the individual paths [25].As we can see in Figure1, data is divided into chunks, and these chunks are transferred through different subflows over different paths. This means that the client and the server in a simple MPTCP connection have multiple IP addresses, which may or may not be in the same network [26]. In case of data centers, MPTCP can therefore be utilized to obtain the ability of fast data transfer; since, at any given time, more than one paths can be used simultaneously as opposed to TCP, which can be seen in Figure 2.Figure 1
Division of MPTCP flow into TCP subflows.Figure 2
Comparison of MPTCP session with normal TCP session.Now, consider the scenario in which the implementation of the precopy algorithm described in Section2 in combination with the MPTCP capability is being carried out.If we considerk number of iterations to occur for the transfer of dirty pages in case of a single active path for data transfer, as was the case in equations (2) and (3), then we can see that for an iteration i, the pages dirtied will be R×i, and these will be transmitted in the i+1-th iteration.As stated by Nathan et al. [27], the page dirtying rate for the i-th iteration is given by the expression:
(5)R=numberofpagesdirtiediniterationi−1×pagesizetimetakenbyiterationi−1.Now, the time taken for an iteration is dependent upon the bandwidth available, which is evident from equation (2). Hence, an increased bandwidth will reduce the page dirtying rate [28]. This implies that less pages will be dirtied in each iteration with the increase in effective bandwidth, and hence the time required for these iterations will decrease. Considering the page dirtying rate to be constant for a migration, then the advantage of added bandwidth is expected to reduce the time ti+1=R×ti/B′, where B′ is the new bandwidth.This implies that though the iterations will require lesser time than in the case of a reduced bandwidth capacity, the actual number of iterations,k will remain the same.This implies that the advantage MPTCP will provide due to increased aggregate bandwidth will be in the form of reduced transmission time rather than reduced number of iterations before the WWS is achieved. Also, as stated in [22], the time required after k iterations to transfer the WWS and the processor state is given by
(6)tk=R×tk−1BThis time period required for the stop-and-copy phase will be reduced by an increase in bandwidth. Therefore, a decrease in service downtime is expected.Assuming that we have an MPTCP connectivity between two host machines withx number of active paths for data transfer, let γ be the segment-wise bandwidth utilization factor, which is explained in the next section.Hence, the aggregate effective bandwidth of the MPTCP connection will beB′=x×γ×B, where B is the bandwidth of each individual path in the MPTCP connection.Using this new bandwidth, equations in Section2 can be written for the MPTCP scenario as
(7)t′0=VB′,(8)t′1=R×t′0B′,t2=R×t′1B′,t′k=R×t′k−1B′,(9)T′0=t′0+t′1+t′2+⋯+t′k=VB′×1−R/B′k+11−R/B′.UsingB′=x×γ×B in equations 7and 8, it can be stated that for an iteration i,
(10)ti′=tixγi+1.From equation (10), it can be inferred that with the increasing number of iterations, the time required for transmission of data will decrease progressively in case of MPTCP connectivity between the hosts. When the value of “i” in equation (10) becomes equal to k, then the time period t′k represents the time spent in the stop-and-copy phase [22], which is synonymous to the service downtime. The value x will always be positive and greater than one for an MPTCP connection, while γ will be a positive fraction representing the degree of utilization of the bandwidth resources.On the other hand, equation (10) also provides the information that in case of TCP connection between the hosts, when x=1, γ is still less than 1. This means that with each successive iteration, with increasing powers of γ, the performance of precopy for transmission of dirty pages will degrade.Therefore, while TCP connectivity causes the performance to worsen with every iteration, MPTCP connectivity actually increases it with the increase in number of iterations till a set number of iterations, which will depend upon the productxγ. As long as the quantity xγ remains greater than one, the performance will increase with increasing number iterations.Rearranging equation 6 is as follows:(11)T′0=∑i=0ktixγi+1.Therefore,(12)xγk+1T′0=∑i=0ktixγk−i.Dividing equation (9) by equation (3), we get
(13)T0′T0=1xγk+1×∑i=0kxγk−iHence, it can be seen that the ratioT0′/T0 will always be less than one if the product xγ>1. Therefore, the total migration time will also decrease with increasing number of active paths for data transfer.This deduction implies that by increasing the number of active paths in an MPTCP connection, there will be a reduction in the service downtime to a negligible nonzero amount. However, this is not the case, and thus the MPTCP connection requires optimization for the reason that with increasing number of paths in an MPTCP connection, there is an increased possibility of network congestion at the points where the subflows reunite. This congestion lowers the value of the factorxγ, as the bandwidth utilization γ is drastically reduced in case of congestion.Focusing on the value ofγ, its relation to x and T0 can be predicted to some extent using equation (11). Partially differentiating equation (11) with respect to γ,
(14)∂T0′∂γ=∑i=0k∂∂γtixγi+1=−t0xγ2−2t1x2γ3−⋯−k+1tkxk+1γk+2.In terms oft0, we have
(15)∂T0′∂γ=t0xγ2−1−2RBxγ−⋯−k+1RkBkxkγk.This series can be converged, and the resulting equation is as follows:(16)∂T0′∂γ=−t0xγ2×B−kγ−kRk+1x−kk+1R−Bγxk+2−B2x2γ2Bγx−R2Figuring out the value ofγ at which the value of To will be maximum or minimum can be achieved by finding the critical points by equating the partial differential in equation (16) to zero.Substituting∂T0′/∂γ=0, equation (16) can be rewritten as
(17)B−kγ−kRk+1x−kk+1R−Bγxk+2−B2x2γ2Bγx−R2=0.(18)As−t0xγ2≠0.Now, Let(19)C1=B−kx−kRk+2k+1,(20)C2=B1−kx1−kRk+1,(21)C3=B2x2.C4=C2k−2.Substituting the above values in equation (17),
(22)C1γk−C4γk−1−C3=0⇒C1−C4γ−C3γk=0.Equation (22) is a polynomial of order k. Here, the value of γ can be represented by the series:
(23)γ=∑m=0∞−1mC1mk−m+1C3mC4mk+1m!∏j=0m−2mk−j=C1C4−C1kC3C4k+1+2kC12k−1C322!C42k+1−3k3k−1C13k−2C333!C43k+1+4k4k−14k−2C14k−3C344!C44k+1−⋯This series will converge if the following condition is satisfied:(24)C1k−1C3C4k<k−1k−1kk,which translates to(25)BxR2×k+1k−1k−2<k−1k−1kk.For this inequality, it can be clearly seen thatk∉0;2, because for k∈0;2, the inequality becomes meaningless. Therefore, k≥3 which is justified as practically the number of iterations is always higher than two. Hence, the series in equation (23) cannot be converged.Now, the constraint on the value ofγ is that γ ∈ (0; 1) If C1, C3, C4>0 and C1<C3+C4, there can be a unique value of γ, which will be the critical point for the equation (11). This can be accomplished by taking a sufficient number of terms so that we get a good approximation of the sum of the series in equation (23).The page dirtying rate, as defined by Nathan et al. [27], is lesser than the value of (Bx), which is quite reasonable. Also, since k and x are positive integers, and γ ∈ (0; 1).Therefore, using the values ofC1, C2, C3, and C4 in equation (25), it can be seen that
(26)γ=RBx×k+1k−2.The higher terms in the series given by equation (23) are not taken into consideration, because of the fact that for k∈3,∞, the value of higher terms will become negligible. At greater values of k, the power of the terms in the denominators of the terms in equation (23) will increase steeply. The first term yields a number that satisfies the condition of γ ∈ (0, 1), while inclusion of the higher terms does not affect its value significantly. This concludes that the value of k, given our constraints is as in equation ((26)).Now,(27)∂2T0′∂γ2=∂2∂γ2∑i=0ktixγi+1=∑i=0k∂∂γ−tii+1xi+1γi+2=∑i=0ktii+1i+2xi+1γi+3.As all the variables in equation (27) are positive, and the critical point given by equation (26) is also a positive value, it can be inferred that the critical point is the point of maxima for the function T0′ with respect to γ with x and k constant.Thus, it is explicit that for the value ofγ given in equation (26), the migration time T0′ will be maximum. This proves that for MPTCP to be advantageous at a given number of active data paths between the hosts, the value of γ must be greater than the expression given by equation (26).


## 5. Optimizing MPTCP Capability

The MPTCP capability offers higher bandwidth for data transfer in clouds and data centers. In theory, by increasing the number of paths in an MPTCP session between two hosts, there should be significant improvement in the data transfer speed as an effect of increased effective bandwidth. One must therefore expect that the number of paths can be increased without any limit to obtain a nearly infinite bandwidth capacity for data transfer. However, it is not the case. While it is true that the increase in the number of paths leads to increased number of TCP subflows in an MPTCP session, their optimum usage is something that depends upon other factors.As explained by Joshi and Kataoka [9], overutilization as well as underutilization of the MPTCP capability by creating fewer or a greater number of subflows than required leads to poor performance in terms of data transfer. [9] already provides an optimization algorithm which has been tested and implemented to provide an improved result than using MPTCP without any subflow optimization in terms of throughput.The SFO given in [9] is modified to obtain the α-best fit algorithm, which is more suited for the cloud environment. Moreover, as the cloud environment deals with bandwidth resources that are in order of Gbps, the optimization of the SFO [9] for improved performance in terms of bandwidth utilization is needed.The MPTCP subflow optimization as elaborated in [9] consists of three functions, the first function responsible for finding the feasible paths for the MPTCP session between the hosts, and the second function is responsible for calculating the optimum number of sub flows, while the third function deals with the assignment of paths to the MPTCP subflows.SFO [9] uses the modified k-shortest path algorithm in order to calculate the set of shortest paths for data transfer between the hosts. While this approach suffices in the experimental topology considered in [9], it might lead to suboptimal bandwidth utilization in case of other available topologies for data centers and cloud environment. Hence, the first optimization we propose is to modify the path finder method to function not on the shortest k-path algorithm, but on maximum bandwidth algorithm, which gives priority to the path with the highest bandwidth capacity during selection.The function responsible for the calculation of optimum number of sub flows has been modified by us to include the aspects of protection against path failure and reliability in performance along with optimization of the selection process.


### 5.1. Theα–Best Fit Algorithm

Before discussing theα-best fit algorithm for the optimization of MPTCP subflows, it is essential to consider a factor that ought to be useful while we deal with substantial amount of bandwidth capacities, as is in the case of data centers. This factor is the segment-wise bandwidth utilization ratio (γ), including this factor will lead to a better reliability of the system, as it gives us a more realistic and practical view of the bandwidth resources. The inclusion of γ increases the accuracy of the algorithm while functioning in real time conditions.This factor is calculated by the standard method, which involves using SNMP objects to know what fraction of the total bandwidth of a link is being considered. The bandwidth of a path is limited by the capacity of the minimum bandwidth link in its constituting links. This means that for a path consisting ofn links, λP=minλi,∀i∈Pi=1n. Therefore, γ is calculated segment-wise.The value ofγ can be calculated as [29]
(28)γ=max∆IfInOctets,∆IfOutOctets×8∆timeperiodbetweenthepolls×IfSpeed,where IfInOctets and IfOutOctets refer to the ingress and egress SNMP objects on a TCP link which are sent repeatedly after a fixed time interval. IfSpeed refers to the interface speed as stated by the specifications of the link.LetS be the total size of the VM data to be transferred during the MPTCP session, τ be the time limit in which the live VM migration should be complete, αd be the demand of bandwidth resources for the transfer of the VM, ONS be the optimum number of subflows, OP be the set of all the optimized paths available to put the sub flows on, FP be the set of all the feasible paths, D be the set of discarded paths, and λ be the bandwidth capacity.The subflow optimization [9] provides a method to divide the MPTCP traffic into a suitable number of sub flows; however, they do not clarify the ramifications of the case when a particular path fails in the MPTCP connection. The comparison between α-best fit algorithm and subflow optimization algorithm [9] is represented in Table 1.Table 1
Comparison betweenα-best fit algorithm and subflow optimization algorithm [9].
Theα–best fit algorithm
Subflow optimization [9]
The maximum bandwidth path is given priority while selection of feasible paths.
The shortest path is chosen while deciding the set of feasible paths.
Discarded paths are stored as they can be used later to make maximum use of the available bandwidth.
Paths once discarded are never used again.
Either the path with bandwidth closest to the demand is used, or the subflow is divided into smaller subflows.
If the ONS is one, it might lead to the selection of a path with far greater bandwidth than the demand.
Protection against failure of a path is taken into account.
Protection against failure is not considered.
It is more reliable to function correctly in real time environment as bandwidth utilization factor is taken into account.
Bandwidth utilization factor is not considered, hence, its reliability in real time environment is limited.In order to address the scenario, in which a path moves to a “down” state, an additional clause has been devised in theα-best fit algorithm. This part of the algorithm makes use of a function named failure protection, which uses the set of all feasible paths (K) and the bandwidth capacity of the failed path to decide which path to redirect the subflow, which was initially on the now failed path, or to divide the subflow further into smaller subflows that are redirected onto other available paths in the set K. It is ensured that the algorithm functions in a way that causes minimum wastage of bandwidth owing to the importance and the order of magnitude of the bandwidth resources in the cloud environment.Algorithm 1: The α-best fit algorithm.
1. Compute
α
d=s/τ
2. Initialise D=ϕ; ONS=0; OP=ϕ; FP←K; Sort K in ascending order of bandwidths
3. Compute
λ
P
∀
P
∈
K
λ
P
=
γ
λ
P
i
=
1
F
P
4. Compute the total capacity of all feasible paths as
∝
=
∑
P
=
1
F
P
λ
P
5. if
α
d
<
α
then
6.  FP←FP-Pmin, Where Pmin is the path ∈ FP with the minimum bandwidth
7.D⟵D∪Pmin
8.Compute the new bandwidth capacity
α
s
=
∑
P
=
1
F
P
λ
P
9.ifαd<αsthen
10.α⟵αs
11.Repeat 5
12. while i=1 to |D do
13.FP⟵FP∪Di
14.αs′=∑P=1FPλP
15.  if
α
d
≥
α
s
′
then
16.  FP←FP-Di
17.  i++18. OP←FP, ON=|OP|
19. if ONS=1 & λP≠αdthen
20.  K←ϕ, K←D, D←ϕ
21.goto 3
22. return OP & ONS
23. if
O
P
i
i
O
P
is
down
t
h
e
n
,
Where
OP
is
the
set
of
Optimal
Paths
returned
from
22
24.  Invoke failure_protection(K,λ(OPi)), OP←OP-OPi+OPindex, where index is a set returned by the function failure_protection
25.  The function failure_protection can be elaborated as:
26. Function FAILURE_PROTECTION(K,λ(OPi))
27. Initpos=0,K′←ϕ, index←ϕ, C=0
28. for j=1 to |K| do
29.  if Kj=λ(OPi) then
30.   pos←j
31.K′⟵K′∪Pkk=1j:Pk∈K
32. C←possible number of combinations inK′ such that ∑k=1K′=λ(OPi)
33. if C=0 then
34.  index ←pos+1
35. else
36.  Choose the combination with minimum number of paths inK′
37.  index←positions of constituting inK′
38. return index


## 5.1. Theα–Best Fit Algorithm

Before discussing theα-best fit algorithm for the optimization of MPTCP subflows, it is essential to consider a factor that ought to be useful while we deal with substantial amount of bandwidth capacities, as is in the case of data centers. This factor is the segment-wise bandwidth utilization ratio (γ), including this factor will lead to a better reliability of the system, as it gives us a more realistic and practical view of the bandwidth resources. The inclusion of γ increases the accuracy of the algorithm while functioning in real time conditions.This factor is calculated by the standard method, which involves using SNMP objects to know what fraction of the total bandwidth of a link is being considered. The bandwidth of a path is limited by the capacity of the minimum bandwidth link in its constituting links. This means that for a path consisting ofn links, λP=minλi,∀i∈Pi=1n. Therefore, γ is calculated segment-wise.The value ofγ can be calculated as [29]
(28)γ=max∆IfInOctets,∆IfOutOctets×8∆timeperiodbetweenthepolls×IfSpeed,where IfInOctets and IfOutOctets refer to the ingress and egress SNMP objects on a TCP link which are sent repeatedly after a fixed time interval. IfSpeed refers to the interface speed as stated by the specifications of the link.LetS be the total size of the VM data to be transferred during the MPTCP session, τ be the time limit in which the live VM migration should be complete, αd be the demand of bandwidth resources for the transfer of the VM, ONS be the optimum number of subflows, OP be the set of all the optimized paths available to put the sub flows on, FP be the set of all the feasible paths, D be the set of discarded paths, and λ be the bandwidth capacity.The subflow optimization [9] provides a method to divide the MPTCP traffic into a suitable number of sub flows; however, they do not clarify the ramifications of the case when a particular path fails in the MPTCP connection. The comparison between α-best fit algorithm and subflow optimization algorithm [9] is represented in Table 1.Table 1
Comparison betweenα-best fit algorithm and subflow optimization algorithm [9].
Theα–best fit algorithm
Subflow optimization [9]
The maximum bandwidth path is given priority while selection of feasible paths.
The shortest path is chosen while deciding the set of feasible paths.
Discarded paths are stored as they can be used later to make maximum use of the available bandwidth.
Paths once discarded are never used again.
Either the path with bandwidth closest to the demand is used, or the subflow is divided into smaller subflows.
If the ONS is one, it might lead to the selection of a path with far greater bandwidth than the demand.
Protection against failure of a path is taken into account.
Protection against failure is not considered.
It is more reliable to function correctly in real time environment as bandwidth utilization factor is taken into account.
Bandwidth utilization factor is not considered, hence, its reliability in real time environment is limited.In order to address the scenario, in which a path moves to a “down” state, an additional clause has been devised in theα-best fit algorithm. This part of the algorithm makes use of a function named failure protection, which uses the set of all feasible paths (K) and the bandwidth capacity of the failed path to decide which path to redirect the subflow, which was initially on the now failed path, or to divide the subflow further into smaller subflows that are redirected onto other available paths in the set K. It is ensured that the algorithm functions in a way that causes minimum wastage of bandwidth owing to the importance and the order of magnitude of the bandwidth resources in the cloud environment.Algorithm 1: The α-best fit algorithm.
1. Compute
α
d=s/τ
2. Initialise D=ϕ; ONS=0; OP=ϕ; FP←K; Sort K in ascending order of bandwidths
3. Compute
λ
P
∀
P
∈
K
λ
P
=
γ
λ
P
i
=
1
F
P
4. Compute the total capacity of all feasible paths as
∝
=
∑
P
=
1
F
P
λ
P
5. if
α
d
<
α
then
6.  FP←FP-Pmin, Where Pmin is the path ∈ FP with the minimum bandwidth
7.D⟵D∪Pmin
8.Compute the new bandwidth capacity
α
s
=
∑
P
=
1
F
P
λ
P
9.ifαd<αsthen
10.α⟵αs
11.Repeat 5
12. while i=1 to |D do
13.FP⟵FP∪Di
14.αs′=∑P=1FPλP
15.  if
α
d
≥
α
s
′
then
16.  FP←FP-Di
17.  i++18. OP←FP, ON=|OP|
19. if ONS=1 & λP≠αdthen
20.  K←ϕ, K←D, D←ϕ
21.goto 3
22. return OP & ONS
23. if
O
P
i
i
O
P
is
down
t
h
e
n
,
Where
OP
is
the
set
of
Optimal
Paths
returned
from
22
24.  Invoke failure_protection(K,λ(OPi)), OP←OP-OPi+OPindex, where index is a set returned by the function failure_protection
25.  The function failure_protection can be elaborated as:
26. Function FAILURE_PROTECTION(K,λ(OPi))
27. Initpos=0,K′←ϕ, index←ϕ, C=0
28. for j=1 to |K| do
29.  if Kj=λ(OPi) then
30.   pos←j
31.K′⟵K′∪Pkk=1j:Pk∈K
32. C←possible number of combinations inK′ such that ∑k=1K′=λ(OPi)
33. if C=0 then
34.  index ←pos+1
35. else
36.  Choose the combination with minimum number of paths inK′
37.  index←positions of constituting inK′
38. return index


## 6. Implementation

The implementation MPTCP on an OpenStack Mitaka IaaS cloud consisting of four nodes–one controller node, two compute nodes, and a block storage node, was done. The networking inside the cloud makes use of the OpenStack Neutron server residing on the controller node, and ML2 is used as the core plugin with Linux Bridge agent as the mechanism driver. The controller node has 16 GB RAM, 250 GB SDD, two network interfaces, while the compute nodes have 32 GB of RAM, 150 GB SDD, and four network interfaces each. The block storage node has 32 GB of RAM, 150 GB SDD on which the Operating System is hosted, and a 500 GB SDD to provide block storage resources to the OpenStack instances. The Operating System installed on all the four nodes is Ubuntu 14.04 LTS. Figure3 shows the host networking implemented for deploying Openstack with MPTCP capability.Figure 3
Host networking layout for implementation.In order to measure the service downtime, instances are created, and they are pinged continuously from the controller node while the migration process from one compute host to another occurs. The time for which the instance remains unavailable is observed as the service downtime. A packet sniffer (Wireshark) is used for this purpose.The service downtime is measured without using MPTCP (normal TCP connection) and then using MPTCP using two and then three paths between the hosts, respectively. Comparisons are made using the obtained results.In order to test the proposed algorithm for optimization of MPTCP connection, we compare the results of the algorithm given in [9] to α-best fit algorithm’s performance on randomly generated bandwidth values for a Multi Path TCP connection.


## 7. Results and Discussion

During the migration of an instance, the service downtime is detected using a packet sniffer (Wireshark). The instances are pinged every millisecond, and the time of their unavailability is measured in terms of the packets not received by them during the duration of the migration process.It can be inferred from Figure4 that when the migration of a CentOS 7 instance with 16 GB RAM is conducted, we have a drop for a significant amount of time in the number of ICMP packets transmitted to the OpenStack instance. This dip is shown graphically in the figure and pertains to the service downtime.Figure 4
Depiction of service downtime in terms of ICMP packets sent during the migration process of an instance.We have also measured the service downtime for 50 migrations of “Cirros” instances with different RAM specifications, and a graph displaying avg downtime vs. RAM of the instance being migrated is shown in Figure5.Figure 5
The variation of average service downtime with RAM for a “Cirros” instance.The results in Figure5 show that with increasing RAM size of the instance, the service downtime is increased. This can be justified by considering that with the increase in RAM, the number of memory pages to be transferred in each iteration of the precopy algorithm increases as per equation (1). This leads to higher number of pages dirtied in every iteration. When the WWS is obtained, or the cap on the number of iterations is reached, with increasing RAM, the amount of data to be transferred increases as seen from equations (2) and (3). Hence, there is an increase in the service downtime.


### 7.1. Results of MPTCP Implementation without Optimization

After the measurement of service downtime in a single path TCP scenario, MPTCP capability is enabled, and the same procedure is carried out using two paths for the MPTCP communication between the compute hosts. The concept for performing this experiment is that with increased number of paths, the VM data will be divided into smaller streams/flows of data and transferred from both the paths at the same time, speeding up the data transfer. The results obtained provide the proof that the assumption made earlier is correct. While the average service downtime with a single path connectivity for data transfer between the compute hosts is 1.12 seconds, by increasing just one path, it is reduced to 0.58 seconds. Hence, it is observed that the service downtime has been reduced by 48% of the original value.Encouraged by the results in Figure6, the number of MPTCP paths was increased to three. The service downtime of random migrations using single path TCP, MPTCP with two paths, and then with three paths has been plotted.Figure 6
The comparison of variation of average service downtime with RAM for a “Cirros” instance using Normal TCP and MPTCP with two paths.While there is a noted improvement with the increase in the number of paths by one, we observed that by adding the third path, the decrease in the service downtime is not satisfactory. The service downtime for random migrations using two path MPTCP connectivity and three path MPTCP connectivity has also been plotted.The justification for this has been provided in Sections3 and 4, which elaborate the functioning of the precopy mechanism with MPTCP and the need for optimization of MPTCP. The increase in the number of active paths to three for data transfer caused an increase in network congestion in the OpenStack management network, which reduced the bandwidth utilization. As the value of γ dropped, the factor xγ dropped as well, thus accounting for the reduced performance as per equation (10).Theα-best fit algorithm proposed in Section 4 is tested for random bandwidth values of the available paths. We have compared the results obtained from the run of the proposed algorithm 4.1 to those obtained after the run of the algorithm given by Joshi and Kataoka for ONS computation [9].


### 7.2. Results from theα-Best Fit Algorithm

In order to demonstrate the comparison of the performance of the algorithm for optimization of MPTCP subflows proposed in this paper, and of the algorithm given in [9], tests were conducted in different aspects. The first aspect to be considered is the optimal number of subflows for different bandwidth demands. Evident from Figure 7(a), this is pretty much the same for both the algorithms. This means that for different values of ∝d, the same number of subflows will be created using either of the algorithms.Figure 7
Performance comparison of theα-best fit algorithm with SFO [9] in terms of bandwidth utilization. (a) Variation in the number of subflows with the increasing bandwidth demand (α). (b) Variation in the aggregate bandwidth (∝s) for a fixed number of subflows. (c) The variation of aggregate bandwidth of the sub flows with time. (d) Variation of the aggregate bandwidth of the subflows with the bandwidth demand. (e) The performance of the α-best fit algorithm in case of path failure.
(a)
(b)
(c)
(d)
(e)Figure7(b) shows the variation of the aggregate bandwidth of the subflows (∝s) with the demand of bandwidth (∝d) for a given value of ONS, thereby displaying the comparison between the α-best fit algorithm and SFO [9]. It can be inferred from the graph that our proposed algorithm delivers precisely the same amount of bandwidth as required during an MPTCP session.Figure7(c) shows the variation in the aggregate bandwidth of the subflows as a function of time. The MPTCP session bandwidth demand is assumed to be fixed at 50 Mbps. It can be seen that the range of variation of the aggregate bandwidth of the optimal paths is narrower, and the extremities are closer to the demand value in case of the α-best fit algorithm.Figure7(d) shows the relationship between ∝s and = ∝d. As ∝d increases, ∝s increases in both the cases. However, in the algorithm given in [9], the rise is steeper in comparison with the α-best fit proposed algorithm. This means that the algorithm proposed in this paper adheres to the demand closer than the initial algorithm. Hence, the bandwidth usage is optimized.In general, the utilization of bandwidth for the same demand in theα-best fit algorithm is more optimized than the initial algorithm. SFO [9] leads to wastage of bandwidth, as more bandwidth than required is taken up for the same data transfer. In case of cloud environments, this makes a huge difference, as the bandwidth is in the order of Gbps.Finally, the action taken by theα-best fit algorithm in case of path failure is illustrated in Figure 7(e). Consider four paths with bandwidths 5, 10, 15, and 20 Mbps. If the demand is fixed to 25 Mbps, and the number of ONS is two, then initially, the 20 Mbps and 5 Mbps paths are chosen to put the subflows on. However, in case of the failure of the 20 Mbps path, the algorithm invokes the path_failure function and chooses the 15 Mbps and 10 Mbps paths to put the subflows on. Hence, the α-best fit algorithm offers a greater degree of optimization along with failure protection when compared to SFO [9].


## 7.1. Results of MPTCP Implementation without Optimization

After the measurement of service downtime in a single path TCP scenario, MPTCP capability is enabled, and the same procedure is carried out using two paths for the MPTCP communication between the compute hosts. The concept for performing this experiment is that with increased number of paths, the VM data will be divided into smaller streams/flows of data and transferred from both the paths at the same time, speeding up the data transfer. The results obtained provide the proof that the assumption made earlier is correct. While the average service downtime with a single path connectivity for data transfer between the compute hosts is 1.12 seconds, by increasing just one path, it is reduced to 0.58 seconds. Hence, it is observed that the service downtime has been reduced by 48% of the original value.Encouraged by the results in Figure6, the number of MPTCP paths was increased to three. The service downtime of random migrations using single path TCP, MPTCP with two paths, and then with three paths has been plotted.Figure 6
The comparison of variation of average service downtime with RAM for a “Cirros” instance using Normal TCP and MPTCP with two paths.While there is a noted improvement with the increase in the number of paths by one, we observed that by adding the third path, the decrease in the service downtime is not satisfactory. The service downtime for random migrations using two path MPTCP connectivity and three path MPTCP connectivity has also been plotted.The justification for this has been provided in Sections3 and 4, which elaborate the functioning of the precopy mechanism with MPTCP and the need for optimization of MPTCP. The increase in the number of active paths to three for data transfer caused an increase in network congestion in the OpenStack management network, which reduced the bandwidth utilization. As the value of γ dropped, the factor xγ dropped as well, thus accounting for the reduced performance as per equation (10).Theα-best fit algorithm proposed in Section 4 is tested for random bandwidth values of the available paths. We have compared the results obtained from the run of the proposed algorithm 4.1 to those obtained after the run of the algorithm given by Joshi and Kataoka for ONS computation [9].


## 7.2. Results from theα-Best Fit Algorithm

In order to demonstrate the comparison of the performance of the algorithm for optimization of MPTCP subflows proposed in this paper, and of the algorithm given in [9], tests were conducted in different aspects. The first aspect to be considered is the optimal number of subflows for different bandwidth demands. Evident from Figure 7(a), this is pretty much the same for both the algorithms. This means that for different values of ∝d, the same number of subflows will be created using either of the algorithms.Figure 7
Performance comparison of theα-best fit algorithm with SFO [9] in terms of bandwidth utilization. (a) Variation in the number of subflows with the increasing bandwidth demand (α). (b) Variation in the aggregate bandwidth (∝s) for a fixed number of subflows. (c) The variation of aggregate bandwidth of the sub flows with time. (d) Variation of the aggregate bandwidth of the subflows with the bandwidth demand. (e) The performance of the α-best fit algorithm in case of path failure.
(a)
(b)
(c)
(d)
(e)Figure7(b) shows the variation of the aggregate bandwidth of the subflows (∝s) with the demand of bandwidth (∝d) for a given value of ONS, thereby displaying the comparison between the α-best fit algorithm and SFO [9]. It can be inferred from the graph that our proposed algorithm delivers precisely the same amount of bandwidth as required during an MPTCP session.Figure7(c) shows the variation in the aggregate bandwidth of the subflows as a function of time. The MPTCP session bandwidth demand is assumed to be fixed at 50 Mbps. It can be seen that the range of variation of the aggregate bandwidth of the optimal paths is narrower, and the extremities are closer to the demand value in case of the α-best fit algorithm.Figure7(d) shows the relationship between ∝s and = ∝d. As ∝d increases, ∝s increases in both the cases. However, in the algorithm given in [9], the rise is steeper in comparison with the α-best fit proposed algorithm. This means that the algorithm proposed in this paper adheres to the demand closer than the initial algorithm. Hence, the bandwidth usage is optimized.In general, the utilization of bandwidth for the same demand in theα-best fit algorithm is more optimized than the initial algorithm. SFO [9] leads to wastage of bandwidth, as more bandwidth than required is taken up for the same data transfer. In case of cloud environments, this makes a huge difference, as the bandwidth is in the order of Gbps.Finally, the action taken by theα-best fit algorithm in case of path failure is illustrated in Figure 7(e). Consider four paths with bandwidths 5, 10, 15, and 20 Mbps. If the demand is fixed to 25 Mbps, and the number of ONS is two, then initially, the 20 Mbps and 5 Mbps paths are chosen to put the subflows on. However, in case of the failure of the 20 Mbps path, the algorithm invokes the path_failure function and chooses the 15 Mbps and 10 Mbps paths to put the subflows on. Hence, the α-best fit algorithm offers a greater degree of optimization along with failure protection when compared to SFO [9].


## 8. Conclusion and Future Enhancements

As can be shown, MPTCP can be quite useful in a cloud setting for live VM migration. This capacity reduced service downtime, allowing for enhanced service quality. The results in Section6 show that adding a single active data channel reduced service downtime by nearly half. In addition to several paths for data transfer between the host machines, the service downtime was decreased from 1.12 seconds to 0.58 seconds. Reduced service downtime enhances end-user experience and overall cloud performance in terms of responsiveness and reliability. We addressed the necessity to optimize MPTCP connections by providing an algorithm 4.1. A more elaborate approach was presented that takes into account not only the optimization of subflows but also the improvement of reliability and performance in case of path breakdown.The proposedα-best fit approach can be used in conjunction with the precopy in the cloud to improve the efficiency of live VM migration. The α-best fit algorithm saves a lot of bandwidth by optimizing its consumption. The algorithm’s failure prevention clause boosts its reliability. Also, the aggregate bandwidth of the α-best fit algorithm’s subflows is less than the SFO’s subflows [9]; yet, it meets the bandwidth requirements. Our results also show that the aggregate bandwidth of subflows from the α-best fit method changes within a narrow region near the bandwidth requirement.The service downtime for live migration of instances can be further reduced by using a separate migration method in conjunction with ours. For example, adaptive memory compression [24], CR/TR motion [30], delta compression [5], and CPU scheduling for synchronization have all been added to precopy. These changes will reduce service downtime. Others like the postcopy algorithm [31] or a hybrid strategy that incorporates the benefits of both precopy and postcopy algorithms can also be employed in the cloud using MPTCP in future.



---

*Source: 2060875-2022-04-22.xml*