# DDPG-MPCC: An Experience Driven Multipath Performance Oriented Congestion Control
## 1 Introduction
### How MPCC Works?  
MPCC aims to independently and asynchronously optimize a local utility function defined for each MPTCP subflow. For example, consider an MPTCP connection \(i\) with \(d\) subflows. When looking from the perspective of a subflow \(j^{\prime}\), the sending rate of all the other subflows except a tagged subflow \(j\) can be assumed to be fixed and represented by \(c_{i}^{k}\). Then, the utility function of the tagged subflow \(j\) denoted by \(U_{i}^{j}\) can be computed based on its own sending rate of \(x_{i}^{j}\) and the experienced round trip time \(RTT_{j}\) as [3]  
\[U_{i}^{j} = \Big{(}\sum_{k\
eq j}^{d}c_{i}^{k}+x_{i}^{j}\Big{)}^{\alpha}- \beta L_{j}\Big{(}\sum_{k\
eq j}^{d}c_{i}^{k}+x_{i}^{j}\Big{)} \tag{1}\] \[-\gamma(\sum_{k\
eq j}^{d}c_{i}^{k}+x_{i}^{j})\frac{dRTT_{j}}{dT}\]  
where \(L_{j}\) is the loss rate and \(\frac{dRTT_{j}}{dT}\) quantifies the delay variation observed by the subflow \(j\), and \(0\leq\alpha<1\), \(\beta>3\) and \(\gamma\geq 0\) are the parameters for balancing performance objectives. However, from a higher MPTCP connection level perspective, the utility function of the MPTCP \(i\) can be estimated as  
\[U_{i}=\Big{(}\sum_{k}^{d}x_{i}^{k}\Big{)}^{\alpha}-\max_{k\in d}\{\beta L_{k}+ \gamma\frac{dRTT_{k}}{dT}\}\sum_{k}^{d}x_{i}^{k} \tag{2}\]  
It has been reported that [5], with MPCC we can achieve better performance by decentralizing optimization across subflows. This can be clearly observed in (1) as the utility function or the utility that will lead to rate changes is dependent on its own locally perceived success, loss, and delay statistics. MPCC adopts the concept of **Monitoring Intervals** (MI), which is the duration of a time long enough to gather sufficient statistics for each of the MPTCP subflows. First, MPCC selects a sending rate for the subflow for each MI, and then it computes the utility value of the subflows (after the defined MI interval to gather the required statistics). In this process, the subflow transitions occur in three different states, namely, _slow-start_, _probing_ and _moving_[2; 3; 5]:  
1. In the _slow start phase_, the sending rate is doubled in every MI until the utility decreases. MPCC can utilize the previous experience of the sending rates to begin the probe.
2. In the _probe phase_, MPCC probes the subflow with a higher sending rate \(x+\omega\) and a lower sending rate of \(x-\omega\), where \(\omega\) is the fraction of the total sending rate of all subflows within the MPTCP connection (that is, \(\omega=x_{i}^{k}/\sum x_{i}^{k}\)). It is worth noting that the probing phase helps to decide the direction in which the rate should be modified, which directs the beginning of the moving phase.
3. In the _moving phase_, MPCC estimates the gradient of the utility function by using the utilities and the rates from the two preceding MIs. With the computed gradient, MPCC mandates the rate change or the step increase/decrease in the direction towards the increasing pattern for the utility of the connection. It can be seen that the MPCC inherited such specific and celebrated mechanisms (e.g., rate amplification and changed bounds) from PCC Vivace [3] to seamlessly adjust the speed of data transport for the Internet. For example, if the estimated utility decreases, then it can quickly switch back to the probing phase.