# A Game-Theoretic Rent-Seeking Framework for Improving Multipath TCP Performance +
## 4 Multiple MPTCP Connections
#### Learning for Continuous Actions  
For learning continuous actions of MPTCP, we propose the adoption of the generally practiced dual-averaging method of [PERSON] [28] in our new game-theoretic setting. Intuitively, the main idea of the learning process is as follows. At each stage of the process, the window adaptation process of path \(i\) generates a new estimate of the individual update of the throughput function at the current action profile, which is possibly subject to loss, channel impairments, and delay uncertainties. With this estimate, one may move into the dual space (for gradients) and translate the performance back into the primal space to select the next action, as shown in Figure 1.  
Starting with some arbitrary estimate, our approach can be captured using recurrence relations as follows (using Equation (5)):  
\[\theta_{i,n,m} = V(\lambda_{i,n,m})\approx V(W_{i,n,m}), \tag{28}\] \[W_{i+1,n,m} = W_{i,n,m}+\gamma_{m}\hat{\theta}_{i,n,m+1} \tag{29}\]  
where \(m\) represents the step of the process, \(\hat{\theta}_{i,n,m+1}\) is the estimate of the individual throughput gradient of the \(i\)th subflow of the \(n\)th MPTCP. To this end, the core components of dual averaging for learning continuous actions are the gradient estimates and the mapping, which determines the actions of the MPTCPs as players. For the MPTCPs' gradient obser  
Figure 1: Primal dual mapping: illustrative representation of dual averaging. vations, we assume that each MPTCP subflow \(i\) has an underlying feedback mechanism that responds with a prediction of the throughput gradients at its current profile. However, such information can be imperfect for several reasons: (i) Predictions are susceptible to errors; (ii) predictions may suffer from channel impairments; (iii) throughput functions may be the stochastic expectations of some random variable (\(\theta_{i}(W_{i})=\mathbb{E}[\hat{\theta}(W_{i};p_{i})]\)[29]), which the MPTCP source can only observe via the realized sample path of gradients. Observe that Equations (17) and (18) hold for both RSF-LB and RSF-DB, which were proposed earlier (see Equations (1) and (2)). Cognizant of such issues, we follow along the lines of [30] and focus on the erroneous feedback mechanism of the form:  
\[\hat{\theta}_{i,n,m+1}=\theta_{i}(W_{n,m+1})+\xi_{i,n,m+1}. \tag{30}\]  
Observe that the noise process \(\xi_{m}=(\xi_{i,n,m})_{i\in\mathcal{N}}\) is an \(L^{2}\)-bounded martingale difference (with respect to filtration) from the history \((\mathcal{F}_{m})_{m=1}^{\infty}\) of \(W_{n,m}\) such that \(\xi_{m+1}\) is not \(\mathcal{F}_{m}\) measurable (however, \(\xi_{m}\) is \(\mathcal{F}_{m}\) measurable). Therefore, \(\xi_{m}\) satisfies the following _zero-mean_ and _finite mean-square error_ properties:  
\[\mathbb{E}[\xi_{m+1}|\mathcal{F}_{m}]=0;\ \ \forall m\in\{1,2,\dots\}. \tag{31}\]  
\[\mathbb{E}[||\xi_{m+1}||^{2}|\mathcal{F}_{m}]\leq\sigma^{2};\ \ \forall m\in\{1,2,\dots\}\text{ and }\sigma\geq 0. \tag{32}\]  
With Equations (31) and (32), the gradient estimates of each path are unbiased and bounded in the mean square conditionally. Specifically,  
\[\mathbb{E}[\hat{\theta}_{m+1}|\mathcal{F}_{m}] = \theta(W_{m}), \tag{33}\] \[\mathbb{E}[||\hat{\theta}_{m+1}||^{2}|\mathcal{F}_{m}] \leq \Theta_{*}^{2};\ \ \forall m\in\{1,2,\dots\}\text{ and }\Theta_{*}>0.\]  
Such an interpretation is generic in the incorporation of a broad range of erroneous processes in our dynamical system of MPTCP data flows, including all compactly supported log-normal, Gaussian, and exponential distributions. Furthermore, both of the above theories can also be relaxed for finite-order moments considering a small bias. Based on the foregoing discussion, an appropriate choice for \(V_{n}\) is \(y_{n}\gets arg\ max_{\theta_{i}}(y_{i},\theta_{i})\). However, such an approach that outputs actions that are closely aligned with \(y_{n}\) has two main problems:  
* Due to uncertainty in the channel dynamics, the _arg max_ action choice can be too aggressive.