# A Brief Review of Multipath TCP for Vehicular Networks
## 3 Congestion Control and Scheduler
### Congestion Control
#### 3.1.2 Congestion Control with AI  
Artificial intelligence (AI) has attracted people's attention and has changed our lives in many aspects, such as health care, manufacturing, navigation, social media, etc. Meanwhile, the above-mentioned conventional MPTCP congestion control algorithm only employ coarse-grained pre-defined rules. Therefore it has some limitations, such as the difficulty in adapting to environmental changes, the bufferbloat problem in heterogeneous networks, underutilization of bandwidth, etc. Hence it is important to apply AI in MPTCP studies. Some studies apply AI technology in multipath TCP congestion control to tune the sender rate dynamically according to the surrounding environment. In [57], a deep reinforcement learning (DRL)-based framework, namely DRL-CC, is proposed. DRL-CC is an experience-driven MPTCP congestion control algorithm. It mainly consists of two processing steps. Firstly, in order to simplify the sophisticated network scenario, all of the possible states of active flows are processed by a representation network to generate a representation. Variable input is possible by using long short term memory (LSTM). Then the representation is fed into the actor-critic networks to derive an action for congestion control. Continuous control is realized by integrating the LSTM into an actor-critic network framework, and thus dynamic and sophisticated state spaces can be handled. Besides deep reinforcement learning (DRL), the characteristic of centralized control of a software-defined network (SDN) is also deemed to be a solution to MPTCP studies. A model free SDN-based framework, called FNDRL-CC, is proposed in [58]. The major components of FNDRL-CC include a representation network, Actor-Critic, and fuzzy normalized function. The fuzzy normalized function is for solving high dimensional problems. The representation is utilized to train the actor and critic. Then the Actor-Critic network together with the representation network provide a congestion control action to the target flow. To deal with the diversity paths of the heterogeneous networks, [PERSON] et al. [59] proposed a learning-based approach, called SmartCC. The sender learns the congestion control rules by observing environments and conducts the congestion control across subflows. In SmartCC, the training part and execution part are decoupled. The SmartCC contains three stages. At the bootstrapping stage, the existing MPTCP congestion control rules are utilized until the state is found in the rule table. If the states have not been optimized yet at the early stage, the e-greedy method is adopted. At the offline training stage, historical experiences are applied to train a congestion control rules by a trainer. A hierarchical tile coding is adopted to map the continuous CC states into discrete state. At the online decision stage, the agent synchronizes with the trainer and makes online decisions. [PERSON] et al. [60] utilized reinforcement learning and cross layer information in a congestion control problem for satellite communications. Named data networking (NDN) is a receiver driven transport architecture. Unlike the commonly used IP address-based routing, NDN has the characteristic of name-based routing, and source uncontrollability. [PERSON] et al. [61] proposed ACCP in named data networks. The ACCP has two phases. First, the source of congestion for each node is predicted by the deep learning-based model. Second, the network congestion level is estimated based on the result of the first phase. The congestion control of entire network would be conducted according to the result of the second phase. ACCP performs better than CHoPCoP [62] and ICP [63] in terms of many aspects such as losses. Table 2 summarizes the studies on MPTCP congestion control.