## 2 Research Background and Related Works  
Heterogeneous V2X is based on VANETs, WLAN, LTE and 5G networks, centered on cloud computing and cloud storage platforms, and assisted by distributed nodes (base stations, roadside units, vehicles and mobile devices, etc.) deployed at the edge of the network. Finally, an intelligent heterogeneous network with edge-cloud collaboration is formed, as shown in Figure 1. The paper discusses in detail the congestion control in heterogeneous V2X based on this architecture. In recent years, the level of car ownership is rapidly increasing, yet the increment of road traffic facilities is very limited, and the consequent problem is increasingly serious traffic congestion. Moreover, the high density of vehicles is very likely to lead to channel congestion, which seriously affects the efficiency of information dissemination. Therefore, scholars at home and abroad have conducted a lot of research on the congestion control problem in vehicle networks. Congestion control has been a hot topic in network transmission research. Algorithms such as NewReno [19], Cubic [20], Vegas [21] and Westwood [22] are traditional TCP algorithms. Among them, NewReno and Cubic are suitable for traditional Internet network environments, use the loss of packets to judge whether there is congestion, and reduce  
Figure 1: Network Architecture of Heterogeneous V2X. the congestion window. Vegas uses delay to determine whether there is congestion. Westwood predicts the transmission capacity of the link to determine whether it is congested, determines the transmission speed according to the number of returned ACK packets, and further adjusts the window size and slow start threshold. In recent years, with the rapid development of the Internet of Vehicles (IoV), scholars at home and abroad have conducted a lot of research on congestion control of the IoV. Based on the designed information delivery rate model, the literature [23] proposes a distributed control strategy for IoV channel congestion to ensure the efficiency of basic safety information dissemination. For congestion control at the MAC layer in the IoV, the literature [24] proposes a congestion control strategy based on a forbidden search algorithm to determine whether to initiate congestion control by comparing with the threshold value. Recent experiments by [PERSON] et al. in the literature [25] have proposed a distributed congestion control strategy based on network utility maximization theory, named Utility-Based Rate Congestion Control (UBRCC). This algorithm obtains the packet sending rate by updating the \"price\" of vehicle congestion, and finally realizes the channel allocation for the safety requirements of an individual vehicle. Previous work on congestion control has indicated that these TCP algorithms are mainly packet loss feedback-based protocols, delay feedback-based protocols and link capacity-based protocols. The mechanism of each of these protocols is to use some defined set of rules to adjust the size of the congestion window. However, due to the significant characteristics of vehicle communications, namely complexity and high-speed mobility, it is obvious that the algorithms above cannot be well-adapted to vehicle communications. Therefore, researchers have proposed a TCP protocol based on learning. Compared with the other three types of traditional TCP protocols, TCP protocols based on learning can better adapt to the network state, and actively learn the network environment parameters to form more comprehensive TCP congestion control rules. Due to the powerful learning capability of deep reinforcement learning (DRL), and with advantages such as autonomous decision making, it can be used to solve problems related to the large state space and action space in vehicular networking, making great progress in areas such as unmanned driving [26; 27; 28], optimal scheduling, intelligent decision making and gaming. DRL has been of considerable interest to the congestion control community in recent years.