title: Radio Resource Allocation in Cellular V2X: From Rule Based to Reinforcement Learning Based Approaches
Creators: name: Hegde, Anupama Ramesh
Publisher: name: Friedrich-Alexander-Universität Erlangen-Nürnberg (FAU)
Year: 2024

description: The wireless communication technology has gained significant attention in the transportation industry over the recent years. Cellular Vehicle-to-Everything (V2X) communication facilitates the information exchange among road users (such as vehicles, pedestrians etc.) and the infrastructure with an intention to improve the overall road safety, driving comfort, traffic efficiency and save energy. Advanced use-cases aim towards enhancing key functionalities of vehicle automation by means of sensor data sharing and cooperative maneuver &amp; trajectory planning. The introduction of the PC5 interface for sidelink (SL) communication within the mobile communication systems, supports direct exchange of messages between users, independent of the cellular network infrastructure. Two types of radio resource allocation modes are supported in Cellular V2X: managed mode and the unmanaged mode. In the managed mode, a User Equipment (UE) remains connected to the cellular network and the process of resource allocation is coordinated by the base station. In the unmanaged mode, a UE selects its radio resources from a pre-configured resource pool without any assistance from the base station. Originally both these modes were developed by considering that the vehicles exchange periodic messages which are safety-critical in nature. The existing rule based radio resource allocation algorithms in both the modes are unable to adapt their selection parameters in the events of aperiodic data traffic patterns resulting from the diverse generation rules of different V2X messaging protocols. We begin this PhD thesis by carrying out system level network simulations within the developed framework Artery-C, where we study the metrics and parameters that influence the performance of the rule-based radio resource allocation in the sidelink modes. In the first step, we derive the baseline conditions where each mode performs to its best efficiency. By varying the generation rules of the messaging protocols, we further analyze the behavior of the modes when V2X data traffic does not follow a specific pattern. Our studies have shown that both the modes suffer from frequent re-allocations because the messages are no longer periodic and the data sizes do not fit into the previously allocated radio resources. This results in poor utilization of the allocated resources. The unmanaged mode is particularly susceptible to radio resource collisions because the vehicles only have partial awareness about the resource selection decisions of other road traffic participants. As a second contribution, we examine the criteria for sidelink mode selection and the possibilities for a mode switching operation within the sidelink modes and also between the sidelink and the cellular (Uu) modes. We have formulated the strategies for mode switching and calculate the latency in each phase of the mode switch procedure. Although the managed mode has shown advantages with regard to allocation and management of radio resources, it is to be noted that a vehicle cannot remain connected to a base station at all instants of time. Also, switching between different modes is not seamless considering the associated latencies in each phase. This leads us towards the goal of improving the efficiency of the allocation &amp; scheduling of radio resources in the unmanaged mode. After a careful review of the enhancements that can be implemented within the rule based algorithm in the unmanaged mode, it was found that there needs to be a mechanism where vehicles can continuously share their resource selection decisions, adapt their selection parameters and even re-evaluate them (if needed) within a grant period. Therefore, we investigated the Reinforcement Learning (RL) based Artificial Intelligence (AI) approaches that facilitate independent learning, adapting and decision making among spatially distributed vehicular agents. We have developed a fully de- centralized multi agent networked Markovian Decision Process (MDP) model of the Cellular V2X communication network where each agent executes an AI based radio resource scheduler. By extending the actor-critic methodology of the RL, we have derived two variants - Independent Actor Critic (IAC) and Shared Experience Actor Critic (SEAC). The results of our evaluations have indicated that both these schedulers have a potential to achieve better radio resource utilization with a reduced risk of radio resource collisions among the agents. Subsequently, it brings about 15 − 20% improvement in the reliability of the communication link which we regard as a valuable contribution. To summarize, this PhD thesis investigates the performance of the rule based radio resource allocation algorithms in Cellular V2X and proposes the qualitative improvements that can be achieved by means of reinforcement learning.