# Autonomous Intersection Management by Using Reinforcement Learning
## 1 Introduction  
Intersection cross-road traffic management is a challenging task. This is due to the desire by cars to share space simultaneously when travelling in different directions. The same place is also sought after by pedestrians for crossing intersections. At a junction, drivers must make a split-second choice while considering their path, the intersection's design, and the speed and direction of other cars, etc. A slight lapse in judgment might result in catastrophic accidents. According to the community database on accidents on the roads in Europe (CARE), the proportion of fatalities in road accidents at intersections was around 11% in 2020 [1]. As a city develops and the population grows, road intersections become the bottlenecks of traffic, leading to traffic congestion, and increased travel time. Moreover, a survey for critical reasons for crashes indicated that drivers cause more than 90% of crashes [2, 3].  
The study of wireless communication for automobiles became popular in autonomous vehicles in the early 2000s, which helps data exchange via vehicle-to-vehicle (V2V) interaction. One of the most widely used dedicated short-range communications (DSRC) is compliant with the IEEE 802.11p standard (physical layer protocol) and the IEEE 1609 standard (network layer protocol). The properties of DSRC in low latency and high mobility make it suitable within the vehicular environment. Numerous autonomous intersection management (AIM) methods that designed advanced intersection-crossing rules without human involvement have been proposed in the literature [4, 5, 6, 7]. However, there is still more that can be done to improve. For example, if multiple requests are sent from different vehicles simultaneously to request passing at an intersection, how should the intersection controller or the negotiation algorithm schedule these requests? This problem must be carefully addressed to prevent unfairness or starvation. In this research, we focus on enhancing the efficiency of crossing policy without demands on prior knowledge. The main idea is that we let the system learn how to schedule the crossing sequence itself by usingthe deep reinforcement learning (RL) algorithm [8]. The agent aims to develop a policy to select actions to achieve the maximum cumulative reward. The object of this kind of learning is for the system to extrapolate, or generalize, its responses so that it acts correctly in situations not present in the training set [9]. In other words, supervised learning only classifies or predicts based on what it learned, that is, exploitation. Intersection management policy is an essential issue in traffic. Many AIM systems have been proposed to enhance performance compared to signal-based approaches. Although AIMs have different designs, they are essentially systems that allocate and schedule various resources. However, in most of these methods, at each time step, they can only consider the frontmost vehicles among incoming lanes (in order to avoid allocating resources to the latter vehicle while the frontmost vehicle has not yet been allocated resources, leading to both vehicles being unable to enter the intersection), then decide whether to allocate resources to these vehicles. In contrast, the signal-based methods can let many vehicles enter an intersection in a short time, as long as the road signal is green. Experiments in [5] show that the performance of the proposed method is worse than the signal-based approach for a high arrival rate. We believe this phenomenon is related to the delay caused by the frequent alternation of right-of-way after extensive computation of internal simulations.  
The remainder of the paper is structured as follows. We present the related work in Section 2. Section 3 describes the overall framework of the proposed DRLAIM system. Section 4 discusses the experimental results. In Section 5, we give the conclusions and future work for this work.