# Autonomous Intersection Management by Using Reinforcement Learning
## 3 Methodology
### Priority Assignment Model  
At each time step \(t\), the agent selects an action \(a_{t}\), and then sends action \(a_{t}\) and corresponding requests to the priority assignment model. According to the type of action \(a_{t}\), we will have three different corresponding policies.  
1. **Type 1: Let vehicles with sent requests in a specific lane to pass.** According to the timestamp of requests in the queue, the priority assignment model will append these requested vehicles into the priority list one by one, i.e., the earlier the time, the sooner the vehicle is appended to the priority list. Then the priority assignment model sends the confirm messages and corresponding priorities to those requested vehicles. Note that the closer to the front in the priority list, the higher the priority.
2. **Type 2: Let all vehicles with sent requests to pass.** The priority assignment model receives all requests of vehicles, and then it will append them to the priority list according to the timestamp of requests; i.e., the earlier the time, the sooner the vehicle is appended to the priority list. Then the model sends the confirm messages and corresponding priorities to those requested vehicles. Note that the difference between type 1 and type 2 actions is that type 2 action appends all requested vehicles to the priority list, and type 1 action only appends requested vehicles in a lane.
3. **Type 3: All vehicles are not allowed to pass.** No requested vehicles are allowed to pass; thus, the priority assignment model does nothing.  
As shown in Figure 4, two vehicles have been assigned priorities, and vehicle \(i\) has priority over vehicle \(j\). In Figure 3(a), the trajectories of the two vehicles have a collision and a corresponding collision region. Both vehicles \(i\) and \(j\) have been assigned priorities so that they can enter the intersection. However, the priority of \(i\) is higher than \(j\), thus \(j\) has to wait until \(i\) passes through the collision region. In Figure 4b, the trajectories of two vehicles have no collision. Thus, although the priority of vehicle \(i\) is higher, they can still cross the intersection simultaneously without waiting. Figure 5 shows an example of how the priority assignment model works. The gray region in Figure 5 represents the cooperative zone. In Figure 5a, the agent selects and allows vehicle \(b\) and vehicle \(c\) to pass. Therefore, the priority assignment model will first append leader vehicle \(b\) to the priority list and then append vehicle \(c\) to the priority list.  
Figure 4: An example of interaction between two vehicles, where vehicle \(i\) has priority over vehicle \(j\). (**a**) Trajectories of vehicle \(i\) and vehicle \(j\) have a possible collision. (**b**) Trajectories of vehicle \(i\) and vehicle \(j\) have no collision.  
Figure 5: An example of selecting different actions. (**a**) Agent selects an action to let vehicles in a specific lane to pass. (**b**) Agent selects the action to let all vehicles that have sent requests to pass.  
Finally, the priority assignment model will send the confirm messages and corresponding priorities to vehicles \(b\) and \(c\). Though vehicle \(e\) is also in the same lane as vehicle \(b\) and \(c\), it has not yet entered the cooperative zone, and thus it has not sent a message to request for priority. So the priority assignment model will not append vehicle \(e\) to the priority list. In Figure 5b, the agent chooses to let all vehicles that have sent request messages to pass the intersection. Therefore, the priority assignment model will append these vehicles to the priority list according to the timestamp of requests; i.e., the earlier the time, the sooner the vehicle is appended to the priority list. The vehicle that has a higher priority can go before the one with a lower priority. The vehicle that is not in the priority list is not allowed to enter the intersection. Figure 6 shows the flow of how intersection control works.  
Figure 6: Flow of intersection control policy.