# Traffic Signal Time Optimization Based on Deep Q-Network
## 1 Introduction  
Traffic congestion is worsening in cities around the world as the number of vehicles continues to increase. Traffic congestion delays the movement of vehicles and has a variety of negative consequences, such as increased travel time, waste of fuel, and increased emissions. To address the traffic congestion problem, transportation planners have proposed various transportation-related policies, including a systematic public transportation system or an alternative no-driving system. With the passing of time, the problem of traffic congestion has rapidly worsened. Therefore, there is a need for research on smart traffic signal control (TSC) that directly controls traffic flow.  
Smart traffic management is an important part of transportation planning in smart cities [1]. Owing to the development of technologies such as the Internet of Things, cloud, and big data, the infrastructure of smart cities has been equipped with the latest technologies. Smart transportation management is a platform that can improve the quality of life, and its requirements are growing. Smart TSC is one of the most effective methods for efficiently reducing traffic congestion at intersections. The current traffic signal is a fixed traffic light that allocates time to a preset cycle. Fixed traffic lights always control the traffic flow with the same time distribution, which has the advantage of simplicity and no complexity. Therefore, fixed traffic lights are suitable for regular traffic flows. They are also suitable for intersections where there is not much traffic variation between lanes. However, traffic delays always occur because the actual traffic flow is unpredictable, and dynamism is bound to occur in various traffic flows.  
Smart traffic lights aim to maximize capacity at intersections by controlling the traffic flow according to the environment. Therefore, they perform better than conventional fixed TSC in environments where the traffic deviations are large between lanes at intersections or the traffic volume changes over time are dynamic. In addition, congestion and environmental pollution can be reduced or wasted green light can be minimized [2]. A wasted green light means that, despite a green light being allocated to the lane, there are no vehicles leaving; therefore, the green light allocated is not being used efficiently. Traffic lights at intersections can be managed in two ways: by adjusting the time of the signalor by adjusting the signal phase sequence. The former minimizes a wasted green light by allocating the appropriate time for each situation, whereas the latter allows efficient traffic light control by distributing the green signal to the most needed lanes. For example, a traffic light that wants to minimize the waiting time of the vehicles determines that the lane with the longest waiting time is the lane that needs the longest green signal.  
Various machine learning methods have been used to control traffic signals, such as fuzzy logic [3], neural networks [4], and computational intelligence [5; 6]. However, fuzzy logic, which controls signals through rule-making, makes it difficult to generate effective rule sets when dealing with multiple intersections. Therefore, a traffic control system, that requires learning of dynamic environments while recognizing the surrounding intersections should apply reinforcement learning (RL) to determine the optimal decisions for the environment. This paper proposes an optimal TSC-based RL that adjusts the time of the signal.  
The remainder of this paper is organized as follows. Related studies are given in Section 2. The system model and the proposed technique are formulated in Section 3. In Section 4, the experimental studies are presented and results are discussed. Finally, the concluding remarks are provided in Section 5.