# Traffic Signal Control with State-Optimizing Deep Reinforcement Learning and Fuzzy Logic
## 1 Introduction  
Nowadays, numerous countries worldwide are experiencing traffic congestion. The rapid growth of the urban population has resulted in a significant increase in the number of vehicles on roads, a root cause of this problem. Consequently, traffic congestion ensues, which results in prolonged travel times, particularly during periods of high traffic demand. In 2023, people living in Bangkok faced a significant loss of time due to heavy traffic, totaling 63 h, which represents an increase of 25 percent compared to the previous year [1]. Traffic lights are the most commonly used tool to manage city traffic. However, problems often arise when traffic lights are not properly adjusted for the number of cars on the road, leading to congestion and longer waiting times. Therefore, in this study, we propose a method that uses deep reinforcement learning with fuzzy logic to improve traffic light control. This method uses data on the number of cars and waiting times to adjust traffic lights according to current traffic conditions, aiming to reduce waiting times and minimize traffic congestion while enhancing safety. When traffic congestion is reduced, vehicles can move more smoothly, reducing the likelihood of accidents or dangerous situations [2] (pp. 208-209). A smooth traffic flow minimizes sudden stops and there is less crowding between vehicles, both common causes of accidents, while improving drivers' responses to various situations and reducing stress, thereby contributing to a safer overall road environment. In 1977, [PERSON] and [PERSON] [3] recommended using fuzzy logic as an approach to controlling traffic lights, which was implemented at a single intersection. Subsequently,many researchers have extensively studied the application of fuzzy logic to traffic signal control. Data were collected from various situations at an intersection, such as the number of vehicles waiting at red lights, the number of vehicles at the intersection to be granted green signals [4; 5; 6], cycle starting time, change probability [5], queue lengths [6; 7], waiting time [8], and total number of vehicles [9]. These parameters have also been used to analyze and determine traffic signal phases [5; 6], or the duration of green signals [4; 7; 10]. The inference engine uses fuzzy rule-based systems and forgoes the need to construct exact mathematical models [7] by creating a rule base that parallels human traffic control reasoning [8; 10; 11]. However, it also has the limitation of being unable to autonomously learn information and thereby resolve highly complex situations. Artificial intelligence (AI) has played a significant role in successfully solving numerous problems. Supervised learning, unsupervised learning, and reinforcement learning [12; 13] have been used to solve multiple problems. These learning methods are classified as machine learning methods, which differ from fuzzy logic control in that fuzzy logic control does not use learning algorithms. Reinforcement learning (RL), also widely known as Q-learning, is a method that involves repetitive learning and the use of calculated Q-values to obtain optimal action selections [14; 15]. The underlying principle mirrors human learning through trial and error, integrating knowledge from real-time data to determine the appropriateness of actions based on the current environmental conditions [16]. There is an evaluator for the results of actions, which is the determined value of the reward. It is used to calculate the Q-value and store it in a table called the Q-Table. This table retains information about all situations and serves as a reference for decision-making [17; 18]. Owing to the complexity of traffic situations and the enormous state space, the limitations of Q-learning in storing data in an excessively large Q-table have led to the integration of deep learning with reinforcement learning. This integrated method is known as Deep Q-learning or the Deep Q-network (DQN).