## 2 Materials and Methods  
Urban traffic flow is a highly complex system with numerous dynamics that can change instantaneously. Many parameters in traffic can change in real time. When examining machine learning methods capable of providing solutions that account for these real-time changes, DRL algorithms inherently meet this need. DRL algorithms attempt to learn through trial and error without any prior knowledge of the environment. During the learning process, they employ reward and punishment methods, aiming to maximize the reward. A few examples are parameters such as road conditions, traffic density, vehicle speed, the status and timing of traffic lights and the number of vehicles waiting at lights. Achieving the optimal speed for vehicles is inherently complex for traffic. A block diagram of the system designed to suggest acceleration to drivers considering these variables is shown in Figure 1.  
In Figure 1, Environment represents the traffic simulation environment provided by the SUMO application. At each step of the simulation environment, the observation values obtained using the Traffic Control Interface (TRACI) provided by the SUMO application were recorded in the replay memory. The observation values recorded in the replay memory consist of an array of 11 elements. A certain amount of data from the replay memory were given as input to the created network. There are 3 outputs in the output layer of the developed network. These outputs are in the form of (\"Accelerate\", \"Maintain speed\" and \"Slow down\"). These actions obtained as outputs were sent back to the simulation environment and applied, and new observation values were obtained. The map used in the study pertains to the city of Elazig and was obtained from the open-source application, Open Street Map [25]. Using the city map obtained, three different routes were established. The purpose of selecting these routes used in the study is that they have a heavy traffic flow, many traffic lights and intersections and the data to be used as input in the system are provided by the municipality. The different routes created using the map consist of a total of four lanes, two for arrival and two for departure. These three different routes were incorporated into the SUMO program using the NETEDIT program, where traffic light information was added and necessary adjustments were made to create the simulation universe. The created simulation environment was transferred to the simulation program to prepare the simulation environment [26; 27; 28]. The study employed the Python programming language and TRACI library [29]. All the programs used are open-source applications. In this study, three different scenarios were created to compare the performance of DRL algorithms in finding the optimum speed for vehicles in urban traffic. These algorithms were applied to these scenarios, and the results were analyzed. The routes used for the three different scenarios created for network training are shown in Figure 2.  
A total of 25 traffic lights were added to the maps shown in Figure 2 through the NETEDIT application to be used in the study. The phase durations of the traffic lights used in the study are expressed in Table 2.  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Group Name** & **Traffic Light Id** & **Red (s)** & **Yellow (s)** & **Green (s)** \\ \hline \multirow{4}{*}{**Scenario 1**} & TLS1 & 65 & 2 & 30 \\  & TLS2 & 35 & 2 & 40 \\ \cline{1-1}  & TLS3 & 40 & 2 & 50 \\ \cline{1-1}  & TLS4 & 50 & 2 & 40 \\ \hline \hline \end{tabular}
\end{table}
Table 2: Phase durations of traffic lights used in the experimental study.  
Figure 1: Block diagram of the method used.  
The phase times expressed in Table 2 are the traffic light phase values used in real-world conditions, taken from the city traffic control center.