# Transportation Mode Selection Using Reinforcement Learning in Simulation of Urban Mobility
## 4 Experimental Setup
### Environment and Ground Rules  
All initial conditions and traffic configurations are described in detail to ensure the reproducibility of the experimental setup used in this study. The experiments were conducted using SUMO 1.16 simulation software, and a campus-scale traffic network of 1 km \(\times\) 1 km was configured. The simulation network includes basic elements such as roads, pedestrian crossings, and traffic lights. In addition, the same area can be recreated using the Open Street Map (OSM) provided by SUMO to replicate the environment. To test different traffic densities, 100, 200, 300, and 400 vehicle/pedestrian configurations were used. The origin and destination points were randomly assigned among all valid nodes in the network. The traffic lights were configured according to the default algorithms of SUMO, and each experiment lasted until the last vehicle completed its journey. The traffic density scenarios were low-density traffic (scenarios where the starting points were far apart), medium-density traffic (scenarios where the starting points partially overlapped), and high-density traffic (scenarios where the traffic lights caused congestion). Vehicle speed limits and congestion measurements were provided by the default settings of SUMO and the TraCI interface. In the future, it is planned to visualize the parameters used, such as starting points, traffic density distributions and traffic light durations, and to document the simulation scenarios in detail. When designing the infrastructure of the learning environment, various vehicles, traffic lights, and the traffic congestion should be taken into consideration. For this reason, it is crucial to find out which vehicles travel faster to a gathering area, taking into account that each traffic environment has its own variables. Although vehicles are not homogeneously distributed in real-world scenarios, we deemed it important to use the same number of each type of vehicle in this study for a fairer evaluation of the experimental results. Reward mechanisms are not based on randomness but on some mathematical distribution functions to provide a certain mathematical basis. In this way, it is ensured that it can be used and generalized in different environments. Four different transportation groups were used in the simulation environment; these are the following: cars, motorcycles, bicycles, and pedestrians. As soon as the simulation environment starts, 400, 800, 1200, and 1600 vehicles, each in equal numbers, appear in the environment to travel from a completely random point to the destination point. In order to test the system performance, a large number of generated vehicles were tested. One of the main objectives here is to evaluate the performance of the SUMO environment and to observe the negative impact of waiting vehicles in heavy traffic. Each transportation vehicle acts in accordance with the traffic rules on the way to the destination point. In the experimental environment, there are various roads that can be found on a university campus, such as pedestrian-only roads, pedestrian-bicycle-only roads, or roads open to all vehicles. This leads to different advantages for the randomly distributed traffic population. A calculation is performed based on the arrival time for the vehicles to find the best route. The optimum arrival time is the average arrival time of the vehicles in that traffic. In this way, late arrivals and early departures are penalized with this reward mechanism. The reason for penalizing those who wait too long is to prevent unnecessary waiting for a vehicle that arrives at a point early. This will cause an overall delay in the system as the vehicles in this situation cannot be re-routed. The main goal is to reach the event area within the best time by traveling closest to the average arrival time. As given in Algorithm 1, a value-based algorithm was used to perform transportation mode selection.