# Transportation Mode Selection Using Reinforcement Learning in Simulation of Urban Mobility
## 2 Related Work  
In traffic optimization studies, transportation mode selection using reinforcement learning is often used to find the best transportation mode. Various studies have been carried out to predict and control a traffic situation using deep learning. In these studies, traffic jam and situation studies were generally carried out [13; 14; 15]. It is also a frequently studied topic for solving complex networks, RL-DRL solutions, and transportation mode selection [16; 17; 18; 19]. The best mode is usually found in such studies, and a reward policy is determined; this process is carried out using MDP. [PERSON] et al. [20] used deep reinforcement learning for a system with traffic lights. In this work, real-world problems were not studied but were discussed theoretically, and the studies focused on reward policies and comparisons with real data, with the results supported by the SUMO simulation environment. Similarly, [PERSON] et al. [21] carried out work to increase efficiency in relation to the inefficient operation of traffic lights and the energy problem caused by long waiting times. They emphasized the need to dynamically adjust real-time traffic information. The problem was solved by using DRL in the model they created. In addition, the reward mechanism was determined using MDP. In order to increase the performance of the proposed model, it was optimized using methods such as Q-learning, which was implemented in the SUMO environment. According to [PERSON] et al. [22], by using Deep Q-Network, the intersections on roads could be autonomously handled. In these studies, a road network determined on SUMO was used and each step was graphed. In general, routing optimization is mentioned, and how the network can be best utilized with learning and reward mechanisms using reinforcement learning is revealed. In order to reduce network delay, [PERSON] et al. [23] created a deep reinforcement learning agent that can automatically adapt to the state of traffic. This agent then suggests customized configurations; however, instead of considering the entire urban transportation network, it just focuses on one vehicle. Order dispatching is a continuous operation, and the state changes of the couriers at each time step resulting from the order assignment action form a sizable but finite MDP that may be resolved by using conventional RL techniques [24]. By choosing the appropriate action when interacting with the environment, the RL aims to maximize the returned reward. A popular model-free RL technique built on the Bellman equation is Q-learning [25]. The typical Q-learning method aims to produce an over-estimation of the Q-value; in contrast, the Double Q-learning algorithm uses two different action-value functions, Q and Q\({}^{\prime}\), where the Q\({}^{\prime}\) function compensates for computing the expected Q-value in the Bellman equation. Additionally, the Double Deep Q Network (DDQN) [26] was created to replace the tabular method of the Q-value with neural networks in order to overcome the disadvantage of discrete states. To increase the learning effectiveness of DQNs, a progressive transfer knowledge learning strategy with related features was proposed to handle the vast quantities of real-world spatiotemporal trip data [27]. In large-scale ride-sharing scenarios, a multi-agent reinforcement learning (MARL) approach was proposed to analyze ride-sharing data in a decentralized manner [28]. [PERSON] et al. [29] unraveled the complex relationships between transportation mode choices and the factors that influence them, which is valuable for transportation planning and development. However, a better understanding of the underlying influencing factors that shape passengers' comprehensive mode choices for travel, as well as the interactive and nonlinear effects between the two, is needed.