# Transportation Mode Selection Using Reinforcement Learning in Simulation of Urban Mobility
## 3 Materials and Methods
### Reinforcement Learning  
Reinforcement learning (RL) refers to a set of algorithms that address sequential decision-making and are capable of making intelligent decisions based on their local environment. A model that instructs an agent on the course of action to follow in a confined environment to maximize a predetermined total reward is known as an RL algorithm. The agent experiments with various strategies, calculating the overall payoff. Following numerous trials, the algorithm establishes a pattern of behavior by learning which behaviors yield more rewards. The algorithm is able to instruct the agent on what to do in any scenario. Figure 3 illustrates this interaction [39]. In Figure 3, \(r_{t}\) refers to the reward the agent receives from the environment, \(a_{t}\) refers to the actions the agent sends to the environment, and \(s_{t}\) refers to the states the agent is in the environment. At any time \(t\), while the agent is in state \(s_{t}\), it performs the action, receives the reward \(r_{t}+1\), and accordingly moves to state \(s_{t}+1\). The value-based implementation of RL is shown in the formula below [40]. The environment (E) is the world in which the agent operates and makes decisions. The agent is the entity applying the RL algorithm to learn and make decisions. The state (S) is a representation of the current situation of the agent in the environment. The action (A) is the set of all possible moves the agent can make in a given state. The reward (R) is the feedback received by the agent from the environment after performing an action, indicating the immediate benefit of that action. The Q-value (Q(s, a)) is the expected total reward for taking action \(a\) in state \(s\). The environment represents the setting or scenario where the agent operates, serving as the backdrop for the agent's decision-making processes. Actions refer to the choices made by the agent in response to the information it gathers from the environment. These actions can be of two types, continuous-valued or discrete-valued, depending on the  
Figure 3: Types of vehicles used in urban traffic and the desired activity area. specific problem and environment at hand. The state embodies the critical information that characterizes the agent's position and condition within the environment. The main reason we used reinforcement learning (RL) is the ability of this method to provide effective solutions in complex, dynamic, and uncertain environments. The problems encountered in choosing transportation modes are not only static but also involve a number of factors that change over time and interact with each other. Such problems create difficulties in producing optimal solutions with traditional methods because existing solutions are usually based on fixed rules and cannot adapt quickly to environmental changes. Reinforcement learning allows agents to continuously learn to make better decisions at each step by interacting with their environment. The agent receives rewards according to the results of each action and optimizes future actions in light of these rewards. This process helps to find the best solution over time, especially in complex and dynamic systems. In choosing transportation modes, RL algorithms can evaluate the advantages and disadvantages of each vehicle (car, bicycle, motorcycle, or walking) in real time. For example, when road density increases, a motorcycle may be preferred over a bicycle, but these changes must be learned quickly based only on instantaneous data. Traditional optimization and modeling techniques usually work with fixed parameters and focus on providing the best solution valid at a given moment. However, this approach cannot quickly account for instantaneous changes (for example, fluctuations in traffic density). RL has a structure that can dynamically adapt to changes and learn from previously encountered situations. This capability facilitates continuous improvement in the selection of transportation modes and enables more accurate and effective decisions with each new experience.