# A Hybrid IVFF-AHP and Deep Reinforcement Learning Framework for an ATM Location and Routing Problem
## 3 The Proposed Methodology
### DDQN-RL for Adaptive Route Optimization with Static Traffic-Aware Distance Matrix  
This study proposes a DDQN-RL approach to optimize the route of an ATM cash transport vehicle. In contrast to traditional approaches, a predetermined traffic-adjusted distance matrix is integrated, which allows the reinforcement learning agent to optimize its route, taking into account traffic congestion at a single-period granularity. The developed model uses DDQN-RL to optimize vehicle route decisions under static but realistic traffic constraints. To achieve this, a predetermined traffic-adjusted distance matrix is integrated into the model, which allows the RL agent to internalize the cost structure of high traffic routes and make optimized decisions accordingly. This approach enables the ATM service agent to dynamically adapt to various real-world operational scenarios, including the addition or removal of ATM locations, sudden changes in demand, and real-time re-routing requests.  
The ATM vehicle routing problem is formulated as a Markov Decision Process (MDP), where the state space represents the current vehicle position, ATM demand levels, and vehicle capacity, while the action space corresponds to selecting the next ATM location to visit. Let S denote the state space, where each state is represented as \(s_{t}=\{D_{t},C_{t},L_{t}\}\), where \(D_{t}\) is the ATM demand vector at time \(t\), \(C_{t}\) is the vehicle's remaining cash capacity, and \(L_{t}\) is the current vehicle location. The action space A is defined as the set of all possible next ATM locations, and the transition function \(T(s_{t}\), \(a_{t})\) dictates the movement of the vehicle based on the selected action \(a_{t}\). The agent receives a reward signal \(R(s_{t},\ a_{t})\) after executing action \(a_{t}\), designed to encourage efficient routing while penalizing unnecessary detours and high-traffic routes. The reward function is defined as follows:  
\[R(s_{t},a_{t})=\begin{cases}-d\Big{(}L_{t},L_{(t+1)}\Big{)}-\lambda T\Big{(}L _{t},L_{(t+1)}\Big{)}\text{, \emph{if} }C_{t}\geq D_{(t+1)}(\text{\emph{successful} }ATM\text{\emph{visit}})\\ -\infty\text{, \emph{if} }C_{t}\leq D_{(t+1)}(\text{\emph{vehicle runs out of cash}})\\ -1000\text{, \emph{if} }L_{t}\text{\emph{is revisited unnecessarily}}\end{cases} \tag{19}\]  
where \(d\Big{(}L_{t},L_{(t+1)}\Big{)}\) represents the precomputed shortest distance between two locations based on the NetworkX-generated distance matrix, and \(T\Big{(}L_{t},L_{(t+1)}\Big{)}\) represents the traffic-adjusted cost of traveling between these locations. The parameter \(\lambda\) is a weighting factor to balance travel distance and traffic congestion. The agent aims to maximize the discounted cumulative reward, defined as  
\[Q^{*}(s_{t},a_{t})=E\Big{[}\sum_{k=0}^{\infty}\chi^{k}R(s_{t+k},a_{t+k})\Big{]} \tag{20}\]  
where \(\gamma\) is the discount factor, ensuring future rewards are considered while maintaining computational stability. The DDQN architecture consists of two neural networks: a primary (online) network \(Q_{\theta}\), responsible for action selection, and a target network \(Q_{\theta}{}^{\prime}\), that stabilizes training by providing updated Q-value estimates. The Bellman update equation for Double DQN is given by  
\[Q_{\theta}(s_{t},a_{t})=R(s_{t},a_{t})+\Upsilon Q_{\theta}{}^{\prime}\big{(}s _{t+1},argmax_{a^{\prime}}Q_{\theta}\big{(}s_{t+1},a^{\prime}\big{)}\big{)} \tag{21}\]  
where the online network determines the next action \(a^{\prime}\), but the value is estimated using the target network to mitigate overestimation bias. Training follows an experience replay mechanism, where past transitions are stored in a buffer and randomly sampled for training to improve stability.