# Mambo-DQN: Adaptively Tunes Visual SLAM Parameters Based on Historical Observation DQN
## 3 Method
### Mamba-DQN Agent  
#### 3.2.1 Mamba-DQN Interaction  
The agent interacts with the environment and obtains \(Observation(s_{t},a_{t},r_{t})\) and combines multiple historical observations \((O_{t},O_{t+1}\cdots O_{t+k})\) into a historical experience \(U_{t+k}\). To facilitate the agent's ability to more effectively learn the intrinsic relationships among historical observations and thereby select the optimal parameters for the environment, we encode and embed the absolute positions of prior experiences. As shown in Figure 2, The Mamba block is used as a learner to learn the historical observation trajectory. The DQN network makes action decisions and updates the loss based on the historical experience Q value. The loss function is defined as Equation (3):  
Figure 2: Mama-DQN agent. As shown in Figure 3, ORB-SLAM3 is utilized as the environment for deep reinforcement learning, while DQN serves as the agent. At the beginning of the process, the agent selects an action (a predefined set of parameter values) and applies it to the environment. Upon receiving the action, the environment undergoes a state transition and returns the resulting reward \(R\) along with the updated state to the agent. Based on the received reward and the current state, the agent determines the next action. Through continuous iterations, the agent optimizes its action selection to identify the optimal parameter configuration. The agent interacts with the environment and obtains the corresponding observation \(\mathit{Observation}(s_{t},a_{t},r_{t})\). It then aggregates multiple historical observations \((O_{t},O_{t+1},\ldots,O_{t+k})\) into a historical experience \(U_{t:t+k}\). To enhance the agent's capability of learning intrinsic relationships among historical observations and improving parameter selection for the environment, we encode and embed the absolute positions of prior experiences. As shown in Figure 2, the Mamba block is employed as a learner to capture the historical observation trajectory. The DQN network is responsible for making action decisions and updating the loss based on the historical experience Q value. The loss function is defined as Equation (3):  
\[\mathit{Loss}_{t+i}(\theta)=E_{\pi}\Big{[}r_{t+i}+(max\,Q(O_{t+i})-Q^{*}(O_{t +i}))^{2}\Big{]} \tag{3}\]  
As shown in Algorithm 1, during observation, we not only focus in the current frame state of the agent but also use the historical observations \((O_{t},O_{t+1},\cdots,O_{t+k})\) to form the context. This paper uses the linear time characteristics of the Mamba block to construct a Mamba learner to perform \"experience learning\" on the context. Algorithm 1 obtains the Q values of all historical observations and uses these Q values to calculate the loss for network training. This context-based historical experience update method produces a more robust agent. As shown in Figure 2, when making action decisions, only the maximum Q value of the most recent observation is used for action decision-making. Figure 3: Agent–environment interaction. In the Mamba-DQN framework, the interaction mechanism between historical observations and the Q-network constitutes the core of efficient temporal modeling. At time step \(t\), the agent obtains an observation sequence \(\{O_{t-k},O_{t-k+1},\cdots,O_{t}\}\) from replay buffer and transforms it into high-dimensional representations using the observation embedding function \(\phi_{obs}\):  
\[E_{obs,t-i}=\phi_{obs}(O_{t-i}),\quad i\in\{0,1,\cdots,k\} \tag{4}\]  
Simultaneously, the corresponding action sequence is encoded through the action embedding function \(\phi_{act}\):  
\[E_{act,t-i}=\phi_{act}(a_{t-i}),\quad i\in\{1,2,\cdots,k\} \tag{5}\]  
To ensure temporal consistency, a time shift operation is applied to the action embeddings:  
\[E^{\prime}_{act,t-i}= \begin{cases}E_{act,t-i+1},&\text{if }i\in\{1,2,\cdots,k\}\\ \mathbf{0},&\text{if }i=0\end{cases}