## 2 System Model and Problem Formulation
The signal received by the IoT receiver is expressed as Equation (5):  
\[y_{R}=\sqrt{\beta P}\ h_{DR}\Big{(}h_{SD}+H_{ID}^{T}\Psi H_{SI}\Big{)}\ sq+ \sqrt{P}\Big{(}h_{SR}+H_{IR}^{T}\Psi H_{SI}\Big{)}\ s+z_{R} \tag{5}\]  
where \(z\sim\mathcal{CN}(0,\,\sigma_{R}^{2})\) represents the additive white Gaussian noise (AWGN) at the IoT receiver with zero mean and variance \(\sigma_{R}^{2}\). The task of the receiver is to detect the value of \(q\). The second term on the right-hand side of Equation (5) includes the interference from the direct path and reflections via the RIS. Ideally, this interference should be eliminated by the self-interference cancellation (SIC) circuit, as referenced in [33]. However, perfect SIC is not always achievable, and errors may occur during the SIC process, leading to potential decoding inaccuracies. As a result, the signal received at the IoT receiver for decoding the IoT device's transmission, after imperfect SIC, can be written as Equations (6) and (7):  
\[\overline{y}_{R}=y_{R}-\sqrt{P}\Big{(}h_{SD}+H_{IR}^{T}\Psi H_{SI}\Big{)}\ \delta \tag{6}\]\[=\sqrt{\beta P}\,h_{DR}\Big{(}h_{SD}+H_{ID}^{T}\Psi H_{SI}\Big{)}\,\,sq+\sqrt{P} \Big{(}h_{SD}+H_{IR}^{T}\Psi H_{SI}\Big{)}\,\,(s-\delta)+z_{R} \tag{7}\]  
where \(\delta\) represents the estimated signal at the IoT device. In general, \(s\) and its estimate \(\delta\) can be considered as jointly Gaussian with a specified correlation coefficient [39]. Specifically, \(s\) can be expressed as \(s=\rho\hat{s}+e\), where \(\hat{s}\sim\mathcal{CN}(0,1)\) and the estimation error \(e\sim\mathcal{CN}\big{(}0,\,\sigma_{e}^{2}\,\big{[}1+\sigma_{e}^{2}\big{]}\big{)}\), which is independent of \(\hat{s}\). Therefore, based on Equation (7), the SINR at the IoT receiver is expressed as Equation (8):  
Figure 1.0  
where \(\eta\in[0,\,1]\) represents the level of imperfection in the SIC process [39]. Finally, the achievable data rate at the IoT receiver is expressed as Equation (9):  
Figure 1.1  
One of the most popular RL algorithms is Q-Learning, which learns an action-value function \(Q(s,a)\) to estimate the expected return of taking action \(a\) in state \(s\). The Q-value function helps the agent choose actions that maximize long-term rewards. The Q-learning algorithm updates the Q-value using Equation (12):  
Figure 1.2  
where \(\alpha\) is the learning rate, \(\gamma\) is the discount factor, and \(max_{a^{\prime}}Q(s_{t+1},a^{\prime})\) is the maximum Q-value for the next state. By repeatedly updating the Q-values based on observed rewards, the agent learns to make better decisions over time. Figure 2 provides a schematic view of the RL process, highlighting the interaction loop between the agent and the environment. Figure 1.3 Schematic view of standard RL tasks. Figure 2 illustrates how the agent continuously interacts with the environment to learn optimal decision-making strategies. At each step, the agent observes the current state of the environment and selects an action based on its policy. This action influences the environment, resulting in a transition to a new state, and the environment provides the agent with a reward reflecting the value of the action taken. The agent then uses the new state and reward to update its policy, aiming to maximize long-term rewards. This cycle of action, observation, and reward repeats continuously, enabling the agent to improve its behavior over time and learn to make better decisions. The arrows in the figure represent the flow of information between the agent and the environment in a learning loop.