## 4 DQN-Based Visual Servoing with FOV Feature Constraints  
Based on the establishment and training of the Q-network in the previous section, we constructed the DQN-based uncalibrated IBVS framework, in which we employ the Kalman filter for mapping matrix estimation, as we did in our previous works [36]. The proposed overall framework is shown in Figure 5, and the overall framework contains a total DQN feature constraint module, feature-motion mapping matrix estimation module, and image processing module. For the image processing module, the main function is to receive the camera image information to extract the object feature \(S\). The estimator's function in the mapping matrix estimation module is to calculate the matrix \(J_{\text{s}}\) based on the real-time feature \(S\).  
For the DQN module, the operation mechanism of its main body is consistent with Figure 2. However, to better integrate with the actual uncalibrated visual servoing task, we apply feature screening to the real-time multidimensional feature information \(S\) from the camera. For the uncalibrated visual servoing task of the 6-DOF manipulator, the number of features used are generally \(n\geq 3\). When there are multiple features in the camera's FOV, if  
Figure 4: Q-value recordings of features at different FOV states S, in an offline camera FOV environment. (a) Q-value record of features at state (3, 1), (b) Q-value record of features at state (1, 6).  
any feature is close to the boundary line between the VS mission area and the constrained area of features, as depicted in Figure 1, the method in Figure 6 always ensures finding a feature closest to that boundary line, and this feature is used as the object of the DQN module constraint, thus realizing FOV constraint control for each feature in the visual servoing task. In addition, for the action output of the DQN feature constraint module, the joint velocities \(\dot{\theta}=\dot{J}(\theta)^{+}V_{c}\) can be obtained from the output variable \(V_{c}\) through the pseudoinverse \(\dot{J}(\theta)^{+}\) of the manipulator's Jacobian matrix. Joint variables \(\theta_{DQN},\dot{\theta}\) are obtained with numerical integration and differentiation. When the output \(\theta_{KFIBVS}\) controls the hand-eye camera, causing the feature to enter the camera's FOV danger area (feature constraint area) and potentially escape the FOV, the DQN constraint module adaptively outputs a constraint complementary variable \(\theta_{DQN}\), which constrains the feature within the FOV's safe area (VS mission area).  
In the framework we designed, we consider three sources of delay in the DQN component: (1) the processing time for converting visual features from the camera's virtual FOV environment into state variables; (2) the computation time required for the Q-network to infer appropriate constraint actions; and (3) the process of converting the DQN output actions into robot joint variables. We set the sampling frequency of the uncalibrated IBVS controller to 2.5 Hz (i.e., the controller calculates new control inputs every 0.4 s and sends them to the robot for execution) to ensure smooth control of the robot. Therefore, the time from receiving the state to generating an action in the DQN component must also be synchronized with the IBVS controller. If the DQN computation delay is too long and cannot be synchronized with the sampling frequency of the IBVS controller, the constraint actions generated by DQN may lag behind the IBVS control output, leading to inaccurate control commands for the robot. Especially in real-time applications, the accumulation of delays may affect the system's stability and convergence speed.  
Figure 5: The proposed DQN-based uncalibrated IBVS framework.