## 2 Materials and Methods
### The Generation of the Virtual Scene
Figure 4: The generation of an interactive interface in which the relative position between the virtual object and the robot arm is consistent with that between the real object and the robot arm. (**a**) The initial interactive interface. (**b**) Generating an interactive interface for virtual objects. \[ \begin{array}{l}\left|\text{x}_{\text{now}}-\text{x}_{\text{saved}}\right|>10^{-5} \\ \left|\text{y}_{\text{now}}-\text{y}_{\text{saved}}\right|>10^{-5}\\ \left|\text{z}_{\text{now}}-\text{z}_{\text{saved}}\right|>10^{-5}\end{array}  \tag{9}\]  
where \(\text{x}_{\text{now}}\), \(\text{y}_{\text{now}}\), and \(\text{z}_{\text{now}}\) represent the current three-dimensional coordinates of the object in Unity, and \(\text{x}_{\text{saved}}\), \(\text{y}_{\text{saved}}\), and \(\text{z}_{\text{saved}}\) represent the three-dimensional coordinates of the object \(\text{obj}_{\text{saved}}\) which are stored in the receiver script. In the virtual scene, because the data type for object coordinates is that of float and the precision of float is approximately 6-9 digits, the threshold is set to \(10^{-5}\). This ensures that objects that have not been moved are not determined as moved objects and prevents the robot arm from unnecessary actions. If identifying moved virtual objects, current three-dimensional coordinates are updated in the saved object message \(\text{obj}_{\text{saved}}\). Then, the object message is sent to ROS via the \(\text{Publish}()\) function. The core Algorithm 2 is given as follows:  
```
Input: name of digital twin obj\({}_{\text{name}}\); Output: moved digital twin obj\({}_{\text{saved}}\);
1:for\(\text{i}=1,2,3,\ldots\text{N}\)do
2:\(\text{obj}_{\text{saved}}=\text{ObjectPoseNow}\Big{(}\text{obj}_{\text{name} }^{(\text{i})}\Big{)}\)
3:if\(\left|\text{x}_{\text{now}}-\text{x}_{\text{saved}}\right|>10^{-5}\) or \(\left|\text{y}_{\text{now}}-\text{y}_{\text{saved}}\right|>10^{-5}\) or \(\left|\text{z}_{\text{now}}-\text{z}_{\text{saved}}\right|>10^{-5}\)then
4:\(\text{x}_{\text{saved}}=\text{x}_{\text{now}}\)
5:\(\text{y}_{\text{saved}}=\text{y}_{\text{now}}\)
6:\(\text{z}_{\text{saved}}=\text{z}_{\text{now}}\)
7:\(\text{Publish}(\text{obj}_{\text{saved}})\)
8:endif
9:endfor
```  
**Algorithm 2**ROS Publisher  
The program sends the target object's category and destination position to ROS via the ROS-TCP-Connector. ROS subscribes to this topic through ROS-TCP-Endpoint. Then, ROS reads the current position of the target object, and the current position of the object determines the coordinates that the robot arm's end effector needs to reach. The server uses the inverse kinematics solver, which can calculate the joint angle of the robot arm according to the position of the end effector to obtain the pose of the robot arm, and the inverse kinematics solver is provided by ROS. Subsequently, the server autonomously controls the  
Figure 5: User moves virtual objects. robot arm to move to the current position of the target object and grasp it, as shown in Figure 6a. Similarly, after ROS obtains the pose of the robot arm at the destination position through the inverse kinematics solver, ROS autonomously controls the robot arm to move to the destination position, as shown in Figure 6b, and places the object at the destination position, as shown in Figure 6c. Finally, the robot arm is controlled to return to the initial position and waits for the next task, as shown in Figure 6d.