# Visuo-Haptic Mixed Reality Simulation Using Unbound Handheld Tools
## 5 Discussion and Future Work  
In this paper, we have demonstrated a novel VHMR concept in which the users work with unbound real-life tools and receive encountered haptic feedback. We identified simulation processes, then picked and synthesized solutions to realize a proof-of-concept setup. However, our solutions could be replaced with other ones. The most essential requirement of the proposed concept is the precise encounter of the haptic-tip and the tooltip. The experiments in Section 4.1 showed that the tool tracker's error in measuring the position of the haptic-tip is just over than 0.15 mm when the tip is stationary, and less than 1 mm when it is moving. For haptic rendering, we used a simple, straightforward approach to build our haptic model and measured force versus deflection data from a real-life object. Because our objective was to assess the accuracy and repeatability of the system in replicating the haptic stimuli. The experiments in Section 4.2 rendered forces up to 12 N, while the accuracy and repeatability are less than 0.2 N. These values are possibly harsh for a surgical soft-tissue manipulation task, which is a potential application for our system, but they are mostly due to our rugged tennis ball scenario. This intuitive scenario enabled evaluation of our system by random users without specific field experience at this phase of development. However, the results show that it is worth implementing sophisticated haptic rendering algorithms to our system for future studies, where we can simulate specific scenarios to fine-tune the individual subsystems based on expert user evaluations. Then, we evaluated the synchronization between visual and haptic rendering. We obtained position data of the same point from both sources and compared them; we found the synchronization error to be just over 0.2 mm. This result is very promising considering the viewing distance, which was more than half a meter. Finally, we conducted a preliminary face and content validation study in which 15 participants tested the system and provided feedback with a questionnaire. Although the feedback was very positive in terms of the overall system and general visuo-haptic perception, issues were noted in tooltip overlay alignment and haptic rendering of the first encounter. It should be noted that two of the top three scoring items are about the positive effect of using unbound tools on the realism in simulation. The other one confirms the success of visuo-haptic synchronization. The ETHD is the enabling component for our unbound tool concept. On the other hand, common features such as rendering haptic texture and providing 6-DoF interaction is a challenge for an ETHD compared to a conventional kinesthetic haptic device. A possible solution for texture rendering is to use a wide end-effector with multiple portions of physical textures, similar to the work of [27]. Then a suitable portion can be encountered with the tool at different times. Currently, our system allows unilateral interaction with a virtual object. We plan to investigate automatic lock and release of the tooltip by the integrated gripper. This will allow bilateral interaction and enable the use of grasping tools such as a forceps in surgical simulation, or pliers in a maintenance training scenario. When the tooltip is locked in the gripper, the ETHD will temporarily be a conventional haptic device that enables 6-DoF interaction. Another constraint of our solution is the haptic workspace, such that it is small in size and limited in directions of interaction. Our \(1.12\times 10^{-3}\) m\({}^{3}\) translational workspace is almost half the one of a Force Dimension omega.6 haptic device, which also has a parallel-kinematic structure and a similar footprint. But the workspace of a conventional device is also used for rendering free motion, whereas in our case the free motion was constrained by the optical tracking workspace, which was \(75\times 10^{-3}\) m\({}^{3}\). Besides, our system rendered ten times greater force and stiffness.