# Using Game Engines for Visuo-Haptic Learning Simulations
## 3 Proposed Solution
### HaDUI: Haptic Device Integration for Unity
#### 3.2.2 HaDIU--Haptic Manager (HM) Component  
The final functional part of HaDIU is its HM component. According to the deployment diagram, three Unity elements are needed to visualize and operate haptic devices. The first one is an HaDIU operability C# script. This script calls linkage functions in the native library and enable the haptic devices' connectivity. To properly use it, HaDIU's DLL needs to be imported to Unity. The second one is the HM object. This is an object that will only have a HM C# script component. This script will be in charge of connecting the HM component to the HC one to initialize the haptic devices' operability, to retrieve their information, and to send calculated interaction forces to each haptic device. In the first part, the HM C# script starts a thread to enable proper interactions of haptic devices. This thread is set to 1000 Hz, which is the necessary frequency to operate haptic devices. Then, it gets the haptic devices' information and sends it to the haptic devices' objects in the virtual scene. Finally, it retrieves interaction parameters from Unity's physics engine, and it calculates and calibrates the resulting interaction force that occurs during the interaction of the haptic device object with other virtual objects via the desired position and the haptic device position difference. Lastly, the third Unity element considered in this component is a haptic device virtual object. The haptic device object is composed of two spheres objects. One is the haptic interaction point (HIP), and the other one is the ideal haptic interaction point (IHIP). The HIP one only has a sphere collider and a rigid body component (Figure 6a), while the IHIP sphere has the visual renderer part of the haptic device object (Figure 6b). This approach was implemented according to the haptic rendering concepts described in [2].  
Once the haptic device object is implemented, an HIP C# script component needs to be created. This gets the haptic device's position, orientation, and buttons' states from the HM C# script. Additionally,  
Figure 6: (**a**) Haptic interaction point object and (**b**) ideal haptic interaction point object.  
it is the one in charge of detecting collisions in the environment using Unity's \"on collisions\" functions. It has to be noted that each haptic device needs its corresponding HIP and IHIP sphere objects in the scene. Finally, by using this approach, the sequence diagram presented in Figure 7 describes how all the proposed architecture's components collaborate during the communication with haptic devices, the interactivity of the haptic device and virtual objects, the calculation of interaction forces, and the transmission of visual and force feedback to the user. The Unity graphics represents the visual rendering or graphic processes, the Unity physics represents all the physics calculations. Unity is designed to run the physics thread while the graphical display is unfolding, and it handles the synchronization between the visual update rate and the frequency of the physics system, which can be set up according to the needs. Once Unity has started, the HaDIU script starts and the communication is made with the haptic devices, both the reading of the positions and the sending of the feedback force. Being a separate thread of execution, it allows haptic devices to maintain a refresh rate above 1 Khz. Moreover, this thread allows synchronization between Unity's physics thread and the haptic control thread.  
Figure 7: HaDIU’s sequence diagram that describes how all the proposed architecture’s components collaborate during the communication with haptic devices and calculation of feedback force according to their interactions with virtual objects.