# A Mixed Reality Interface for a Digital Twin Based Crane
## 2 Materials and Methods
### Mixed Reality Application for the Overhead Crane
#### 2.3.4 Communication Module  
The communication module functioned as a gateway between the crane and all the other modules of the application. We leveraged the RestSharp API in the Unity project to send HTTP requests (either a query request to read data, or a mutation request to write data) to the crane GraphQL wrapper. This way, the application could access the crane data, which would either be read and displayed in the dashboard of the visualization module or be written and modified by the fixed/movable target control of the interaction module.  
Figure 6: Spatial mapping mesh, holograms, and _image target_ in the Unity scene: the pose of the mesh, the holograms of fixed/movable targets, and the bounding box were determined by the pose of the _image target_, with the mesh functioning as a spatial reference for the scene design.  
Functional Components of Communication Module  
Figure 8 illustrates the three functional components in the communication module.  
Figure 8: Functional components of the communication module and data flow: The interaction module set the execution of the watchdog function, which then sent mutation requests to the GraphQL server for setting the access code and updating the watchdog value constantly. On the other hand, the interaction module also provided the selected target data to the control function, which then computed the control parameter values and sent them as mutation requests. Meanwhile, the visualization module received constant updates of the crane status from the monitor function through its query requests from the server, the selected target data from the interaction module as well as the execution status of each function within the communication module.  
Figure 7: Spatial Relationship Graph (SRG) of the registration setup: we got the spatial transformation between holograms and the HoloLens device through their relations with the _image target_.  
* **Watchdog function**, which enabled the interaction module by setting the initial access code and updating the watchdog value constantly through mutation requests.
* **Monitor function**, which enabled constant access to the selected crane real-time data, including all the crane status data displayed on the dashboard through query requests.
* **Control function**, which enabled the interaction module by constantly modifying the movement speed and direction of each crane subsystem, based on the difference between the current location and the target location, through mutation requests.  
Figure 8 also illustrates the data flow among the communication module, the other modules of the application, and the crane GraphQL server. Within the application, the interaction module gave the signal to call the watchdog function, and determined the target data flowing to the control function, while the visualization module received the crane status data flowing from the monitor function, the selected target data from the interaction module as well as the overall communication module status, then displayed them on the dashboard. Between the application and crane GraphQL server, both the watchdog function and the control function sent mutation requests, while the monitor function sent query requests to the server, which together enabled the application to exchange data with the crane in a bi-directional manner.