# A Mixed Reality Interface for a Digital Twin Based Crane
## 2 Materials and Methods
### Mixed Reality Application for the Overhead Crane
#### 2.3.2 Visualization Module  
The visualization module enabled users to directly monitor the crane real-time status through a dashboard, view the instructions on how to use the application, as well as adjust the interface (MR scene) in the way that they preferred by switching on/off the toggle of certain hologram.  
Workflow of Dashboard  
Figure 5 illustrates the dashboard with the crane real-time status data, which was grouped into three sections.  
* **\"Crane Status\"**, which included load value, sway control feature (on/off), positions and movements of the three subsystems (for trolley and bridge: stop/forward/backward, for hoist: stop/up/down). The data were fetched from the GraphQL server.
* **\"Crane Target\"**, which displayed the name of the selected target (fixed target 1/2/3/4/5 or movable target) and the target position of three dimensions. The data were fetched by reading the corresponding parameters in the scripts attached to the _Game Objects_ of fixed and movable targets that were selected.
* **\"Communication Status\"**, which showed if the behaviors of the watchdog, monitor and control running in three different threads started, as well as displayed the updated value and status of the watchdog. The data was fetched from the communication script.  
Figure 5: Screen capture of the dashboard from the Unity project in the play mode: the information displayed in the dashboard consisted of three sections, the “Crane Status”, “Crane Target” and “Communication Status”.  
In the Unity project, the dashboard as a _Game Object_ was placed under the \"HoloLens Camera\" hierarchy, meaning that the dashboard would always be displayed right in front of users' view at a fixed distance. This way, users could check the information easily wherever they were moving towards or looking at.  
Workflow of Instructions  
In order to help new users to start, the application was empowered with interactive instructions. When users started the application, the welcome page came first, followed by detailed information on how to register the space. After users confirmed the completion of the registration step, the main hologram components appeared in the scene, including the fixed and movable targets, dashboard, and scene-switch toggles. The next page then introduced what the those new holograms were and where they were located. From this page, users could select the \"quick guide\", which consisted of three pages showing what users could do with the application (i.e., to monitor and control the crane, adjust the scene, as well revise the guide). Users could also choose to skip the guide and directly start crane operation.  
Workflow of Interface Adjustment  
During the instruction phase, several holograms appeared and stayed in the scene, which provided users with components to interact with and information to view. However, certain information and components could be redundant for the crane operation at times, and their visibility would even potentially affect user experience.  
For this reason, we introduced the interface adjustment to this prototype. With scene-switch toggles, users could switch off certain hologram components (i.e., dashboard, movable and fixed targets) in the default scene to clear up the view, and enable them again any time later. Additionally, the three pages of \"quick guide\" could be displayed again whenever users needed a recall on how to use the application.  
Development of Visualization Module  
The components in the visualization module were created with prefab UI building blocks from MRTK to make them visually appealing and operationally interactive. Those building blocks, such as the collision-based buttons and toggles, were configured to have audio-visual feedback for various types of inputs.