# Extended Reality Application Framework for a Digital-Twin-Based Smart Crane
## 2 Materials and Methods
### The XR Environment Development  
Figure 5 depicts the structure of the service layer. The XR environment development contains three sub-modules, the model module, the function module, and the communication module. The model module determines the generally used models that keep consistency with the DT-based crane. The function module realizes the crane features and AR or VR functions related to the corresponding device. The communication module is responsible for interaction with the network layer on the Unity side. Figure 5: Overview of the service layer. It involved two subsections, namely hardware setup and software development. **Model Module:** To guarantee the consistency and interoperability of the model in the XR environment, our research provided a standard modeling method, with the crane as a study case. Figure 6 displays the \"Illmatar\" crane model in Unity. The method consists of the following three steps. 1. Identify the key moving system, and disassemble the corresponding parts in the CAD or original software: The crane was divided into four main moving parts, namely, the bridge, trolley, hoist, and hook block (see Figure 2). In Siemens NX, each moving part was disassembled as an isolated file. 2. Convert to the Unity compatible format: The STP file is the most general format that is compatible with most industrial CAD software, such as AutoCAD, Siemens NX, and SolidWorks. In the meantime, STP is a format for 3D graphics files and supports product model data exchange, which is commonly used as the transition format between the different software. There are two kinds of methods for model import in Unity: one is exporting models as FBX or OBJ format by the specific plugin; the other is directly exporting the model as the corresponding file, such as max or blend, which can be converted automatically by Unity. The former method is suitable for the case in which the model originates from industrial 3D software. In our research, the original model file was exported as STP format, through the specific plugin, and converted to the OBJ format. 3. Set up the coordinate system and establish the parent-child relation of the import parts: There is no coordinate relation for the imported model. The coordinate system should be reestablished in the Unity, as well as the parent-child organization of the components according to the movement relation. **Function Module:** The crane features are implemented by the script-based components, including Inching, Snag Prevention, and Micro Speed. Inching provides a way to approach a load destination with great accuracy. It allows the operator to make small inching movements. When the inching feature is activated, the movement will stop after it has traveled the predefined inching distance. The Snag Prevention function will automatically stop the moving crane when the crane hook catches on another object. Micro Speed allows the operator to use the whole joystick movement range while not going over the required maximum speed. The crane features were developed for the VR training application. XR functions, such as VR locomotion, VR object grab, AR hand tracking, and UI interaction, were realized by the component-based toolkit, XRTK, and MRTK. **Communication Module:** First of all, it is necessary to define the data format of the model. Table 3 illustrates the definition of data in the XR environment, including the design parameter of the crane, the status data of the running crane, and the data concerning the controller. The design data is from Table 1, describing the constant design parameters. The  
Figure 6: “Illmatar” crane model in Unity. data is stored as a component in Unity as the constant private value. Later, all the data will be accessed from the DT (Digital Twin) Document. The DT document describes the metadata and features of a single digital twin, which is under development. The document is designed to be used along with a Data Link that connects the features of the digital twin behind a single access point [64].