# Enhanced and Combined Representations in Extended Reality through Creative Industries
## 3 Materials and Methods
### Implementation
#### 3.2.3 Multimodal Interaction in Virtual Reality  
The interactions developed for the user's experience with the information produced by the CT scans were based on the user interface (UI) graphical element of the panel offered by the Unity game engine and on the options of the controllers of the Oculus Rift S virtual  
Figure 2: The 3D representations of the fragments A–G of the Antikythera Mechanism and five relevant indicative instances of their internal structure. reality device (RQ3). Primarily, users could interact with the 3D representations of the artifacts physically, as they could pick them up and hold each fragment with their virtual hands through the touch controllers (RQ2). Regarding the interactions with the inner structure of the fragments (RQ2), two functionalities were developed: (a) the discrete linear exploration and (b) the fast linear exploration. Both functionalities were active when the user was standing in the predefined area of a specific fragment, which was marked through a triggered collider (Figure 3a,b). The pipeline (RQ3) of multimodal interactions in the virtual museum of the Antikythera Mechanism is depicted in the flowchart (Figure 4). Figure 4: The flowchart of the user experience for the combined display of 3D representations and X-ray toolkit in virtual space. Figure 3: Multimodal interactions with the combined representations of the fragments in the virtual museum for the Antikythera Mechanism. In (**a**), the user holds objects with her left hand and examines the 3D representation of fragment A, while the right-hand controller interacts with the inner structure of the artifact. Similarly, in (**b**), the user examines, in a deeper layer, the non-visible parts of the corroded fragment. Discrete Linear Exploration  
When selecting discrete linear exploration, the user can use the A and B buttons to study the interior of the fragments. Snapshots are stored and sorted in a table. Each time the user presses the A key, the function OnClickNext is executed, so the display surface shows the next snapshot. Conversely, whenever the user selects the B button, the function OnClickPrevious is executed, so the display surface shows the previous snapshot. For each fragment, the range in which the user can interact is from 0 to 100 (see Algorithm 1). Through this interaction, the study of the internal structure is made instance by instance (RQ1, RQ3). ```
public void OnClickNext(string nameImage) { if (imgNumber < 100) {  imgNumber++;  nameImage = imgNumber.ToString ();  imageXrayA.sprite = Resources.Load<Sprite> (\"FragmentA/x_fragment_A_\" +  nameImage) as Sprite; } public void OnClickPrevious(string nameImage) { if (imgNumber > 0) {  imgNumber-;  nameImage = imgNumber.ToString();  imageXrayA.sprite = Resources.Load<Sprite> (\"FragmentA/x_fragment_A_\" +  nameImage) as Sprite; } }
```  
**Algorithm 1**Script in C# for discrete linear exploration on the fragment A of the Antikythera Mechanism. Fast Linear Exploration  
When selecting fast linear exploration, the user can use the Fire 1 and Fire 2 triggers to study the interior of the fragments. Each time the user presses the SecondaryIndex trigger, a sequence of consecutive snapshots is displayed on the display surface in ascending order as a natural stream. Conversely, whenever the user selects the SecondaryHand trigger, a sequence of consecutive snapshots in descending order is displayed on the display surface as a natural stream. As long as one of the triggers is pressed, the user is given the illusion that the fragment comes to life (RQ1, RQ3). These interactions are controlled by the Update function in Unity (see Algorithm 2).