# Dynamic Mixed Reality Assembly Guidance Using Optical Recognition Methods
## 4 Application
### Development  
The logic of the application is driven by two absolute CSs, one for the assembly setup, and the other for the component box. Both setups have complex non-symmetric features, they are not exceedingly reflective of light, and have minimal to no movement, which lends them for easy recognition with MR devices. Creating the CSs is done only once at the beginning of the MR experience, because the position, rotation, and state of all other components could be inferred from the position of the initial CSs. After calibration, the prepared workstation appears as in Figure 5.  
Then, the iterative steps can be performed for each component, starting from the first component selection. When the selected part has been validated as correct, the application jumps to the next step: placement. When the part is positioned, assembly validation starts. When it is recognized as correct, the app goes again to the component selection step (for the next component). If the validation is not completed, the app goes into the correction sub-step, until the component is correctly set on the assembly. For each component selection step, the user selects one part from the component box to assemble next, and the app validates the selection of the correct part from the assembly sequence, informing the operator of any possible error (see Figure 6). Figure 5: Prepared workstation after calibration of the absolute CS. Figure 6: Component selection step; selection of the roof is incorrect (red color), the app suggests in green the correct component to be selected (banner). Then, the component graphic guides the user to assemble the component correctly onto the product (see Figure 7a,b): an animation shows the correct part moving from the starting position in the component box to the final assembly position, and with the right orientation. At every step, a validation guidance graphic (VGG) is displayed to verify the correct positioning of the components. The VGG will be thoroughly discussed in Section 4.4. After correct validation through the VGG, the app proceeds with the next step in the list, performing the same operations just described for each part. If the validation step is not completed, the correction sub-step starts, offering a new animation with audio to correct the position or orientation of the component. The application features two types of graphics for visual guidance: static graphics guide the user to select the correct component from the component box at each step and warn them of wrong component selection; moving graphics guide users to the correct positioning of components onto the assembly. The static component graphics are green when the correct component is selected, and red when it is incorrect. Such graphics appear both in relation to the component box and the assembly setup. The \"Audio Manager\" controls the audio logic according to the ongoing step of the application. Thanks to a text-to-speech program, the app provides voiced instructions. Two types of audio clips play automatically in a sequence at the start of each step: clips that play only once, and loop audio clips that keep repeating until the step is completed. Audio clips that contain crucial information for completing the assembly tasks are played in a loop. The application also features a distinct sound that is played when actions are performed correctly, and a distinct sound for when the application recognizes incorrect actions. The audio serves to enhance the user's experience and immersion with the application, as well as provide additional information difficult to give through the UI (e.g., screwdriver selection, placing label text facing up). The application also includes UI-panels to display written information on the assembly procedure for each step. The \"UI-Panel Manager\" controls the panels, ensuring that only relevant information is visible. The UI-panels have buttons to allow the transition to the following step. The manual transition option is included in case assembly validation logic fails.