# Comparing Augmented Reality-Assisted Assembly Functions--A Case Study on Dougong Structure
## 4 Major Assisted Functions
### Augmented Reality (AR)-Based System  
Figure 7 shows the framework design of the AR-based system. Compared to the 3D viewer version, this one provides intelligent functions that prevent the user from taking or recognizing incorrect parts. The user learns the assembly task from an AR animation that precisely superimposes virtual models with real components in 3D space. Occlusion processing is applied to the animation using the Z-buffer method [19] based on the depth data captured using a commercial RGB-D camera, Kinect 2. The Kinect device generates the ambient intelligence required by 3D object recognition and tracking, i.e., the ability to estimate which objects exist in the environment.  
In computer graphics, occlusion culling is the process used to determine which models and parts of models are not visible from a certain viewpoint. In AR scenes, not only among virtual models, occlusions also occur between virtual models and real objects in a scene. In this work, the result of occlusion culling between real parts and the virtual model displayed for assembly is highly relevant to influencing the user's spatial reasoning. The Kinect v2 device was installed in front of the worktable to capture the depth information of real parts in 3D space. Based on the depth data, the Z-buffering method decides which elements of a rendered scene are visible, and which are hidden.  
Vuforia [20] is an object recognition library designed for AR applications and commonly used in the industry. This library provides an automatic object recognition functionality supported by learning models. Compatible with Unity, it helps streamline the development complexity and extend  
Figure 5: Framework design of 3D viewer-based system.  
Figure 6: Screenshots of assembly demonstration function.  
its real-world applications. Vutoria builds the recognition function of an object from training data that provide salient feature information for constructing the underlying learning model.  
The feature information may come from various object attributes such as geometry, color, and texture. Unfortunately, the original Dougong models are made of wood and lack discernible contrast in their appearance (see Figure 8a). Thus, additional information needs to be added to enhance the identification ability of the training model. When observing the actual Dougong architecture, it was discovered that some of the enamels were decorated with various colored patterns. Similar decorative patterns were designed and attached to the Dougong models in cooperation with an experienced designer. The result can be seen in Figure 8b. These patterns provide the feature information required to build the object recognition ability. They implicitly enable the marker tracking of the Dougong components, the performance of which is more stable than markerless tracking.  
The AR system starts by showing the first component model on the screen. It preserves most design features implemented in the 3D viewer system. The user can freely change the view angle of the models during the assembly process. The color labeling helps distinguish parts between different assembly steps. The click button controlling the assembly status remains the same. One newly added function is the automatic part confirmation. The user places the part (which they considered the one to be assembled) in front of the smartphone camera. The system will enquire from the learning model and send out a message to indicate whether that part is the correct one. This function prevents the user from taking a wrong part. When the system recognizes the correct part, it will confirm with a \"correct\" message and start to display an AR animation that demonstrates the assembly process, as shown in  
Figure 8: (**a**) Original Dougong models and (**b**) models decorated with colored patterns.  
Figure 7: Framework design of AR-based system.  
Figure 9. Once the current step is completed, the user will need to click the next button to take the next step. A similar operation repeats until the entire assembly process is completed.