Chu Culture Allusion Exhibition System through the View of Augmented Reality in Heterogeneous Environments

Inheriting historical culture innovatively is the irresistible trend of information era, and AR(Augmented Reality) is an advanced technique for this purpose. This paper introduces an AR system for historical allusion learning in heterogeneous environments, which consists of a multi-media database oriented to Chu culture, and an AR interaction system. By using AR interaction system, users could see the virtual sources and the real environment simultaneously. This lively interacting method would encourage the learning and spreading of historical allusions. Introduction Culture inheritance has been a global issue for its essentiality to human spiritual civilization progress. While it has been widely viewed that cultural inheritance and high-tech are opposite, AR (Augmented Reality), as an advanced computer technique, is still favored by traditional culture protectors since its significant effects for culture transmission. In recent years, experts and scholars have done tremendous researches on culture inheriting with the help of high-tech from various perspectives. AR is a technique which superposes computer-generated perceptual information upon real-world environment and enables users see them together [1] . According to Jiang Wenjun(2015), there is an increasing reliance on high-tech for the protection and development of traditional culture [2] . Furthermore, three novel solutions based on outdoor augmented reality are presented for the digital reconstruction of the famous royal garden of Yuanmingyuan, which proved the extensive application prospects of AR in culture inheritance [3] . In this paper, we chose several historical allusions from Chu culture, which is a distinctive part of Chinese civilization, collected relevant materials in different types to built a digital media library, and applied it on an AR Chu culture allusion exhibition system. Critical Technologies Based on different hardware environments, we designed three corresponding solutions respectively, and the critical technologies are as follow: Augmented Reality Augmented Reality(AR) is a variation of Virtual Reality(VR). It combines subjects including computer graphics, optoelectronic imaging, multisensor, image processing and computer vision. It allows users to see the real world, with virtual objects superimposed upon or composited with the real world. An AR system must have the following three characteristics: combining real and virtual, interacting in real time and registered in 3D [4] . Image Recognition and Registration The image recognition principle of this system is based on scheming out the nature features of the target image and preserving them into the database. Then the system uses real-time detection to judge if the nature features of current detected image and the ones of the target image are matching(Fig. 1). Specific to the development of this system, the first step is uploading the target image onto Vuforia cloud platform, then the platform will transform it into Tracking format and we import it into the local database. While the recognition process, the image which camera recognizes will be transformed into Tracking format as well and match with the target image. If the image is recognized successfully, the virtual components appointed in controlling scripts will appear in predetermined area(Fig. 2). Figure 1. Target image and its nature features. Figure 2. Controlling script and Image recognition. 3D Modeling and Animation 3D Models in this system are built in Maya, polygonal modeling is the main method being used. The first step is choosing the fundamental polygon which is closer to the shape of the model. Then the Extrude and Split Polygon Tool helps to do the reshaping. After the shape is settled, adding material and texture mapping on the model, using light and shadow effects to render detailed model. Animations are built by Mecanim Animation System in Unity. During the animation-making process, different AR scenes and characters are designed based on different allusions. Each character owns an Animator Controller. After creating animation nodes which represent different states of the character, setting up transitions among them. If it’s necessary, editing controlling scripts to control the animation together with the Animator Controller(Fig. 2). Developing Environment The system uses an MVC(Model-View-Controller) architecture design. The management of Model Layer is with the use of MySQL. Controller Layer developing depends on Unity and Vuforia SDK. In this layer, each allusion has a corresponding Scene, in which there are GameObjects, pictures, videos and other forms of components managed by scripts programmed in C#; Vuforia SDK provides encapsulated classes for image recognition and registration. View Layer mainly contains GUI(Graphical User Interface), which was developed by Photoshop, FairyGUI and Unity. Integrated Design The development goal is to design and implement an AR Chu culture allusion exhibition system with technology including automatic image recognition, AR immersive videos, AR animation interaction and so on in terms of shorten the distance between user and traditional culture. System Structure The whole system(Fig. 3) is consist of two subsystems: the database management system and the allusion exhibition system. Figure 3. System structure. Application Condition As it stated in 1.1, the system could operate on AR glasses, mobile device(Android/iOS phones, tablets, etc.) and PC. Users should download and install the appropriate version of system apk in their equipment(client), then sign up and log in to transmit data with the server. The AR glasses being used for developing and testing is XLoong Techlens T2 AR glasses, which are based on Android operating system and consist of hardware including two OLED virtual display screen, a camera, various sensors and a remote control. In other word, the glasses are equivalent to a smartphone with different interaction method. After download the apk, users could interact with the system by gestures and remote control. During the using process, once the target image is caught, the virtual component will be shown on the OLED screen, and the user will see the mixture of virtual and real environment(Fig. 4). Figure 4. Operating on AR glasses. Figure 5. Operating on mobile device. On mobile device or PC, after apk installing, users use gestures and touchscreen/mouse device to interact with the system. While being used, device camera will be invoked to recognize image. If the image matches the target image, the coordinate virtual components will be shown on device screen(Fig. 5). User Experience Design User Experience(UE) refers to a person’s emotions and attitudes about using particular product, system or service. It includes the practical, experiential, affective, meaningful and valuable aspects of human–computer interaction and product ownership [6] . When designing the user interface(UI), we choose black as the background color to construct the immersion. The detailed button designs are listed in Table.1. Table 1. Table headings. Button Type Icon Manipulation Effect Advantage Normal Button e.g. buttons in Fig. 9 Click Execute what the button means Simple and clear Back Button e.g. button in the top right corner of Fig. 10 Click Back to the upper layer Make relationships between layers clear Loop List Button e.g. button in the middle of Fig. 10 Drag and click the picture in the middle Enter the chosen allusion Clear and interesting Virtual Button e.g. buttons in Fig. 12 Conceal the button in front of the camera Execute what the button means Strengthen the sense of immersion Yellow seal-shaped buttons bring users a sense of history. Meanwhile, the huge contrast between yellow and black decreases the disturbance of using and makes the system more user-friendly. Implementation Method The implementation of the system follows the following process(Fig. 6). The completed system includes two systems: the database management system and the allusion exhibition system. Figure 6. Implementation process. In database management system(Fig. 7), database administrators could add, edit or delete allusions and their related data. Figure 7. Database. Figure 8. Sign up and log in module. Figure 9. Catalog module management system. In allusion exhibition system, anyone could register account and log in(Fig. 8) as a normal user. Once logged in, users could choose allusion which they interested in to learn all forms of relevant information and interact with its AR scene. Catalog module(Fig. 9) collects a series of well-known Chu culture allusions. Users could drag the loop list leftwards or rightwards to look over all allusions and click the picture to enter the Introduction module. Figure 10. Introduction module. Figure 11. Normal display module. Introduction module(Fig. 10) contains image and literature materials as well as Normal display and AR Display buttons, once being clicked, they will skip to Normal display module(Fig. 11) and AR display module(Fig. 2) respectively.