# Digitized Visual Transformation of Grotto Art Using Deep Learning and Virtual Reality Technology

Authors: Hao Xie
Journal: Scientific Programming
 (2022)

---

## Abstract

The purpose is to convert the contents of the grotto murals into a digital vision based on deep learning (DL) and virtual reality (VR) technology to protect the precious grotto murals. The grotto murals are analyzed and modeled by introducing the related concepts of the digital library, using VR technology, and taking the cultural relics data acquisition technology as the basic framework. Then, the cultural relics information collection technology under DL is combined with VR technology, the color and materialization concepts of grotto murals are integrated, and a digital recognition model of grotto mural restoration and simulation is constructed. The color analysis scheme of grotto murals is established. Through the analysis of color and material, the comprehensive recognition rate of this color model can reach 70%, and the material recognition of color painting can also ensure one-to-one restoration. Besides, the model can be applied to the simulation and restoration of grotto murals and provide some help for the protection of grotto murals. This paper has practical application value for the protection and restoration of mural art.


---

## Body

## 1. Introduction

Grotto murals have always been one of China’s most precious artistic treasures. Chen et al. [1] believed that their artistic and historical reference values were immeasurable. Ma et al. [2] also pointed out that the grotto murals added rich color to China’s long history. Stratford et al. [3] pointed out that most of the grottoes were located in harsh areas such as the Gobi and the desert, with more sandstorms all year round. Hence, the erosion of the grottoes becomes increasingly serious, causing severe damage to the treasure. Armitage et al. [4] held that how to protect the grotto murals better had always been a difficult problem for archaeologists in recent years. Some studies have adopted network video technology to record the grotto murals to let more posterity enjoy the charm of grotto murals. However, Yin et al. [5] found that this method only took videos and pictures that could not be appreciated from multiple angles. He et al. [6] believed that the type of technology adopted to restore the grottoes is crucial.In studying wind sand erosion in grottoes, Liang et al. [7] pointed out that wind was the main dynamic condition affecting Gobi dust emission, and surface dust content was a significant factor determining dust emission. Besides, the contribution of wind to PM10 emission is greater than surface dust content. The higher the height is, the greater the weight of friction velocity is and the greater the damage to the grottoes is. Mogao Grottoes is one of the most representative grottoes in the Northern Wei Dynasty. Bao [8] pointed out that the “Crouching Tiger, Hidden Dragon” painting on the south wall of the grottoes had high artistic value. However, this painting’s original appearance has changed with the changes in dynasties and the development of the times. In particular, the color of the whole grotto has undergone subversive changes, which has led to multiple misunderstandings, indicating that the preservation of grotto murals is significant. Wu et al. [9] found that the tomb mural was a special mural buried underground. Due to the narrow exit of the tomb passage, the tomb mural is excavated by dividing the whole mural into blocks, which loses a lot of information between blocks. Therefore, digital image technology is adopted to restore the missing part to achieve a certain purpose of preservation. For the restoration of grottoes, Zhao et al. [10] combined semantic acquisition with knowledge association logic in information images and innovated the image resource analysis mode and the concept of digital restoration. The 4S-IAM model was proposed to connect historical graphics resources, local archaeological achievements, and mural image ontology resources effectively. However, Bi et al. [11] found that the tunnel wall’s heat and moisture transfer process changed in space and time. The complex heat and moisture transfer directions change periodically. Besides, the water transfer in the tunnel wall reaches a stable state faster than the heat transfer. The damp heat cycle plays a vital role in the development of mural degradation. Hence, the research on the grottoes preservation technology is one of the most crucial works in archaeology. The digital visual transformation of grotto mural art contributes to psychological influence and affects subjective well-being. However, because residents have a less visual appearance to perceive and experience the global contemporary architecture and urban design trends, this paper uses the mobile virtual reality (VR) platform to study the art of grotto murals. The content of model inspection is the degree of digital transformation of grotto mural art.The purpose is to realize the digital visual transformation of the mural art in grottoes and improve the probability of mural preservation and restoration. VR technology is used and combined with deep learning (DL). A virtual scene that is completely consistent with the real murals can be restored by analyzing the style and materials of grotto murals. The use of VR enables observers to observe and record grottoes anytime and anywhere through specific equipment. First, the concepts of grotto murals, VR, and DL are briefly introduced. The main research contribution is to identify the color and material of the grotto murals by combining DL and VR technology, judge the system’s performance, and provide a reference for the preservation and recording of the grottoes. The main research innovation is to use VR technology to simulate real grottoes, rather than using a single image restoration and recording to complete the preservation of grottoes. This paper has crucial reference value for the preservation and restoration of murals.


## 2. Literature Review

Many scholars have conducted relevant research on DL technology. Zhou et al. [12] explored and studied the application of DL in food and investigated dozens of studies using DL as a data analysis tool to solve problems and challenges in the food field. It includes food identification, calorie estimation, and quality detection of fruits, vegetables, meat, and aquatic products. The survey results show that DL is superior to other methods. Using DL as a tool for food quality and safety inspection can improve the accuracy of food safety inspection. Van et al. [13] studied DL technology in histopathology, summarized its current application situation in this field, and described the application challenges of artificial intelligence in histopathology. Research shows that DL can improve the speed and accuracy of medical diagnosis. Jacob et al. [14] studied the application of DL in image acquisition of the Internet of Things. Good test results show that the accuracy of DL can improve the accuracy of image recognition. Besides, Samadbeik et al. [15] studied the application of VR technology in medical group teaching. The results show that using virtual education technology for laparoscopic surgery training can improve the accuracy of medical practice. Bashabsheh et al. [16] studied the application of VR technology in architectural teaching and used VR technology to present some three-dimensional models of construction stages. The results show that VR software can better realize model construction compared to traditional teaching methods. To sum up, grotto mural art’s reproduction and digital transformation with the support of DL and VR technology can enhance the protective effect of precious cultural relics murals. The purpose is to establish a visual model of grotto mural art and compare the previous research methods with the research methods here. Wang et al. [17] studied the identity recognition algorithm based on DL, introduced the traditional method of person Re-ID, and analyzed its advantages and disadvantages. The results show that the combination of DL and traditional methods is vital. Laxton [18] evaluated and studied the surrealism of painting in the 19th century. The results show that modernist painting needs to be explored and recognized by DL technology. DL has crucial reference value for the digital transformation of grotto art. Li et al. [19] used DL to improve immune peptides in proteomics. The results show that compared with the existing methods, the DeepRescore rescoring method enhances the sensitivity and reliability of mass spectrometry binding peptides and new antigen identification. Farazmand [20] believed that grotto art was an ancient art exhibition integrating architecture, sculpture, and painting used for religious activities. It is a work with a high combination of artistry and practicality, so it has the role of artistic decoration and artistic protection of sculpture art. Duan et al. [21] found that colored sculptures existed in grotto temples in about 25 provinces, cities, and autonomous regions in China. Wang et al. [22] pointed out that China’s most famous grotto art included Qiuci Caves, Mountain Grottoes, Mogao Caves, Yungang Grottoes, The Longmen Grottoes, and Dazu Rock Carvings. To sum up, combined with the relevant research on grotto art, the use of DL and VR technology can provide technical support for restoring and protecting grotto art.


## 3. Methods

### 3.1. Deep Learning

DL is a machine learning method, which integrates the prediction results of multiple classifiers into a learning model to improve robustness and classification performance. VR is a computer simulation system that combines virtual and reality to create and experience a virtual world. The integration method of DL can improve the classification from any single classifier, but the combination classifier is not always meaningful. It is worth combining different classifiers only when the detection rate is high. The color classification of the grottoes is also obvious. Figure1 shows several main types.Figure 1
Main shapes of grotto murals.Dunhuang Mogao Grottoes is a place integrating modeling, color sculpture, and murals. Due to the constraints of the space environment and light conditions, the color structure and effect of murals in Mogao Grottoes are closely related to the grotto environment. According to the recorded stroke fragments, Singha and Manib [23] found that the colors used include black, yellow, bright red, vermilion, stone cyan, and stone green. Black accounts for the largest proportion and has high saturation. Figure 2 displays the main color proportions and classifications of the grotto murals.Figure 2
Main color classification of grotto murals.Nishanthi [24] found that color impacts people’s senses and psychology. The effect of Dazu Rock Carvings has both religious appeal and guiding effect. Qin and Song [25] pointed out that people used imperial cave statues in Dazu Rock Carvings to pray for blessings. The strong religious atmosphere created by the color of Dazu Rock Carvings makes believers feel immersive. In Buddhist art, red has the spiritual meaning of submission and appeal. Therefore, the colors of the grotto murals generally have two general meanings in Figure 3.Figure 3
The color implication of grotto murals.From the color art perspective, combined with religious culture, Chinese traditional culture, and field data, this paper summarizes and analyzes the color of Dazu Rock Carvings through the DL model, which lays a theoretical foundation for further displaying the color art of Dazu Rock Carvings.


### 3.2. Cultural Relics Information Acquisition Technology under DL

Han et al. [26] found that the work on the acquisition of two-dimensional cultural relics information had been carried out in worldwide cultural relics protection departments, transforming the pattern, color, and text information contained in cultural relics into digital information. After investigation, Xu et al. [27] found that the commonly used method was to take photos of cultural relics and obtain optical information using photography technology. Ordinary film can be used to take pictures and input them into the computer after shooting, or a digital camera can be directly used to obtain digital information. However, for multiple complex cultural relics, the research on how to shoot to ensure sufficient information is obtained and how to deal with the shooting information is not perfect.Chang and Guo [28] found that an image-based rendering (IBR) technology was to generate scene images from different viewpoints based on some pregenerated images or environment maps. Compared with the traditional rendering technology, it has three distinct differences in Table 1.Table 1
The uniqueness of IBR technology.
The uniqueness of IBR technologyImages under IBR technology are generally drawn with separate scenes. The sharpness of the image depends on the scene’s complexity.The images adopted by IBR technology can be drawn manually or photographed on the spot and used comprehensivelyIBR technology has a low computer load and can be applied and deployed on any multimedia deviceGuan et al. [29] believed that, according to the graphics theory, the microscopic properties of the scene surface were finally reflected in the bidirectional reflectance at each point of the scene surface. The traditional realistic graphics rendering technology uses texture images to describe the reflective attributes of each point on the scene surface to simulate the rich texture details. IBR technology is based on this theory. Its core technology is to draw the scene with any viewpoint position using a group of images (specific viewpoint position) of related objects or environments. It uses these texture images to express the microinformation of the scene surface in the environment. IBR technology can be divided into three types in Figure 4:Figure 4
IBR technology classification.Multiple models are stored in the form of multiresolution to meet the needs of real-time rendering or progressive transmission of network models. At this time, the multiresolution model representation needs to be compressed. Li et al. [30] used grids to generate regular or adaptive subdivision mechanisms, which could be used to generate models with multiple resolutions and provide an effective compression mechanism. However, they require that the input grid can meet the sub-subdivision connectivity.Many grids cannot meet this requirement, so the three steps of image recording in Figure5 are formulated.Figure 5
Three steps of IBR image processing.Bashir et al. [31] believed that the visualization technology of adding time dimension information was a very crucial technology for multiple cultural relic visualization systems. For example, to demonstrate the damage or evolution process of cultural relics, or the evolution process of a site over time, time information needs to be combined with visual information.


### 3.3. VR Technology and Its Analysis and Operation in Grotto Murals

Bashabsheh et al. [16] pointed out that virtual image technology is usually a communication technology. It is based on digital technology to combine vision and hearing. Virtual image technology includes various technologies, such as photography, projection, information transmission, multimedia display, sensory transmission, limb motion capture, and imitation. It provides users with virtual images through devices. Shin et al. [32] believed that virtual technology helps people understand abstract concepts, which are interactive images and scenes composed of text, images, videos, and other information. The main features are interactive, virtual, and immersive experiences. The rendering process of VR is analyzed. After collecting data elements in the image processing center, three-dimensional image modeling is conducted through sensors. Figure 6 shows the mapping process of VR technology.Figure 6
Mapping process of VR technology.Lungu et al. [33] found that the research on virtual image push algorithms in China is incomplete and not intelligent enough, the accurate push is difficult, and the processing speed is low. There are lots of college students with different personal preferences. Many resources are wasted in massive virtual videos and images. Using a single algorithm cannot meet the needs of variability. Hence, a new hybrid recommendation algorithm is a feasible scheme to solve this problem.The loss of resolution accuracy is caused in the splicing process, but the reason is in the shooting ring step. The main reasons for the loss are camera position error, lens aberration, projection distortion, and mural wall fluctuation. A complete operation step is calculated after considering these factors comprehensively. Liang et al. [34] revealed that, in image and video shooting calculation, Slen-opti is generally adopted to represent the shooting pixel accuracy of the lens, dist means the distance, and len represents the lens type. Moreover, the fineness of the digital image must be determined before shooting. The digital simulation of grotto murals must have high resolution. However, too high precision and resolution will increase work difficulty and waste a lot of resources. Therefore, the minimum standard of resolution is to be able to clearly identify the lines on the mural. The specific situation needs to be judged according to the actual situation. If the inclination angle of the mural wall is θ and the final mural resolution is M, the shooting angle of the mural resolution is(1)PPI=Mcosθ2.When the maximum fluctuation angle of the mural wall does not exceedθ, the lens is LEN model, the elevation angle of the lens is defined as φ, and the resolution of the camera is W × H, the distance L between the camera lens and the mural wall shall be(2)L=Wcosθ2000Mctgϕ2.At the last shooting, the horizontal or vertical moving distanceL of the lens is calculated as follows:(3)L=Slen−optL,Lencosθ1000M,where 1000M represents the pixel size of the mural photographed by the camera. Many factors affecting murals’ restoration cannot be accurately measured and calculated. This problem is effectively solved by obtaining lens distortion data through experiments and using the estimated value to determine the selection box to eliminate the influence of wall fluctuation. It also ensures the final results’ accuracy and has high feasibility and practicability.Lv et al. [35] believed that a general problem of uneven illuminationtook results in inconsistent brightness of the images. This situation can be improved by using the reflector to adjust the lighting direction of the flash lamp. However, it is still not ideal for preserving cultural relics.The maximum light intensity is defined asLmax. In a region, M points are adopted for calculation. If the nearest four points are L0, L1, L2, and L3, respectively, and point M is taken as the coordinate (x, y), the illumination intensity L of point M is(4)L=5−x5−y100L0+5+x5−y100L1+5−x5+y100L2+5+x5+y100L3.It is adjusted by using the light intensityL and the maximum light intensity Lmax. For the most edge point, the brightness value LA is processed, and the new brightness value LB is(5)LB=LA×Lmax−LLmax.This adjustment method is efficient in processing speed and has little effect on the variation range of brightness. Color sculpture data acquisition includes three-dimensional structure data and surface texture data. These two data acquisition methods are different. The two-dimensional scanning method is adopted to directly obtain three-dimensional structure data of color sculpture. However, for the surface texture processing of color sculptures, the required data can be obtained only after field shooting.


### 3.4. Colour Analysis Scheme of Grotto Murals

When reconstructing murals with VR technology, it is essential to analyze the reasons for the rich colors of Dazu Rock Carvings, find out their comparison methods, and reproduce the original style of Dazu Rock Carvings as much as possible. Paakki et al. [36] found that Dazu Rock Carvings are rich in color due to the use of many complementary colors. Some studies mentioned that, from the perspective of chromatics, the contrast of complementary colors is the physiological and psychological response of human visual balance. Artists often use this visual instinct to apply complementary colors to the color structure of the image simultaneously or continuously in the process of color painting to enrich the image and reduce the visual fatigue of the viewer. In different cave murals, the structure and production materials of murals are different, which increases the color complexity of mural scenes and brings certain difficulties to mural protection. Hence, a reasonable scheme needs to be designed for the color analysis of the grotto murals.The color of the Dazu Rock Carvings extracted from the scene is visible. Unlike the color displayed by computer monitors with operations, how to convert the color from reality to computer value is a difficult problem for this paper. There are two kinds of existing colors of Dazu Rock Carvings. First, the color is still visible; second, the color is missing in varying degrees. According to the above two situations, a comprehensive technical route is formulated through consulting massive literature and optical professionals to realize real and virtual color conversion. Figure7 displays the specific technical route.Figure 7
The digital color conversion process.Figure7 displays that the color analyzer further obtains color data by dividing light and color materials in mural materials, simulates and restores colors through VR technology, compares similar colors to form a color space, and fills the color image database. Shi et al. [37] proved that most of the pigments used in the color painting of grotto murals are mineral pigments, which have stable chemical properties and are not easy to change color. However, after a long history, the color change of the Dazu Rock Carvings is diversified. The same original color will have different degrees of color decay and falling off due to different distribution and influencing factors such as light, weathering, and humidity. Figure 8 presents the digital application process of cave murals established by VR technology and DL.Figure 8
Digital simulation process of grotto murals.In the digital preview of grotto mural images, the principle of camera interactive control is to simulate the first-person perspective of human beings. Thereby, virtual characters can be set in virtual scenes. Moreover, the interactive control function of the camera can be realized by controlling the role perspective. Therefore, this digital simulation process will be used to analyze and identify the colors and materials of grotto murals. Murals in a cave are analyzed. Figure9 presents the results.Figure 9
Mural text analysis and model establishment.Figure9 displays that, for the images on the traditional grotto murals, first, the text is output through the sensor, then the mural repair and text extraction are carried out through the processor, and the digital simulation model is further established to analyze the mural text.


### 3.5. VR Technology and Artistic Modelling Analysis of Grotto Murals

The deep application of VR technology has expanded people’s imagination and cognitive range. It can conduct highly simulated virtual reconstruction of the historical scenes of the grotto murals and build the scenes and elements that once existed only in the human brain or imagination one by one. Moreover, it allows people to travel as in the real world while maintaining all the perceptual states of the real world so that the audience can truly experience the scenes previously seen in sci-fi blockbusters. This high coincidence between reality and virtual scene enables people to enjoy and feel the cave mural culture at any time. No matter where they are, they can enter the virtual cave mural scene for appreciation. These precious historical and cultural heritages should be historical materials that people can read anytime and anywhere so that people can widely spread the historical and cultural spirit and focus on protecting the national treasure cultural relics.The difference between VR technology and modern smartphones or other smart devices lies in whether it can present a perfect virtual world for people and whether people can get a new experience of infinite resources and space in this virtual world that cannot be felt and touched in real life. Maybe one day in the future, people can work in the virtual world and complete more work more relaxed and efficient. Moreover, they can even use completely different ways and tools from the current stage to maximize the liberation and improvement of productivity. The equipment used here is a handheld three-dimensional laser scanner, and the spatial measurement accuracy is 0.02 mm + 0.04 mm/m. The point and surface resolutions are 0.025 mm and 0.100 mm, respectively. The data transmission interface is universal serial bus 3.0, and the scanning range is 310 × 350 mm.


## 3.1. Deep Learning

DL is a machine learning method, which integrates the prediction results of multiple classifiers into a learning model to improve robustness and classification performance. VR is a computer simulation system that combines virtual and reality to create and experience a virtual world. The integration method of DL can improve the classification from any single classifier, but the combination classifier is not always meaningful. It is worth combining different classifiers only when the detection rate is high. The color classification of the grottoes is also obvious. Figure1 shows several main types.Figure 1
Main shapes of grotto murals.Dunhuang Mogao Grottoes is a place integrating modeling, color sculpture, and murals. Due to the constraints of the space environment and light conditions, the color structure and effect of murals in Mogao Grottoes are closely related to the grotto environment. According to the recorded stroke fragments, Singha and Manib [23] found that the colors used include black, yellow, bright red, vermilion, stone cyan, and stone green. Black accounts for the largest proportion and has high saturation. Figure 2 displays the main color proportions and classifications of the grotto murals.Figure 2
Main color classification of grotto murals.Nishanthi [24] found that color impacts people’s senses and psychology. The effect of Dazu Rock Carvings has both religious appeal and guiding effect. Qin and Song [25] pointed out that people used imperial cave statues in Dazu Rock Carvings to pray for blessings. The strong religious atmosphere created by the color of Dazu Rock Carvings makes believers feel immersive. In Buddhist art, red has the spiritual meaning of submission and appeal. Therefore, the colors of the grotto murals generally have two general meanings in Figure 3.Figure 3
The color implication of grotto murals.From the color art perspective, combined with religious culture, Chinese traditional culture, and field data, this paper summarizes and analyzes the color of Dazu Rock Carvings through the DL model, which lays a theoretical foundation for further displaying the color art of Dazu Rock Carvings.


## 3.2. Cultural Relics Information Acquisition Technology under DL

Han et al. [26] found that the work on the acquisition of two-dimensional cultural relics information had been carried out in worldwide cultural relics protection departments, transforming the pattern, color, and text information contained in cultural relics into digital information. After investigation, Xu et al. [27] found that the commonly used method was to take photos of cultural relics and obtain optical information using photography technology. Ordinary film can be used to take pictures and input them into the computer after shooting, or a digital camera can be directly used to obtain digital information. However, for multiple complex cultural relics, the research on how to shoot to ensure sufficient information is obtained and how to deal with the shooting information is not perfect.Chang and Guo [28] found that an image-based rendering (IBR) technology was to generate scene images from different viewpoints based on some pregenerated images or environment maps. Compared with the traditional rendering technology, it has three distinct differences in Table 1.Table 1
The uniqueness of IBR technology.
The uniqueness of IBR technologyImages under IBR technology are generally drawn with separate scenes. The sharpness of the image depends on the scene’s complexity.The images adopted by IBR technology can be drawn manually or photographed on the spot and used comprehensivelyIBR technology has a low computer load and can be applied and deployed on any multimedia deviceGuan et al. [29] believed that, according to the graphics theory, the microscopic properties of the scene surface were finally reflected in the bidirectional reflectance at each point of the scene surface. The traditional realistic graphics rendering technology uses texture images to describe the reflective attributes of each point on the scene surface to simulate the rich texture details. IBR technology is based on this theory. Its core technology is to draw the scene with any viewpoint position using a group of images (specific viewpoint position) of related objects or environments. It uses these texture images to express the microinformation of the scene surface in the environment. IBR technology can be divided into three types in Figure 4:Figure 4
IBR technology classification.Multiple models are stored in the form of multiresolution to meet the needs of real-time rendering or progressive transmission of network models. At this time, the multiresolution model representation needs to be compressed. Li et al. [30] used grids to generate regular or adaptive subdivision mechanisms, which could be used to generate models with multiple resolutions and provide an effective compression mechanism. However, they require that the input grid can meet the sub-subdivision connectivity.Many grids cannot meet this requirement, so the three steps of image recording in Figure5 are formulated.Figure 5
Three steps of IBR image processing.Bashir et al. [31] believed that the visualization technology of adding time dimension information was a very crucial technology for multiple cultural relic visualization systems. For example, to demonstrate the damage or evolution process of cultural relics, or the evolution process of a site over time, time information needs to be combined with visual information.


## 3.3. VR Technology and Its Analysis and Operation in Grotto Murals

Bashabsheh et al. [16] pointed out that virtual image technology is usually a communication technology. It is based on digital technology to combine vision and hearing. Virtual image technology includes various technologies, such as photography, projection, information transmission, multimedia display, sensory transmission, limb motion capture, and imitation. It provides users with virtual images through devices. Shin et al. [32] believed that virtual technology helps people understand abstract concepts, which are interactive images and scenes composed of text, images, videos, and other information. The main features are interactive, virtual, and immersive experiences. The rendering process of VR is analyzed. After collecting data elements in the image processing center, three-dimensional image modeling is conducted through sensors. Figure 6 shows the mapping process of VR technology.Figure 6
Mapping process of VR technology.Lungu et al. [33] found that the research on virtual image push algorithms in China is incomplete and not intelligent enough, the accurate push is difficult, and the processing speed is low. There are lots of college students with different personal preferences. Many resources are wasted in massive virtual videos and images. Using a single algorithm cannot meet the needs of variability. Hence, a new hybrid recommendation algorithm is a feasible scheme to solve this problem.The loss of resolution accuracy is caused in the splicing process, but the reason is in the shooting ring step. The main reasons for the loss are camera position error, lens aberration, projection distortion, and mural wall fluctuation. A complete operation step is calculated after considering these factors comprehensively. Liang et al. [34] revealed that, in image and video shooting calculation, Slen-opti is generally adopted to represent the shooting pixel accuracy of the lens, dist means the distance, and len represents the lens type. Moreover, the fineness of the digital image must be determined before shooting. The digital simulation of grotto murals must have high resolution. However, too high precision and resolution will increase work difficulty and waste a lot of resources. Therefore, the minimum standard of resolution is to be able to clearly identify the lines on the mural. The specific situation needs to be judged according to the actual situation. If the inclination angle of the mural wall is θ and the final mural resolution is M, the shooting angle of the mural resolution is(1)PPI=Mcosθ2.When the maximum fluctuation angle of the mural wall does not exceedθ, the lens is LEN model, the elevation angle of the lens is defined as φ, and the resolution of the camera is W × H, the distance L between the camera lens and the mural wall shall be(2)L=Wcosθ2000Mctgϕ2.At the last shooting, the horizontal or vertical moving distanceL of the lens is calculated as follows:(3)L=Slen−optL,Lencosθ1000M,where 1000M represents the pixel size of the mural photographed by the camera. Many factors affecting murals’ restoration cannot be accurately measured and calculated. This problem is effectively solved by obtaining lens distortion data through experiments and using the estimated value to determine the selection box to eliminate the influence of wall fluctuation. It also ensures the final results’ accuracy and has high feasibility and practicability.Lv et al. [35] believed that a general problem of uneven illuminationtook results in inconsistent brightness of the images. This situation can be improved by using the reflector to adjust the lighting direction of the flash lamp. However, it is still not ideal for preserving cultural relics.The maximum light intensity is defined asLmax. In a region, M points are adopted for calculation. If the nearest four points are L0, L1, L2, and L3, respectively, and point M is taken as the coordinate (x, y), the illumination intensity L of point M is(4)L=5−x5−y100L0+5+x5−y100L1+5−x5+y100L2+5+x5+y100L3.It is adjusted by using the light intensityL and the maximum light intensity Lmax. For the most edge point, the brightness value LA is processed, and the new brightness value LB is(5)LB=LA×Lmax−LLmax.This adjustment method is efficient in processing speed and has little effect on the variation range of brightness. Color sculpture data acquisition includes three-dimensional structure data and surface texture data. These two data acquisition methods are different. The two-dimensional scanning method is adopted to directly obtain three-dimensional structure data of color sculpture. However, for the surface texture processing of color sculptures, the required data can be obtained only after field shooting.


## 3.4. Colour Analysis Scheme of Grotto Murals

When reconstructing murals with VR technology, it is essential to analyze the reasons for the rich colors of Dazu Rock Carvings, find out their comparison methods, and reproduce the original style of Dazu Rock Carvings as much as possible. Paakki et al. [36] found that Dazu Rock Carvings are rich in color due to the use of many complementary colors. Some studies mentioned that, from the perspective of chromatics, the contrast of complementary colors is the physiological and psychological response of human visual balance. Artists often use this visual instinct to apply complementary colors to the color structure of the image simultaneously or continuously in the process of color painting to enrich the image and reduce the visual fatigue of the viewer. In different cave murals, the structure and production materials of murals are different, which increases the color complexity of mural scenes and brings certain difficulties to mural protection. Hence, a reasonable scheme needs to be designed for the color analysis of the grotto murals.The color of the Dazu Rock Carvings extracted from the scene is visible. Unlike the color displayed by computer monitors with operations, how to convert the color from reality to computer value is a difficult problem for this paper. There are two kinds of existing colors of Dazu Rock Carvings. First, the color is still visible; second, the color is missing in varying degrees. According to the above two situations, a comprehensive technical route is formulated through consulting massive literature and optical professionals to realize real and virtual color conversion. Figure7 displays the specific technical route.Figure 7
The digital color conversion process.Figure7 displays that the color analyzer further obtains color data by dividing light and color materials in mural materials, simulates and restores colors through VR technology, compares similar colors to form a color space, and fills the color image database. Shi et al. [37] proved that most of the pigments used in the color painting of grotto murals are mineral pigments, which have stable chemical properties and are not easy to change color. However, after a long history, the color change of the Dazu Rock Carvings is diversified. The same original color will have different degrees of color decay and falling off due to different distribution and influencing factors such as light, weathering, and humidity. Figure 8 presents the digital application process of cave murals established by VR technology and DL.Figure 8
Digital simulation process of grotto murals.In the digital preview of grotto mural images, the principle of camera interactive control is to simulate the first-person perspective of human beings. Thereby, virtual characters can be set in virtual scenes. Moreover, the interactive control function of the camera can be realized by controlling the role perspective. Therefore, this digital simulation process will be used to analyze and identify the colors and materials of grotto murals. Murals in a cave are analyzed. Figure9 presents the results.Figure 9
Mural text analysis and model establishment.Figure9 displays that, for the images on the traditional grotto murals, first, the text is output through the sensor, then the mural repair and text extraction are carried out through the processor, and the digital simulation model is further established to analyze the mural text.


## 3.5. VR Technology and Artistic Modelling Analysis of Grotto Murals

The deep application of VR technology has expanded people’s imagination and cognitive range. It can conduct highly simulated virtual reconstruction of the historical scenes of the grotto murals and build the scenes and elements that once existed only in the human brain or imagination one by one. Moreover, it allows people to travel as in the real world while maintaining all the perceptual states of the real world so that the audience can truly experience the scenes previously seen in sci-fi blockbusters. This high coincidence between reality and virtual scene enables people to enjoy and feel the cave mural culture at any time. No matter where they are, they can enter the virtual cave mural scene for appreciation. These precious historical and cultural heritages should be historical materials that people can read anytime and anywhere so that people can widely spread the historical and cultural spirit and focus on protecting the national treasure cultural relics.The difference between VR technology and modern smartphones or other smart devices lies in whether it can present a perfect virtual world for people and whether people can get a new experience of infinite resources and space in this virtual world that cannot be felt and touched in real life. Maybe one day in the future, people can work in the virtual world and complete more work more relaxed and efficient. Moreover, they can even use completely different ways and tools from the current stage to maximize the liberation and improvement of productivity. The equipment used here is a handheld three-dimensional laser scanner, and the spatial measurement accuracy is 0.02 mm + 0.04 mm/m. The point and surface resolutions are 0.025 mm and 0.100 mm, respectively. The data transmission interface is universal serial bus 3.0, and the scanning range is 310 × 350 mm.


## 4. Results

### 4.1. Analysis of Colour Recognition Effect of Grotto Murals by the Digital Analysis Model

Many mineral pigments are adopted in the grotto murals. Moreover, with the years of sand erosion and man-made destruction, the color of many murals has significantly changed compared with the original color. Thereby, in the whole process of simulated restoration with VR technology, identifying the original color of grotto murals has become a basic work. In this section, the digital analysis model established above will be employed to identify some colors of grotto murals, to judge whether the model can accurately identify and verify colors and to prove the model’s color recognition performance. There are two painting methods: stone carving and relief. Red, yellow, green, and black colors are selected for recognition and analysis. Figure10 shows the results.Figure 10
Color recognition results: (a) red, (b) yellow, (c) green, and (d) black.
(a)(b)(c)(d)Figure10 suggests that, in the system analysis results for red, the average recognition rate of stone carvings is between 80% and 90%, while the color recognition rate of relief carvings is in a steady rising state, with the highest point reaching 80%. After running for more than 2250 s, the recognition rate begins to decline. In the recognition results of yellow, the recognition rates of stone carvings and relief carvings are rising at the beginning, and the best recognition rates of relief carvings and stone carvings are 85% and 90%, respectively. After 1600 seconds, the recognition rate of relief decreases, the overall trend is up, and the recognition results are satisfactory. In the green recognition results, the overall trend of the recognition rate is on the rise, and the highest recognition rates of relief and stone carvings are 80% and 90%, respectively. After 2500 s, the recognition rate of both begins to decline. In the first 500 s, the recognition rate data fluctuate in both cases. Finally, in black recognition, the recognition rate of stone carving is stable throughout the whole process, maintains at about 50%, and shows a downward trend after 2500 seconds. The recognition rate of relief is low at first, only 10%. With the system’s running, the recognition rate of relief rises and reaches 65% at most. It reveals that the system is currently in an unstable state. The recognition rate of color is sometimes high and sometimes low, but the recognition rate of red, yellow, and green can reach more than 80%. The recognition rate can be accepted and used for actual detection. However, the recognition rate of black is not high. The possible reason is that all colors will change into various darker colors after being eroded and damaged, and the recognition interference of black is more obvious. Hence, optimizing the recognition rate of dark colors will be the main work in the follow-up research.


### 4.2. The Type of Color Material Recognized according to the Light of Color

Finally, the color of the grotto murals is verified and recognized using the digital recognition system. The results show that, except for black, the recognition rates of red, yellow, and green can meet daily work needs. The low recognition rate of black may be due to the lack of rich colors, so black is not often used in murals. This section will use the proposed system to analyze color drawing materials. Specific materials need to be preserved in a specific way, so analyzing mural material categories is also crucial in preserving grotto murals. Tan et al. [38] analyzed the specific colors of model painting materials. Table 2 presents the results.Table 2
Drawing material analysis results of specific colors.
ItemsRecognition resultsRecognized colorOcherStone greenStone cyanGypsumGraphiteGoldenSilver goldMaterial typeHematiteMalachiteLapis lazuliCalcium sulfateBlack lead powderPhlogopiteVermiculiteThe results in Table2 prove that the digital recognition model can clearly identify each unique color. Currently, the above seven colors are identified and analyzed. The identification material of the system for ochre is hematite, malachite for stone green, lapis lazuli for stone cyan, black lead powder for graphite color, phlogopite for golden, and vermiculite for silver gold. It reveals that the recognition accuracy of the color model is relatively accurate. It can accurately identify substances with a single color, which ensures the overall analysis of one-to-one recognition results, and the recognition accuracy of the model is high. However, given the low recognition rate of black in the previous section, the model needs to be further optimized to ensure that the recognition accuracy of other colors is also significantly improved.


## 4.1. Analysis of Colour Recognition Effect of Grotto Murals by the Digital Analysis Model

Many mineral pigments are adopted in the grotto murals. Moreover, with the years of sand erosion and man-made destruction, the color of many murals has significantly changed compared with the original color. Thereby, in the whole process of simulated restoration with VR technology, identifying the original color of grotto murals has become a basic work. In this section, the digital analysis model established above will be employed to identify some colors of grotto murals, to judge whether the model can accurately identify and verify colors and to prove the model’s color recognition performance. There are two painting methods: stone carving and relief. Red, yellow, green, and black colors are selected for recognition and analysis. Figure10 shows the results.Figure 10
Color recognition results: (a) red, (b) yellow, (c) green, and (d) black.
(a)(b)(c)(d)Figure10 suggests that, in the system analysis results for red, the average recognition rate of stone carvings is between 80% and 90%, while the color recognition rate of relief carvings is in a steady rising state, with the highest point reaching 80%. After running for more than 2250 s, the recognition rate begins to decline. In the recognition results of yellow, the recognition rates of stone carvings and relief carvings are rising at the beginning, and the best recognition rates of relief carvings and stone carvings are 85% and 90%, respectively. After 1600 seconds, the recognition rate of relief decreases, the overall trend is up, and the recognition results are satisfactory. In the green recognition results, the overall trend of the recognition rate is on the rise, and the highest recognition rates of relief and stone carvings are 80% and 90%, respectively. After 2500 s, the recognition rate of both begins to decline. In the first 500 s, the recognition rate data fluctuate in both cases. Finally, in black recognition, the recognition rate of stone carving is stable throughout the whole process, maintains at about 50%, and shows a downward trend after 2500 seconds. The recognition rate of relief is low at first, only 10%. With the system’s running, the recognition rate of relief rises and reaches 65% at most. It reveals that the system is currently in an unstable state. The recognition rate of color is sometimes high and sometimes low, but the recognition rate of red, yellow, and green can reach more than 80%. The recognition rate can be accepted and used for actual detection. However, the recognition rate of black is not high. The possible reason is that all colors will change into various darker colors after being eroded and damaged, and the recognition interference of black is more obvious. Hence, optimizing the recognition rate of dark colors will be the main work in the follow-up research.


## 4.2. The Type of Color Material Recognized according to the Light of Color

Finally, the color of the grotto murals is verified and recognized using the digital recognition system. The results show that, except for black, the recognition rates of red, yellow, and green can meet daily work needs. The low recognition rate of black may be due to the lack of rich colors, so black is not often used in murals. This section will use the proposed system to analyze color drawing materials. Specific materials need to be preserved in a specific way, so analyzing mural material categories is also crucial in preserving grotto murals. Tan et al. [38] analyzed the specific colors of model painting materials. Table 2 presents the results.Table 2
Drawing material analysis results of specific colors.
ItemsRecognition resultsRecognized colorOcherStone greenStone cyanGypsumGraphiteGoldenSilver goldMaterial typeHematiteMalachiteLapis lazuliCalcium sulfateBlack lead powderPhlogopiteVermiculiteThe results in Table2 prove that the digital recognition model can clearly identify each unique color. Currently, the above seven colors are identified and analyzed. The identification material of the system for ochre is hematite, malachite for stone green, lapis lazuli for stone cyan, black lead powder for graphite color, phlogopite for golden, and vermiculite for silver gold. It reveals that the recognition accuracy of the color model is relatively accurate. It can accurately identify substances with a single color, which ensures the overall analysis of one-to-one recognition results, and the recognition accuracy of the model is high. However, given the low recognition rate of black in the previous section, the model needs to be further optimized to ensure that the recognition accuracy of other colors is also significantly improved.


## 5. Conclusion

Grotto murals are an art treasure of great value in China. They record bits and pieces of China’s long history. Their historical value rises naturally since China is a large country with a long history. However, due to many years of wind and sand erosion of Gobi and desert and man-made unintentional or even intentional damage, the murals’ patterns are incomplete and colors have faded. Thereby, a measure to repair the damage to grotto murals or preserve their complete appearance is urgently needed. However, the repair process is extremely complex and difficult, which will consume a lot of resources, and it is not optimistic after repair, so a method is needed to preserve it. According to the current advantages of DL technology and VR technology, this paper puts forward a way to preserve the murals by simulating and restoring the grotto murals based on VR technology. First, the grottoes’ related concepts, materials, and colors are introduced. Then, the cultural relics information acquisition technology in the field of DL is put forward and combined with VR technology to build a digital recognition model for grotto murals. The results of color recognition of grottoes show that the model can accurately identify the colors of red, yellow, and green except black. Besides, it can identify the drawing material of each specific color one-to-one. It can be concluded that the digital recognition system can accurately record and simulate the restoration of grotto murals, which is of great significance for the preservation simulation of grotto murals using VR technology. However, the model’s discrimination rate for black is currently low. Some reasons are summarized, and this problem will be mainly solved in the follow-up work.



---

*Source: 5106036-2022-08-29.xml*