# Research on the Identification and Application of Immovable Cultural Relics in the Historic City of Macau Based on Condition of Intelligent Remote Sensing Technology

Authors: Qiang Zhao
Journal: Wireless Communications and Mobile Computing
 (2021)

---

## Abstract

The archeological sites are a heritage that we have gained from our ancestors. These sites are crucial for understanding the past and the way of life of people during those times. The monuments and the immovable relics of ancient times are a getaway to the past. The critical cultural relics however actually over the years have faced the brunt of nature. The environmental conditions have deteriorated the condition of many important immovable relics over the years since these could not be just shifted away. People also move around the ancient cultural relics that may also deform these relics. The machine learning algorithms were used to identify the location of the relics. The data from the satellite images were used and implemented machine learning algorithm to maintain and monitor the relics. This research study dwells into the importance of the area from a research point of view and utilizes machine learning techniques called CaffeNet and deep convolutional neural network. The result showed that 96% accuracy of predicting the image, which can be used for tracking human activity, protects heritage sites in a unique way.


---

## Body

## 1. Introduction

The following study presents the challenges in modern times the heritages face due to the encroached human activity and the environmental conditions. The use of machine learning has been proposed as the solution for the identification and recognition of the sites of cultural heritage and relics. The city of Macau has very well preserved heritage sites that contain a rich plethora of information about the past. China has been bestowed with numerous sites of heritage, which contains relics that are of great importance for the future to study the past. There are 77000 culture relics found in china [1]. The monitoring of the sites is important, and this could be achieved through the blend of satellite imagery and machine learning algorithms. All the observations make us take better decisions about how to deal with the situation so that better methods of protection can be developed for preserving the relics. Due to the modernization, several large-scale projects started to be built. At present, Macau focused on protecting immovable culture relics and deployed laws to minimize the damage to relics. There are some significant challenges addressed in identifying the immovable culture relics. They are insufficient enforcement and development control, poor physical planning mechanism, and popular participation and poor funding. In order to predict cultural relics, many researchers have used support vector machine and radial basis function neural network method [2, 3]. The researcher found that the radial basis function neural network method has effective generalization and nonlinear mapping ability [4]. By using the gradient descent method, the minimum value of objective functions is attained, and network structure becomes complex. A support vector machine avoids the local optimization and overfitting but the training time is higher in case of high sample data. Hence, it is significant to identify an effective method to identify the cultural relics [5]. The paper presents the data from the identification process along with the detailed results of the machine algorithms used.


## 2. Study Area

Macau is located in the country of the People’s Republic of China. The city is constructed on the area, which was reclaimed from the sea, and hence most of modern-day Macau was the coastal area of a group of islands that were connected by land reclamation. The city was under the colonial rule of Portugal before being handed over to the Government of China and since then has been classified as a Special Administrative Region of the country. There are numerous sites of archaeological importance in the city of Macau. The city of Macau hosts several UNESCO world heritage sites and due to this reason, the city has seen a major influx of tourists in recent years. The Dragon Boat Festival is held in the city of Macau, and numerous visitors visit the city at that time of the year [6]. All the UNESCO world heritage sites are located in the Historic Centre of Macau. The name was put on the list long back in the year 2005. Some of the buildings were subjected to severe effects of environmental pollution with time. The overall number of sites that are included in the list totals 25. The list of sites is divided into two zones. The sites have been subjected to air pollution more due to the proximity of their location to the modern city. This proximity has also actually increased the influx of tourists. In the second list, only one site is present which was damages due to the construction activities, i.e., Guia Fortress. The overall fortress was used by the military in colonial times. The most famous landmark in the fortress is the chapel and the lighthouse. Some of the prominent features of zone 1 include the Company of Jesus Square, Cathedral Square, and St. Dominic’s Square. The Ruins of St. Paul is another major heritage site in the city of Macau. The area where the monuments are located is actually in extreme proximity to the modern-day city, and hence the impact has been severe on the sites. The main city of Macau is currently the most densely populated region in the world and hence the effects on the heritage sites. The heritage center as it is known is a mix of eastern and western-style architectures. The buildings constructed are also a mix of aesthetic and technological styles. All the mentioned factors prove that the buildings are actually very crucial for research and hence require utmost preservation effort [7]. In addition, the sites show a very wide diversity in terms of religious affinity, and there are rarely any such examples where a mix of religious identity is found in the construction of buildings built in the same era. This shows the convergence of the religious identities and hence picks the interest of the researchers. Some of the examples include the A-Ma Temple that was created to worship the sea goddess Mazu, whereas St. Joseph’s Seminary and Church were the followers of Christianity. The mentioned sites are nearby and are in between two small hills locally referred to as Barra and Mount.The heritage sites are also at risk due to the frequent paths taken by people around the sites. The data shows that the people frequent these paths as morning walk paths for jogging and other activities that increase the pressure on the sites since they are exposed to the people hence more chances of degradation [8]. The mapping data of Figure 1 was obtained from the Google Earth Engine that was crucial for the completion of this study.Figure 1
Map of Macau (source: Google Earth Engine).


## 3. Methods

The data was collected using the Google Earth Engine that is the map providing entity for this project. Google Earth Engine was developed to generate a database that would contain the images captured from the satellites. It also contains the data accumulated from the satellites which is the dataset generated from geospatial data which is used in this particular case. The Google Earth Engine was developed for use by researchers and professionals to under the differences that occur in the landform over time [9]. The Google Earth Engine has been initially released to the public in the year 2001 and has since then seen immeasurable success for the datasets provided by them which has helped researchers in the field of geological, geographical, hydrological, and seismic activity research [10]. The data set used for this search is the satellite images using remote sensing technology. Satellites are used for the collection of data. The images are given as input and classify the images. The grid of the images is taken as Xmax and Ymin. All the satellites, which are used, have been listed in Table 1.Table 1
List of satellites.
Satellite dataResolution usedNoteSpectral resolutionSpatial resolutionQuickbird4 mOwned by digital globeVis-NIR(i) 4 m (Vis-NIR)(ii) PAN (1 m)Landsat 5 TM30 mUSGS and NASA do operations of the satellite jointlyVis-IR-thermal(i) 30 m (Vis-NIR)(ii) PAN (15 m)ASTER15 mIt is a sensor developed by Japan to monitor terrain data of earth from orbitN/A(i) ASTER GDEM (15 m)All the satellites used show high-resolution images that are used for the project of image analysis. Landsat 5 TM is extremely suitable for the production of images that are related to places with high urbanization. The satellite can be used for the process of understanding the changes that are caused in the landscape over the years. The images that were derived from the Quickbird satellite are used for the process of change to deformation of the structure of the land. Then, the ASTER sensor was used for the images that have been used for the process of making the image data related to the activity, which is the tectonic activity of the site to be evaluated in this project [11]. These images, which are captured from the database of the Google Earth Engine, are crucial for the project since they are required for the geospatial analysis of the area. The images that are used here are all remotely collected, and hence the data is not produced from the direct inspection of the sites by the direct intervention of the humans.


## 4. Methodology

Machine learning was introduced back in the twentieth century but the computational ability of the computer in those times was not as fast, and hence their implementation for general research purposes could be only initiated in the twenty-first century with the introduction of computers with high memory and processing power. The algorithm of machine learning trains from the dataset that is provided to the algorithm. The algorithm learns how to identify the images through an iterative process and finally delivers the results [12]. Modern-day machine learning is a part of artificial intelligence and hence does not require any commands after the algorithm has been developed, and a training dataset has been provided to it. In recent times, the use of machine learning has increased manifold due to its ability to analyze complex data, which was otherwise impossible with the traditional methods with reliable accuracy [13]. Data processing is a crucial task of machine learning. As the term data processing suggests, it is the transformation of data from something incomprehensible for humans to a form that the people generally understand [14]. After the machine learning algorithm performs its analysis, it presents the processed data, which is in the form of graphs, or visuals that can be used for understanding the underlying patterns in the data. Two machine learning methods have been used in the case of detection and further classification of the relics of the immovable type. The data which has been used in this case is trained using the algorithm of deep convolutional neural network and CaffeNet [15]. The work of identification of the sites through the traditional physical method is not just time-consuming but also requires a lot of resources both in terms of human labor and money. The generally used machine learning algorithms include the SVM, i.e., support vector machine and decision trees. The SVM however is not able to deliver good results with the image datasets that contain many data [16]. SVM and decision trees can overfit the data that requires other algorithms for the process of analysis of the datasets. In recent times, the results from various data prove that the deep CNN and CaffeNet have better efficiency in the types of data described in the project [17]. The efficiency achieved in the mentioned algorithms like deep CNN and CaffeNet has been over ninety-six in the recent research that has been carried out in the field of machine learning.Figure2 represents the flowchart of the proposed methodology. This figure highlights process of evaluating the accuracy of using machine learning algorithms.Figure 2
Flowchart of the proposed method.


### 4.1. Deep Convolution Neural Network

The data is a very large dataset and hence requires better machine learning algorithms for the process to be completed [18]. Pattern is to be recognized from the satellite is many layered; hence, deep CNN is used. The deep convolutional neural network can be used for the process of separating the data, which is not analyzed, by the use of linear filters that are not capable of handling large image datasets. In deep CNN, the algorithm uses all the filters of the basic types sequentially to arrive at the pattern that is to be observed. The term “Deep” in the deep convolutional neural network refers to the number of layers in the CNN architecture that has been used [19]. The normal line of thought is that the number of layers must be more than ten for the algorithms to be considered as deep convolutional neural network. Many similar networks can be sued for the process including Deep Belief Nets and recurrent neural networks [20]. The deep convolutional neural network uses many types of layers which are embedded in one another. The basic structure of the deep convolutional neural network is the multilayer perceptron [21]. In the case of deep CNN, the most basic fundamental unit, which is the neuron, is present in a specific layer, and there are numerous neurons present in that layer. However, every single neuron in any layer is connected with all the neurons of the other layers. The neurons pass on the information from one layer to another when required for the process of decision-making [22].The main idea to develop the deep CNN technology evolved from the modern-day human brain structure. The human brain is composed of many neurons that are used for the process of making the work of thinking that is being replicated in the deep CNN. The use of deep CNN is extensively seen in the datasets involving images since the algorithm while working with these types of datasets does not need to go through the step of pre-processing [23].In the machine learning algorithms, the requirement of filters is more, and the machine learning engineers design these filters. The machine learning algorithm of deep convolutional neural networks however generates these filters on their own thus simplifying the complicated process of machine learning and recognition of the sites of relics in Macau City [24]. The process is also referred to as the automatic feature extraction process.The convolutional neural networks use the input image and then carry out the image recognition by using the network weights that are associated with the layers that are present in the deep CNN [25]. The more the number of layers, the better the result is obtained but the extra layers help to bring out the low features from the satellite image. Another way, the algorithm uses to make the process, and better is the use of the pooling layers. These layers are inserted between the regular convolutional layers for the process of decreasing the amount of data fed to the layer that is termed as the input [26]. This allows the input layer to control the value and requirements of computation, and then the overall model, which generally gets overfitted, does not over fit in these conditions [27].Table2 shows the number of kernels that are required for the process of making the deep CNN work. Some architecture could be used for the process of implementation of and one such major example is of AlexNet. There are some major differences between the traditional neural networks and current deep convolutional neural networks. The traditional ones were not able to process the images like the deep CNN due to their inability to understand the high-resolution images. The traditional neural networks suffered from many problems and were able to only work on data that was available in the format of pixels [28]. The architecture of the deep convolutional neural network is such that it can take into view the whole of the visual image rather than the approach that was taken by the traditional algorithms.Table 2
Deep convolutional neural network layers.
Layer typeSizeNumber of kennelsNumber of neuronsImage input224×224×396150528Convolution ReLu11×11×3256253440Channel normalization43264ReLu4096Convolution3×3×256Channel normalization384ReLu384Channel normalization256ReLu4096Convolution4096


### 4.2. Caffenet

CaffeNet is a deep convolutional neural network. The algorithm was developed from deep convolutional neural network which was developed by a student named Yangqing Jing. The individual developed which pursuing his research and Ph.D. degree. The individual was studying at UC Berkeley. The BSD 2-Clause License is used for the distribution of the algorithm as this forms its standard license of distribution [29]. Other developers who contributed to the project carried the research forward.Some features make CaffeNet stand out from the rest of the pack. CaffeNet is recently used in most industries because it has been extremely speeding efficiently in terms of processing power. The algorithm could use only a single good processor and with it can carry out the process of image recognition for about 60 million photos or images [30]. This gives it a strong edge over the other algorithms. Hence, when computed down to the value of a single image, it stands at computing a single image in just 1 ms. The speed is the most outstanding factor and is more than any other machine learning program which is generally used [31]. The code, which is used for the development of the algorithm, is openly available and can be used by the other researchers, and then the modifications can better the performance of the algorithm. The algorithm can be made to run on both types of units found in computing devices, i.e., CPU, which stands for Central Processing Unit, and GPU that stands for Graphics Processing Unit [32]. The algorithm is designed in such a way that the algorithm can actually be developed on a Graphics Processing Unit and then used in the generally used personal computers and other devices.The implementation of Caffe code is done using the computer language C++. The algorithm CaffeNet shares overwhelming similarities with another deep convolutional neural network, i.e., AlexNet. Both the algorithms mentioned are actually of the pretrained type and possess a feature that is named as the fast feature embedding [33]. The process of data augmentation is eliminated from the process.Figure3 shows the architecture of the Caffenet algorithm of deep convolutional neural network.Figure 3
Caffenet architecture (source: Jia et al., 2014).


## 4.1. Deep Convolution Neural Network

The data is a very large dataset and hence requires better machine learning algorithms for the process to be completed [18]. Pattern is to be recognized from the satellite is many layered; hence, deep CNN is used. The deep convolutional neural network can be used for the process of separating the data, which is not analyzed, by the use of linear filters that are not capable of handling large image datasets. In deep CNN, the algorithm uses all the filters of the basic types sequentially to arrive at the pattern that is to be observed. The term “Deep” in the deep convolutional neural network refers to the number of layers in the CNN architecture that has been used [19]. The normal line of thought is that the number of layers must be more than ten for the algorithms to be considered as deep convolutional neural network. Many similar networks can be sued for the process including Deep Belief Nets and recurrent neural networks [20]. The deep convolutional neural network uses many types of layers which are embedded in one another. The basic structure of the deep convolutional neural network is the multilayer perceptron [21]. In the case of deep CNN, the most basic fundamental unit, which is the neuron, is present in a specific layer, and there are numerous neurons present in that layer. However, every single neuron in any layer is connected with all the neurons of the other layers. The neurons pass on the information from one layer to another when required for the process of decision-making [22].The main idea to develop the deep CNN technology evolved from the modern-day human brain structure. The human brain is composed of many neurons that are used for the process of making the work of thinking that is being replicated in the deep CNN. The use of deep CNN is extensively seen in the datasets involving images since the algorithm while working with these types of datasets does not need to go through the step of pre-processing [23].In the machine learning algorithms, the requirement of filters is more, and the machine learning engineers design these filters. The machine learning algorithm of deep convolutional neural networks however generates these filters on their own thus simplifying the complicated process of machine learning and recognition of the sites of relics in Macau City [24]. The process is also referred to as the automatic feature extraction process.The convolutional neural networks use the input image and then carry out the image recognition by using the network weights that are associated with the layers that are present in the deep CNN [25]. The more the number of layers, the better the result is obtained but the extra layers help to bring out the low features from the satellite image. Another way, the algorithm uses to make the process, and better is the use of the pooling layers. These layers are inserted between the regular convolutional layers for the process of decreasing the amount of data fed to the layer that is termed as the input [26]. This allows the input layer to control the value and requirements of computation, and then the overall model, which generally gets overfitted, does not over fit in these conditions [27].Table2 shows the number of kernels that are required for the process of making the deep CNN work. Some architecture could be used for the process of implementation of and one such major example is of AlexNet. There are some major differences between the traditional neural networks and current deep convolutional neural networks. The traditional ones were not able to process the images like the deep CNN due to their inability to understand the high-resolution images. The traditional neural networks suffered from many problems and were able to only work on data that was available in the format of pixels [28]. The architecture of the deep convolutional neural network is such that it can take into view the whole of the visual image rather than the approach that was taken by the traditional algorithms.Table 2
Deep convolutional neural network layers.
Layer typeSizeNumber of kennelsNumber of neuronsImage input224×224×396150528Convolution ReLu11×11×3256253440Channel normalization43264ReLu4096Convolution3×3×256Channel normalization384ReLu384Channel normalization256ReLu4096Convolution4096


## 4.2. Caffenet

CaffeNet is a deep convolutional neural network. The algorithm was developed from deep convolutional neural network which was developed by a student named Yangqing Jing. The individual developed which pursuing his research and Ph.D. degree. The individual was studying at UC Berkeley. The BSD 2-Clause License is used for the distribution of the algorithm as this forms its standard license of distribution [29]. Other developers who contributed to the project carried the research forward.Some features make CaffeNet stand out from the rest of the pack. CaffeNet is recently used in most industries because it has been extremely speeding efficiently in terms of processing power. The algorithm could use only a single good processor and with it can carry out the process of image recognition for about 60 million photos or images [30]. This gives it a strong edge over the other algorithms. Hence, when computed down to the value of a single image, it stands at computing a single image in just 1 ms. The speed is the most outstanding factor and is more than any other machine learning program which is generally used [31]. The code, which is used for the development of the algorithm, is openly available and can be used by the other researchers, and then the modifications can better the performance of the algorithm. The algorithm can be made to run on both types of units found in computing devices, i.e., CPU, which stands for Central Processing Unit, and GPU that stands for Graphics Processing Unit [32]. The algorithm is designed in such a way that the algorithm can actually be developed on a Graphics Processing Unit and then used in the generally used personal computers and other devices.The implementation of Caffe code is done using the computer language C++. The algorithm CaffeNet shares overwhelming similarities with another deep convolutional neural network, i.e., AlexNet. Both the algorithms mentioned are actually of the pretrained type and possess a feature that is named as the fast feature embedding [33]. The process of data augmentation is eliminated from the process.Figure3 shows the architecture of the Caffenet algorithm of deep convolutional neural network.Figure 3
Caffenet architecture (source: Jia et al., 2014).


## 5. Results

Due to the increased population, the impacts of cultural relics have been deeper on the environment. The project was initiated to find the method of remotely finding out about the location of the relics since these could not be moved to a safer environment [34]. The identification process has also been associated with the human activity of the people around the sites that is recorded from the map data, which shows the places where the human have the most activity. There was a combination of machine learning algorithms and intelligent remote sensing technology [35]. The results using the CaffeNet and deep convolutional neural network are used for the process with an accuracy of around 90% and higher limit of about 96%, respectively.The area that is shown in Figure4 is the tectonic activity that can be observed through the satellite data. The tectonic activity is important for understanding the data of the structure that is present in the building. The blue areas show lesser tectonic activity, whereas the brighter areas show more activities.Figure 4
Tectonic activity in the heritage site area.The city road network is the map that has been depicted in Figure5. The blue lines running through the pink patch show the constructed roads in the area. The more of the roads, the more the chances of the pollution; hence, the monuments may get affected.Figure 5
City of road network.Figure6 shows the human tracks of human activity in the area near the heritage site of Macau. The points in the red and blue show the human activity in the form of the tracks which is possible in the area and is caused by the people going out for walking in the area [36]. The tracks also present the fact that these are the places where the monitoring of the sites is required to huge human influence in the area.Figure 6
Image of the human tracks in the heritage area.


### 5.1. GEP Findings

Gene expression programming is the full form of the abbreviation GEP. It uses the famous tree structure of the machine learning algorithms. GEP is widely used for machine learning algorithms due to its ability to dynamically change the structure and present the data. The data of the GEP findings, which were recorded, have been presented below in the table, which has been presented in Table3.Table 3
Results from GEP findings.
Results of trainingResults of test of validationResults of test doneAccuracy derived in the training: -0.9150Accuracy in the validation process: -96.00Accuracy in the test process: -96.00Loss derived in the training process: -0.38Loss in the validation process: -0.41Loss in the test results: -0.41The processor, which was used in the process, was the NVIDIA. The size of the epoch, which was used in the process, was 30.Figure7 shows that the value of the loss is decreasing with the increase in the epoch value. The lesser the value of loss, the better is the results that are derived from the deep convolutional neural network and CaffeNet [37, 38]. However, when the data loss is resulting near zero, any further decrease in the loss results in the overfitting of the model. When the model gets overfitted, then the model is not valuable.Figure 7
Loss in training and validation.Figure8 represents the training and validation data for the machine learning algorithm that has been developed. The study findings stated that implementation of RVM-DP and SVM-DP methods in predicting immovable relics has generated 90% of accuracy [39]. This study showed 99% accuracy in the process of evaluation. The results show that the algorithm generally does well with the data, which is the training dataset; however, the same cannot be guaranteed for the test data; though, the accuracy of the data in the test well reaches above 95%. This is because of the reason that the algorithm sometimes is unable to recognize the data from the unknown satellite images.Figure 8
Training and validation accuracy.


## 5.1. GEP Findings

Gene expression programming is the full form of the abbreviation GEP. It uses the famous tree structure of the machine learning algorithms. GEP is widely used for machine learning algorithms due to its ability to dynamically change the structure and present the data. The data of the GEP findings, which were recorded, have been presented below in the table, which has been presented in Table3.Table 3
Results from GEP findings.
Results of trainingResults of test of validationResults of test doneAccuracy derived in the training: -0.9150Accuracy in the validation process: -96.00Accuracy in the test process: -96.00Loss derived in the training process: -0.38Loss in the validation process: -0.41Loss in the test results: -0.41The processor, which was used in the process, was the NVIDIA. The size of the epoch, which was used in the process, was 30.Figure7 shows that the value of the loss is decreasing with the increase in the epoch value. The lesser the value of loss, the better is the results that are derived from the deep convolutional neural network and CaffeNet [37, 38]. However, when the data loss is resulting near zero, any further decrease in the loss results in the overfitting of the model. When the model gets overfitted, then the model is not valuable.Figure 7
Loss in training and validation.Figure8 represents the training and validation data for the machine learning algorithm that has been developed. The study findings stated that implementation of RVM-DP and SVM-DP methods in predicting immovable relics has generated 90% of accuracy [39]. This study showed 99% accuracy in the process of evaluation. The results show that the algorithm generally does well with the data, which is the training dataset; however, the same cannot be guaranteed for the test data; though, the accuracy of the data in the test well reaches above 95%. This is because of the reason that the algorithm sometimes is unable to recognize the data from the unknown satellite images.Figure 8
Training and validation accuracy.


## 6. Conclusion

The study used deep convolutional neural network for evaluating the accuracy of the data sets. This machine learning algorithm can be used for the image recognition process, and the accuracy delivered was quite high. The data was obtained from the satellites, which are used for the collection of data about the terrain of the earth, and this integrates well with the current technology of machine learning. The advancement in the field of machine learning has made the maintenance of heritage sites much easier. The study also presents a unique way to do the tracking of human activity around the relic sites. For future research, the advanced machine learning techniques like gradient boosting techniques to analyze the accuracy.



---

*Source: 6566432-2021-10-25.xml*