# Applications of 3D Image Using Internet of Things in the Exhibition of Classical Architecture Art Style

Authors: Xusheng Xie; Xin Wen; Fengfeng Deng
Journal: Mobile Information Systems
 (2021)

---

## Abstract

Classical architecture is an architectural, cultural heritage with specific historical value. It is also a city with historical and cultural heritage and even a testimony of the profound historical culture of a country. Therefore, urbanization is unavoidable, and it directly influences historical buildings. This study aims to combine the three-dimensional image techniques and the Internet of Things (IoT) technology to research the development of classical architectural artistic style. This article presents the acquisition equipment, methods, precautions, and data processing of real-life 3D image data of the classical architectural heritage. We realized the online publishing of real-world 3D services of classical architectural heritage through the real-world 3D display system developed through the Internet of Things and mobile terminals. The model was verified through simulation tests. The combination of image processing techniques and analysis methods such as simulated annealing improved the accuracy of the prediction model. The proposed model can provide data support for policy formulation, technical intervention, and targeted field investigation on architectural heritage by screening research objects.


---

## Body

## 1. Introduction

The world is facing the largest wave of urban expansion in history. The entire world is currently experiencing challenges in terms of balancing cultural heritage protection and the requirements of urbanization [1]. Previously, the preservation of culture, nature, and ecosystems was considered a tool for expediting economic growth; today cultural, social, and eco-environmental factors are considered important aspects of the process of sustainable development in addition to economic development. All cultures and civilizations are believed to be vital enablers of sustainable development [2, 3]. Based on these viewpoints, a consensus has been reached among various international organizations that protecting cultural and natural forms of heritage worldwide is an important goal of sustainable development [4, 5].The development of digital technology is gradually changing the way people absorb and spread culture. The digitization of classical heritage and buildings is a new type of architectural protection and maintenance [6]. It combines traditional architectural technology with advanced computer networking and image processing technology to develop a real three-dimensional system to display real scenes, which can reproduce the true appearance of the entire classical building [7]. UNESCO promotes the “Memory of the World” project which aims that in the period of increasing development of computer network technology, the cultural heritage of the world should also be shared worldwide [8]. Currently, many scholars have conducted research on the three-dimensional (3D) modeling and visualization technology of excellent historical buildings, mainly focusing on 3D modeling and 3D laser scanning. In terms of three-dimensional modeling, the three-dimensional model of historical buildings can be drawn directly using virtual reality, which can effectively display the shape and layout of historical buildings [9, 10]. As the world’s largest imperial complex of Ming and Qing dynasties, the Forbidden City in Beijing is also one of the largest and best-preserved wooden structure ancient buildings in the world. The Forbidden City established a digital museum in 2007 and opened the Website of the Digital Forbidden City [11], where the historical buildings and historical materials of the Forbidden City are displayed through the Internet.Classical architecture is an architectural, cultural heritage with specific historical value. It is also a city with historical and cultural heritage and even a testimony of the profound historical culture of a country [12]. Urban expansion and construction have increased the difficulty of protecting historical buildings. In this context, this article combines three-dimensional image processing and IoT technology to carry out the research and construction of the classical architectural, artistic style. We studied the key technologies in the process of 3D digitalization of historical architectural heritage based on the mobile terminal and preliminary investigation and analysis of users. We explored the functions that the display system needs to achieve and the corresponding interface design [13]. In addition, this paper studies the construction of real three-dimensional scenes and the factors that affect the performance of system operation and analyzes performance and efficiency.The rest of the paper is structured as follows: Section2 is about material and methods. In Section 3, different preprocessing methods and simulated annealing algorithms are discussed. In Section 4, the results are presented, and finally, the conclusion is given in Section 5.


## 2. Material and Methods

### 2.1. Visual Interactive Technology

Multi-dimensional data are very common in real life and has very important value [14]. In real life, when data dimensions continue to increase and data volume gradually expands, data analysts need to rely on some auxiliary tools to complete decision-making and analysis [15]. Because human’s intuitive cognitive ability is affected and restricted by the real world, it is difficult to directly receive and understand the information of objects above three dimensions. After processing multi-dimensional data, it can be reduced to a way that humans can directly receive and understand the information above three dimensions [16]. When the multi-dimensional data are reduced to a two-dimensional or three-dimensional graphic image, it is convenient for humans to understand and comprehend the information of the object. In addition, it will greatly improve the observer’s perception and utilization of the data. Therefore, multi-dimensional data visualization and image processing technology have a wide range of applications in the process of classical architectural data analysis.


### 2.2. Data Collection of Classical Buildings Based on IoT

The data collection based on IoT is mainly divided into the investigation of the data collection system and analysis of the data processing architecture. The acquisition system mainly focuses on the energy consumption optimization of the system itself [17, 18]. The related literature proposes a data processing architecture that can complete the collaboration of edge computing and cloud computing. The analysis of architecture proposes to train data analysis models on the cloud server-side and directly call the cloud model on the edge side to complete data analysis and device feedback control. Edge computing architecture helps to achieve effective and fast analysis of IoT data [19]. Figure 1 describes the process of realizing three-dimensional classical architecture. The Internet browser is used to collect the architectural images of different buildings.Figure 1
The process of real-life three-dimensional realization of classical architecture.The collected image information is used to obtain real 3D data. Finally, a 3D scene of the architecture is created.


## 2.1. Visual Interactive Technology

Multi-dimensional data are very common in real life and has very important value [14]. In real life, when data dimensions continue to increase and data volume gradually expands, data analysts need to rely on some auxiliary tools to complete decision-making and analysis [15]. Because human’s intuitive cognitive ability is affected and restricted by the real world, it is difficult to directly receive and understand the information of objects above three dimensions. After processing multi-dimensional data, it can be reduced to a way that humans can directly receive and understand the information above three dimensions [16]. When the multi-dimensional data are reduced to a two-dimensional or three-dimensional graphic image, it is convenient for humans to understand and comprehend the information of the object. In addition, it will greatly improve the observer’s perception and utilization of the data. Therefore, multi-dimensional data visualization and image processing technology have a wide range of applications in the process of classical architectural data analysis.


## 2.2. Data Collection of Classical Buildings Based on IoT

The data collection based on IoT is mainly divided into the investigation of the data collection system and analysis of the data processing architecture. The acquisition system mainly focuses on the energy consumption optimization of the system itself [17, 18]. The related literature proposes a data processing architecture that can complete the collaboration of edge computing and cloud computing. The analysis of architecture proposes to train data analysis models on the cloud server-side and directly call the cloud model on the edge side to complete data analysis and device feedback control. Edge computing architecture helps to achieve effective and fast analysis of IoT data [19]. Figure 1 describes the process of realizing three-dimensional classical architecture. The Internet browser is used to collect the architectural images of different buildings.Figure 1
The process of real-life three-dimensional realization of classical architecture.The collected image information is used to obtain real 3D data. Finally, a 3D scene of the architecture is created.


## 3. 3D Data Processing

### 3.1. Classical Architecture 3D Image Mosaic

Photo stitching or image stitching is the process to combine a group of low-resolution or small-view images with overlapping areas using certain processing techniques to synthesize new high-resolution, large-view images [20]. The combined images contain all the information of the original images before stitching. Image stitching, generally speaking, is to shoot an image sequence about the scene from all the aspects in a scene [21]. For the group of image sequences, stitching is performed, and the stitched image is comprised of all the information of the scene. The mesh division technique which is similar to Mesh is widely then applied to divide the mosaic surface. For example, the American pavilion of the Montreal Expo designed by Berkminster Fuller is a 76-meter-diameter three-quarter sphere. The entire arch is framed by numerous outer triangles and inner hexagonal welded steel pipe grids. Countless transparent plastic materials are sealed between [22]. Lofting is another image processing method that takes a two-dimensional object as a section along a certain path to form a complex three-dimensional object. It has two forms: the first is to sweep through the track, take a curve or curved surface as the cross section of the track, and give it a path to “move” along the path to produce a continuous curved surface. The track can be one or multiple [23]. The second type is the curved surface formed by the transition of each of them and create multiple curved cross sections. The cross section can be a closed curve or a point.


### 3.2. Stitching Data Optimization Based on Simulated Annealing

In this study, the two constraint conditions were the node delay and energy consumption constraint. The purpose of constraint allocation was to determine the allocation plan in the shortest time possible and to fairly allocate the best guaranteed time slot (GTS) resources in each superframe period to ensure communication quality of service (QoS) [24]. Common target allocation problems can be solved by the genetic algorithm, ant colony optimization algorithm, Markov chain state optimization algorithm, and other methods. Due to the variability of the communication status of the sensor network nodes, this study uses the simulated annealing algorithm to find the best resource allocation plan and uses the Metropolis acceptance criterion to include the nonoptimal new solutions in the annealing process into the solution set to avoid the algorithm from falling into the local optimum:(1)f′netj=fnetj1−fnetj.The simulated annealing algorithm is a simulation of the solid combustion annealing process. The model compares the combinatorial optimization problem with the solid combustion annealing process. The mechanism of this method is to start from an initial solution, go through continuous iteration and transformation comparison until the iteration meets the termination condition, and produce the optimal global solution:(2)oj=fnetj.To prevent the algorithm from falling into a local optimal situation, the algorithm creates a random factor and sets the acceptance probability criterion:(3)∑r=1lxjr=xj,j=1,2,…,m,(4)netk=∑j−1qωjkoj.According to equations (3) and (4), the mathematical principle of the algorithm is that the objective function of the problem is the internal energy of the solid in the current microscopic state:(5)Mnφ¨a+hnφa,φ˙a=ut+ρt,ρt=−ΔMφ¨a−Δhφa,φ˙a+dt.The stronger the searchability of the algorithm is, the longer the optimization time will be. For less complicated optimization problems, the population size “m” set by the general algorithm is about 50. If “m” is small, the possibility of falling into the locally optimal solution is very high. In other words, the larger the population number “m,” the stronger the algorithm’s optimizing ability, but it does not increase infinitely:(6)ρt<ρ¯t.The optimal solution of the problem is the microscopic molecular state of the current solid annealing, and the current solution will be randomly perturbed to generate a new solution during the iterative process of algorithm solving. Then, the algorithm judges whether the termination condition set in advance is met. If it is met, stop the search and output the return value. If not, go to the previous step:(7)SplitiXi=−∑j=1kpjmlogpjm.When “m” increases to a certain level, the optimization ability of the regrowth algorithm will no longer increase significantly. The particle dimension is the space dimension, which is determined by the design variables:(8)St=a1t,a1t,…,ant.(9)γxi,St:i|ait≥xi.In this paper, the method of decreasing weight is applied to make the algorithm faster in the early stage of calculation, that is, when it is far from the optimal solution position:(10)φ¨a=a1φ˙a+a2φa+bu+fd.In the later stages of the algorithm calculation, when it is closed to the optimal solution, the search speed cannot be too fast, so that the particles can search for the optimal solution more accurately in the local area.


## 3.1. Classical Architecture 3D Image Mosaic

Photo stitching or image stitching is the process to combine a group of low-resolution or small-view images with overlapping areas using certain processing techniques to synthesize new high-resolution, large-view images [20]. The combined images contain all the information of the original images before stitching. Image stitching, generally speaking, is to shoot an image sequence about the scene from all the aspects in a scene [21]. For the group of image sequences, stitching is performed, and the stitched image is comprised of all the information of the scene. The mesh division technique which is similar to Mesh is widely then applied to divide the mosaic surface. For example, the American pavilion of the Montreal Expo designed by Berkminster Fuller is a 76-meter-diameter three-quarter sphere. The entire arch is framed by numerous outer triangles and inner hexagonal welded steel pipe grids. Countless transparent plastic materials are sealed between [22]. Lofting is another image processing method that takes a two-dimensional object as a section along a certain path to form a complex three-dimensional object. It has two forms: the first is to sweep through the track, take a curve or curved surface as the cross section of the track, and give it a path to “move” along the path to produce a continuous curved surface. The track can be one or multiple [23]. The second type is the curved surface formed by the transition of each of them and create multiple curved cross sections. The cross section can be a closed curve or a point.


## 3.2. Stitching Data Optimization Based on Simulated Annealing

In this study, the two constraint conditions were the node delay and energy consumption constraint. The purpose of constraint allocation was to determine the allocation plan in the shortest time possible and to fairly allocate the best guaranteed time slot (GTS) resources in each superframe period to ensure communication quality of service (QoS) [24]. Common target allocation problems can be solved by the genetic algorithm, ant colony optimization algorithm, Markov chain state optimization algorithm, and other methods. Due to the variability of the communication status of the sensor network nodes, this study uses the simulated annealing algorithm to find the best resource allocation plan and uses the Metropolis acceptance criterion to include the nonoptimal new solutions in the annealing process into the solution set to avoid the algorithm from falling into the local optimum:(1)f′netj=fnetj1−fnetj.The simulated annealing algorithm is a simulation of the solid combustion annealing process. The model compares the combinatorial optimization problem with the solid combustion annealing process. The mechanism of this method is to start from an initial solution, go through continuous iteration and transformation comparison until the iteration meets the termination condition, and produce the optimal global solution:(2)oj=fnetj.To prevent the algorithm from falling into a local optimal situation, the algorithm creates a random factor and sets the acceptance probability criterion:(3)∑r=1lxjr=xj,j=1,2,…,m,(4)netk=∑j−1qωjkoj.According to equations (3) and (4), the mathematical principle of the algorithm is that the objective function of the problem is the internal energy of the solid in the current microscopic state:(5)Mnφ¨a+hnφa,φ˙a=ut+ρt,ρt=−ΔMφ¨a−Δhφa,φ˙a+dt.The stronger the searchability of the algorithm is, the longer the optimization time will be. For less complicated optimization problems, the population size “m” set by the general algorithm is about 50. If “m” is small, the possibility of falling into the locally optimal solution is very high. In other words, the larger the population number “m,” the stronger the algorithm’s optimizing ability, but it does not increase infinitely:(6)ρt<ρ¯t.The optimal solution of the problem is the microscopic molecular state of the current solid annealing, and the current solution will be randomly perturbed to generate a new solution during the iterative process of algorithm solving. Then, the algorithm judges whether the termination condition set in advance is met. If it is met, stop the search and output the return value. If not, go to the previous step:(7)SplitiXi=−∑j=1kpjmlogpjm.When “m” increases to a certain level, the optimization ability of the regrowth algorithm will no longer increase significantly. The particle dimension is the space dimension, which is determined by the design variables:(8)St=a1t,a1t,…,ant.(9)γxi,St:i|ait≥xi.In this paper, the method of decreasing weight is applied to make the algorithm faster in the early stage of calculation, that is, when it is far from the optimal solution position:(10)φ¨a=a1φ˙a+a2φa+bu+fd.In the later stages of the algorithm calculation, when it is closed to the optimal solution, the search speed cannot be too fast, so that the particles can search for the optimal solution more accurately in the local area.


## 4. Analysis Results

In the realization of a real 3D scene, preparing real 3D scene data is the first step to create a real 3D scene presentation. Based on prepared data, a real 3D image was created on the website, which can be browsed on the Internet using a web browser [25–27]. During the browsing process, the data and scene construction were adjusted according to the browsing results and effects to achieve the best results. For this purpose, a popular JavaScript 3D display library is available, developed by Ricardo Cabell, which can be used for the 3D rendering of scenes and models [28].The advantage of using Threes JS in the JavaScript library was that one can quickly write 3D scene and model rendering code with JavaScript syntax and can effectively use web acceleration [29]. Generally speaking, for system testing, all aspects of the system’s functional operation, system reliability, stability, and system compatibility are tested before the system needs to be delivered to ensure that the system is running no matter where it is [30–32]. The system test mainly included three aspects, namely, function test, performance test, and the compatibility test. The results in Figure 2 show the regional distribution of classical buildings in a certain area. Likewise, Figure 3 depicts the three-dimensional display results of the regional distribution of classical buildings in a certain area.Figure 2
Regional distribution of classical architecture in a certain area.Figure 3
The three-dimensional display difference results in the regional distribution of classical buildings in a certain area.The results of data analysis and processing are displayed graphically. Using a combination of points, lines, and surfaces, the various and difficult-to-observe datasets were drawn into charts, keeping in mind the capability of the human eye to recognize shapes [33].Due to the adoption of the framework and the fuzzy set theory to quantitatively analyze the matching calculation results of the energy curve data, there is no need to add redundant calculations in the visualization process, and the fuzzy set calculation results were converted into spatial position drawings and quickly analyzed. Figure4 shows a schematic diagram of the node distribution of classical buildings in a certain area, and Figure 5 describes the efficiency of digital restoration of classical buildings in a certain area, respectively. From the entire digital development process, we can conclude that the direction of research has been improved from the initial stage of establishing picture and image data and establishing virtual scenes to the digitization of real scenes and the degree of digital display. This is because of the development of digital computer technology that another possibility has been brought to protect historic buildings. Considering the historic building itself, the real-life three-dimensional display and dissemination of historical architectural heritage are also developing in terms of how to protect and develop historical buildings. The proposed method explored a path that can protect and develop historic buildings and make more interesting historical buildings. Moreover, it is a platform where people can access the details of historical buildings.Figure 4
Schematic diagram of node distribution of classical architecture in a certain area.Figure 5
Digital restoration efficiency of classical buildings in a certain area.


## 5. Conclusion

The development of digital technology is gradually changing the way people absorb and spread culture. The digitization of classical buildings is a new type of architectural protection. In terms of data processing, this article focused on the processing of panoramic data. It introduced the stitching process of panoramic data as a whole. From data preprocessing to feature extraction, feature matching, and feature fusion, all the steps introduce panoramic images. We introduced the topological relationship and the road network generation, the editing of trajectory, and the summary of data. Many problems were also encountered in the design and implementation of the overall process. In the preprocessing steps, there are many manual processes, which consume the workforce and time and affect the progress of the system implementation. Therefore, in future work, we will integrate the equipment as a whole, start with the data collection, and integrate the equipment and the preprocessing of the data.



---

*Source: 2283354-2021-07-27.xml*