1A Deep learning Framework for Cultural
Heritage Damage Detection for Preservation;
Based on the case of Heunginjimun andYeongnamnu in South Korea
Sang-Yun LEE*, Daekyeom Lee***ETRI (Electronics and Telecommunications Research Institute), South Korea**SEASON Co., Ltd., Sejong, Korea
syllee@etri.re.kr, daek29@season.co.kr

In general, there are many restrictions on investigations for safety inspection due to the uniqueness of cultural heritages. Methods
such as visual inspection and non-destructive inspection, which are mainly used as inspection methods, are regularly carried out, butthere are limitations on time and cost. This is insufficient to identify and respond quickly when an abnormal symptom appears incultural heritage. As a basic study of system development for rapid abnormal detection of architectural, cultural properties throughDeep Learning, this paper organized a Deep Learning framework for detecting tilt in buildings for the roof of Heunginjimun Gate
(Korea Treasure No. 1) and Yeongnamnu Pavilion (Korea Treasure No. 147). A framework was developed using a Convolutional
Neural Network (CNN). As a result of an application, EfficientnetB0 and EfficientnetB2 models showed excellent accuracy in detecting
the tilt of the roof of Heunginjimun with an average accuracy of 99.66% and 99.69%, respectively. In addition, EfficientnetB0,EfficientnetB2, and Shufflenet_v2 models showed excellent accuracy in detecting tilt of the roof of Yeongnamnu with 98.81%, 99.80%,and 98.48% accuracy. Additionally, the Grad-CAM experiment was conducted as a basis for whether the model made the proper
judgment to confirm the criteria for determining abnormal detection according to the results of each model. These findings quickly
detect abnormalities occurring in cultural heritages from the perspective of cultural heritage management and preservation, enabling
rapid response, and are valuable for research on artificial intelligence technology related to cultural heritages.
Keyword—Conventional Neural Network, Cultural Heritage, Grad-CAM, Preservation

Sang-Yun Lee(B'94–M'96–D'07) was born in South Korea 1971, is a principal researcher at Police Science & Public
Safety ICT Research Center of Digital Convergence Research Laboratory in ETRI. He has been working at ETRI since1999. In 2008, he received Ph.D. in Electronics and Telecommunications Engineering at the University of Hanyang(Rep.
of Korea). He has been developing technologies in the fields of Broadcasting Communication, System Software,Embedded Software, Artificial Intelligence, and etc.
His main research interests have been in Computational Sciences. Currently, he is involved in developing a technique todetect displacement using Artificial Intelligence technology for CCTV images of cultural assets. He also leads severalprojects including disaster management for cultural heritage. Since 2016, He acts as an editor of the Study Group 16(SG16) in the ITU-T and has been developing international standards. He is an international standards expert at TTA
and a member of the Korean ITU-T Research Committee. He has been carrying out more than 10 government-funded
projects. He is author of more than 70 scientific papers and has registered more than 10 patents.
Daekyeom Lee (B’ 16 – M’18) is a Lead of AI Convergence Technology Research Team at Season. From 2021 to present, hehas been working at Season Co., Ltd. He graduated from Dongguk University (Rep. of Korea) with a double major in energy
systems and economics in 2016 and received a master's degree in engineering from Sejong University (Rep. of Korea) in 2018.
Since 2018, he has conducted research in statistics, data analysis, and artificial intelligence.
His main research interest was big data analysis and data science. Additionally, after joining Season Co., Ltd., he has beenworking as a team leader developing artificial intelligence technology in terms of data utilization. He mainly works on theapplication of artificial intelligence technology using manufacturing data, object detection using deep learning, andimprovement of learning speed through transfer learning. In addition, he is conducting joint research on technology utilization incooperation with related research institutes and companies and has authored 8 domestic and foreign journals.