## 3 Performance Analysis of SEM-LEWS Based on the Simulation Results of 8-Year Histories
### Establishment of Historical Landslide Database Using Indirect Field-Checks  
In our study, a historical landslide database was used to analyze and validate the developed landslide early warning system. The database consists of information on 317 reported landslides that occurred in Busan between 1999 and 2016. Primary information on these historical landslides was obtained from various sources, including landslide documents from the local government of Busan, articles from internet news archives, landslide files obtained from the National Institute of Forest Science, as well as information from several district offices and a local university in Busan. These primary sources, in a text data sheet format, generally included the following information: date of landslide occurrence, landslide location using either a road-name or land-lot format, surrounding landmarks or buildings, area of damage, and recovery costs. In an effort to construct a point feature GIS dataset that identifies landslide source areas, all landslide locations reported in the primary sources were indirectly field-checked using maps provided by Google Earth and two South Korean web portals  
Figure 5: Performance analysis framework of the landslide early warning system for Busan applying the sequential evaluation method. (Naver and Daum). These map services provide annual satellite imagery, as well as aerial photographs, and street views to the public (Figure 6). The aerial photographs shown in Figure 7 represent the indirect field-check method, where a reported landslide location sourced from primary data sheets was aerial-photographed by a web portal service (Kakao Map) before and after the landslide (Figure 7a and b, respectively). By comparing the two photographs, information on the precise location of the landslide source area and landslide type (debris flow or slope failure) could be obtained. Figure 6: Map services provided by Google Earth and South Korean web portals including (**a**) street view [mar]; (**b**) satellite imagery (Google Earth) and (**c**) aerial views [34]. Figure 7: Explanation of the indirect field-check method using map services from a web portal [35], with aerial photographs taken (**a**) before and (**b**) after landslide occurrence. Based on these processes, a database consisting of 317 historical landslide data points was established in the form of GIS point feature classes (Figure 8a). Within this context, Table 2 specifies the number of GIS landslide data points according to landslide occurrence date and spatial precision class. The GIS historical landslide data was classified into three hierarchical groups according to the spatial precision of the estimated landslide source area. First-class spatial precision data included data with landslide initiation source areas clearly confirmed through indirect field-checks. Second-class spatial precision data included data with specific landslide locations, but with a lower confidence level compared to first-class data, due to a lack of confirmation through indirect field checks. Error distances of second-class data points from actual landslide initiation source areas were estimated to be less than 50 m. The first and second-class data were considered applicable for refined validation analyses on spatial performances of advanced models. Third-class spatial precision data, however, included somewhat randomly marked landslide locations within the respective boundaries of administrative areas provided by the primary sources. This is due to the provided administrative boundaries being too broad to specify landslide locations, in addition to failure to find evidential areas of landslide occurrence from indirect field checks. The maximum error distance of third-class data points from actual landslide initiation source areas was estimated to be approximately 400 m. Due to such low spatial precision, the third-class landslide data was only considered for analyses that were less sensitive to the spatial precision of historical landslide data.