## 5 Conclusions  
By collaborating with the city government, this research developed a regional-scale and test-operational landslide early warning system that applies the sequential evaluation method. Subsequently, the early warning performance of the developed system was comprehensively analyzed to derive intuitive and pragmatically useful information for the public as well as administrative officials. The analysis was based on simulated early warning level maps for each hourly time step and a collection of historical landslide data that classify 186 slope failure and 36 debris flow data during the rainy seasons (June to October) from 2009 to 2016. The analysis was divided according to two independent focuses: the system's prediction statistics regarding historical landslide data and its temporal performance. The comprehensive results of this research are summarized as follows:  
* According to real-time recorded or forecasted rainfall data, the developed system determines five warning level classes by consecutively applying three different evaluation stages: statistical, physically based, and geomorphological evaluations. * Each of the five warning level phases has its own interpretations derived from the theoretical backgrounds of the evaluation thresholds. Most notably, the highest warning level class, \"Emergency\", was defined to discriminatively issue warnings regarding potential debris flow initiations. * The system exhibits nowcasting and forecasting capabilities and is able to generate two types of early warning level maps depending on the management interest: specific areal warning maps or administrative division warning maps. * As a result of the prediction statistics analysis, 81% (150/186) of historical slope failure data and 86% (31/36) of historical debris flow data were correctly predicted by the \"Alert\" and \"Emergency\" levels, respectively. * Four slope-failure events and five debris-flow events were classified as missed-warning errors, although these were determined as minor local failures of artificial slopes and small-scale erosive soil flows. * As a result of analyzing temporal performance in a study region that is characterized by almost homogeneous rainfall, geo-properties, and hence the initiation timing of warning levels, miscellaneous quantitative information associated to the timing of early warnings was derived to provide answers for pragmatic questions including: \"How much lead time is given?\", \"What is the duration of the issued early warning?\", and \"What is the probability of a landslide occurring given that an early warning has been issued?\"
* During the study period, the system generated a total of 30 warning events. Among the events, five warning events were cases that involved high-class warning levels, and landslides occurred in four out of these five warning events. It was inferred that there exist critical amounts of rainfall that serve as boundaries for triggering landslide events. The development and validation of a LEWS are complex tasks that are never conclusively achieved in any single occasion, but instead requiring robust and consistent support and investment over time for progressive improvements, periodic updates, and adaptations of the technology. Close collaboration and feedback between researchers and operating authorities are essential. In addition, clear divisionsof legal responsibility between authorities and organizations associated to landslide early warnings is another important prerequisite to fully realize the implementation and sustainable operation of a territorial LEWS. Conceptualization, S.-R.L. and J.-Y.P. ; methodology, J.-Y.P. and S.-R.L. ; software, S.-R.L. ; validation, J.-Y.P. ; formal analysis, J.-Y.P. ; investigation, S.-R.L., Y.-T.K., [PERSON]-Y.P., and D.-H.L. ; resources, S.K. and Y.-T.K. ; data curation, S.K. and D.-H.L. ; writing--original draft preparation, J.-Y.P. ; writing--review and editing, S.-R.L. ; visualization, J.-Y.P. and D.-H.L. ; supervision, S.-R.L. ; project administration, Y.-T.K. ; funding acquisition, S.-R.L.