# An Application of ANN Ensemble for Estimating of Precipitation Using Regional Climate Models

Authors: Dongwoo Jang
Journal: Advances in Civil Engineering
 (2021)

---

## Abstract

Climate change scenarios are used for predicting future precipitation. More detailed regional climate change scenarios are being used through dynamic downscale based on global circulation model results. There is a global tendency to utilize simulated precipitation data from downscaled regional climate models (RCMs) suitable for each country. In Korea, there are studies for improving the accuracy of climate change scenario precipitation forecasts compared with observed precipitation. In this study, the precipitation of five regional climate models and actual observed precipitation provided in Korea are applied to ANN (artificial neural network), which suggests ways to improve prediction accuracy for precipitation. The ANN ensemble of RCMs simulates the actual observed precipitation more accurately than the individual RCM. In particular, it is more effective inland than in coastal areas, where precipitation patterns are complex. Pearson correlation coefficient of ANN is high as 0.04 compared with MRA. It is expected that more detailed analysis will be possible if it is applied not only to four cities but also to other regions in Korea. If observed precipitation data are collected in sufficient quantity, the applicability of the ANN model will widen.


---

## Body

## 1. Introduction

Construction of future climate scenarios under global warming conditions, at a local or regional scale, is necessary for the assessment of climate change impacts on economic activities such as agriculture or energy production [1]. General circulation models (GCMs) represent the most satisfactory approach to predicting future climate changes [2, 3], but their present low spatial resolution measuring a few hundred kilometers makes their output problematic for use in impact studies [4, 5].Various developed methods have been applied to downscale from GCM coarse resolution output to finer spatial scale (e.g., local stations or river catchment). Physically based numerical models such as regional dynamical models that work at a finer spatial scale are nested within a GCM [6]. Current applications of this method have revealed its capacity to reproduce fine-scale features of regional climates [7–9]. In spite of the substantial improvement in computer abilities in recent decades, this numerical model method remains computationally very demanding [10].In recent years, a number of reports and papers within the meteorological community have adopted neural network as a method to downscale from large-scale atmospheric circulation to regional climate variables [11, 12]. Several application methods are developed for the purposes of constructing climate change scenarios [3, 10].Other authors have applied ANN (artificial neural network) models to calculate problems on a seasonal scale [13–15] and to short-term (0 to 6 hourly) predictions [16]. Pattern recognition and Atlantic blocking detection using ANN are becoming gradually important [17, 18]. And there are various researches about advanced ANN such as M5 model tree (M5Tree), multivariable adaptive regression spline (MARS), least-square support vector regression (LSSVR), and gene-expression programming (GEP) models and various applications of ANN [19–25].ANN models can be trained to establish the closest mathematical relationship between atmospheric circulation and the local climate, without predefined limitations. Thus, this method is able to capture some of the nonlinear relationships between the local climate and large-scale circulation [1]. In particular, it has been shown that there is always a certain configuration of the multilayer perceptron that can arbitrarily approximate any continuous function [26, 27]. In this sense, it is helpful to regard the multilayer perceptron as a very powerful multiple regression technique [28]. Gardner and Dorling (1998) have shown that most atmospheric sciences ANN applications have employed standard feed-forward configuration of the multilayer perceptron [1, 29].Downscaling methods are broadly classified as dynamic or statistical. Statistical downscaling is the development of statistical relationships between local climate variables and large-scale predictors. Dynamic downscaling is based upon nesting a finer scale regional climate model (RCM) of up to 10 km horizontal resolution within GCMs. Downscaling methods can be used to improve the spatial resolution of GCM output in order to overcome the challenge of assessing climate-related impacts on water resources at the catchment scale [30–32].For the prediction of future climate conditions, global climate models were used. The grid resolution of current GCMs is in the order of hundreds of kilometers. However, the resolution required by environmental impact models is less than 10 km.Therefore, as Tsanis et al. indicate, changes in precipitation and/or temperature predicted by GCMs for the Special Report on Emissions Scenarios (SRES) in small Mediterranean river basins cannot be used to assess extreme hydrological impacts, without previously applying a downscaling methodology. Two main downscaling techniques are available to represent this subgrid variability: (a) a dynamic approach using regional climate models (RCMs) and (b) a statistical approach [33].Compared with RCMs, statistical methods include simpler computational procedures as well as the ability to produce results that can be directly compared with the regional observed data [34]. Therefore, statistical downscaling techniques have been widely used for climate impact studies [32, 35–39].A method exists in which it is possible to apply an ensemble of several models in such a way as to reduce uncertainty in climate change scenarios [40]. This ensemble method uses a single-model ensemble (SME) that combines various initial conditions and a multimodel ensemble (MME) that combines multiple model results.These include an ensemble method that averages the values of climate models, a Reliability Ensemble Average method that combines weights according to the accuracy of global models, and a method of using a median for each model. A method of synthesizing the values is also used [41–43].In this study, observed precipitation data were compared with estimated precipitation data using the RCM. Via multilayer perceptron of the artificial neural network, it also helps to improve the accuracy of precipitation prediction by using an ANN ensemble for precipitation data generated from the RCM. For this purpose, data on RCM presented by the Korea Meteorological Administration and Coordinated Regional Climate Downscaling Experiment (CORDEX) were collected and applied to artificial neural networks.Data for the artificial neural networks comprised the monthly precipitation of five RCMs used in Korea from 2006 to 2015. During that period, simulated precipitation by RCM at the same point as the observed rainfall was compared through the Pearson correlation coefficient in four major cities in Korea.Through the coefficient of determination, the model closest to the actual precipitation was estimated, further estimating new precipitation by using the RCM precipitation data as an independent variable of the artificial neural network.Based on the Pearson correlation coefficient, the optimal RCM for each city was selected. An ANN model was then built showing precipitation by the selected RCM as an independent variable. ANN models using five RCMs of four observatories were simulated, and ANN simulation cases were determined by the RCM’s combination in order of lowest determination coefficients. Accuracy assessment was performed in order to compare the precipitation data generated by the ANN model with the data actually observed.


## 2. Theoretical Background

Artificial neural network (ANN) was used as a statistical analysis method to find correlations between measured and simulated values of precipitation by the RCM. In addition, accuracy assessment was performed to compare the ANN-derived simulation results with measured data. Section2 covers the theories of the statistical methods used.


### 2.1. ANN Analysis for Determining Precipitation from RCM

The analysis procedure of the artificial neural network consists of input, hidden, and output layers, as shown in Figure1. Learning and output are done through backpropagation algorithms.Figure 1
Schematic diagram of a multilayer feed-forward neural network [37, 38].According to Haykin (1994), a neural network is a massively parallel distributed processor that has a natural propensity for storing experiential knowledge that can then be made available for use. The neural network resembles the human brain in two respects: knowledge is acquired by the network through a learning process, and interneuron connection strengths (known as synaptic weights) are used to store the knowledge. The ANN procedure used is a feed-forward network type with input, hidden, and output layers [44].Neurons in the input layer simply act as a buffer. Those in different layers are interconnected by means of weights. The neurons in the hidden and output layers are called the activation function, which when used here is a sigmoidal activation function. The input to each neuronj in the hidden layer is the sum of the weighted input signal xi (∑wjixi=netj, in which wji is the interconnecting weight between neuron j in the hidden layer and neuron i in the input layer). Output yj from the neuron is given by [44, 45](1)yi=f∑wjixi=11+e−netj.Analysis by error propagation algorithm compares the calculated value in all directions of the neural network with the target value and adjusts the connection strength (weighting factor) so that the square sum of the errors is minimized. This is performed until the error meets a certain value. When the iteration is finished, the final value is calculated as output.


### 2.2. Accuracy Assessment for Selecting Optimal ANN Model

In evaluating the accuracy of the leakage ratio using the ANN developed during research, an error ratio analysis was performed to evaluate the difference between the actual and model values. Accuracy assessment can be estimated by comparing the actual measured value with that generated by model simulation.The mean absolute error (MAE), mean square error (MSE), BIAS, and G value were used to evaluate the accuracy of the estimation results. The calculation method of each equation is shown in the following equations:(2)MAE=1n∑i=1nzxi−z^xi,(3)MSE=1n∑i=1nzxi−z^xi2,(4)BIAS=1n∑i=1nzxi−z^xi,(5)G=1−∑i=1nzxi−z^¯xi2∑i=1nzxi−z¯2×100,where z^xi is the estimated value at i and z¯ is the mean value of total usage data.If the MAE and MSE are smaller, the estimated value indicates higher accuracy. The nearer the BIAS is to zero, the less biased the estimation result is. A G value of 100 indicates a perfect estimation. A negative G value indicates it as being less reliable than using an average of data values as a predictor [45].


### 2.3. Pearson Coefficient for Correlation Analysis

In statistics, the Pearson correlation coefficient (PCC), also referred to as Pearson’sr, is a measure of the linear correlation between the variables X and Y. It has a value between +1 and −1, where 1 is a total positive linear correlation, 0 is no linear correlation, and −1 is a total negative linear correlation. Widely used in the sciences, PCC was developed by Karl Pearson in the 1880s from a related idea introduced by Francis Galton [46–48].Correlation analysis is a method of analyzing the linear relationship between two variables in probability theory and statistics. The two variables can be correlated with each other from an independent relationship, and the strength of the relationship between them is the correlation coefficient, as defined in(6)rxy=∑xi−x¯yi−y¯∑i=1nxi−x¯2∑i=1nyi−y¯2,where x¯ and y¯ are mean values of x and y.The PCC (r) is also used to determine the relationship between two variables with a coefficient of determination (r2). The PCC is used to explain the linear relationship between the two variables. However, it may prove insufficient to determine the accuracy between the target and simulated values.Though the results of the two variables are positively correlated, the linearity of the linear equation cannot be accurately estimated according to its slope and constant value, so an error rate analysis is performed when evaluating accuracy. If the linear relationship between the comparison variables is shown, it is concluded that the relationship between the two variables is closely related to the tendency of the trend line and the degree of convergence of the values [45].


## 2.1. ANN Analysis for Determining Precipitation from RCM

The analysis procedure of the artificial neural network consists of input, hidden, and output layers, as shown in Figure1. Learning and output are done through backpropagation algorithms.Figure 1
Schematic diagram of a multilayer feed-forward neural network [37, 38].According to Haykin (1994), a neural network is a massively parallel distributed processor that has a natural propensity for storing experiential knowledge that can then be made available for use. The neural network resembles the human brain in two respects: knowledge is acquired by the network through a learning process, and interneuron connection strengths (known as synaptic weights) are used to store the knowledge. The ANN procedure used is a feed-forward network type with input, hidden, and output layers [44].Neurons in the input layer simply act as a buffer. Those in different layers are interconnected by means of weights. The neurons in the hidden and output layers are called the activation function, which when used here is a sigmoidal activation function. The input to each neuronj in the hidden layer is the sum of the weighted input signal xi (∑wjixi=netj, in which wji is the interconnecting weight between neuron j in the hidden layer and neuron i in the input layer). Output yj from the neuron is given by [44, 45](1)yi=f∑wjixi=11+e−netj.Analysis by error propagation algorithm compares the calculated value in all directions of the neural network with the target value and adjusts the connection strength (weighting factor) so that the square sum of the errors is minimized. This is performed until the error meets a certain value. When the iteration is finished, the final value is calculated as output.


## 2.2. Accuracy Assessment for Selecting Optimal ANN Model

In evaluating the accuracy of the leakage ratio using the ANN developed during research, an error ratio analysis was performed to evaluate the difference between the actual and model values. Accuracy assessment can be estimated by comparing the actual measured value with that generated by model simulation.The mean absolute error (MAE), mean square error (MSE), BIAS, and G value were used to evaluate the accuracy of the estimation results. The calculation method of each equation is shown in the following equations:(2)MAE=1n∑i=1nzxi−z^xi,(3)MSE=1n∑i=1nzxi−z^xi2,(4)BIAS=1n∑i=1nzxi−z^xi,(5)G=1−∑i=1nzxi−z^¯xi2∑i=1nzxi−z¯2×100,where z^xi is the estimated value at i and z¯ is the mean value of total usage data.If the MAE and MSE are smaller, the estimated value indicates higher accuracy. The nearer the BIAS is to zero, the less biased the estimation result is. A G value of 100 indicates a perfect estimation. A negative G value indicates it as being less reliable than using an average of data values as a predictor [45].


## 2.3. Pearson Coefficient for Correlation Analysis

In statistics, the Pearson correlation coefficient (PCC), also referred to as Pearson’sr, is a measure of the linear correlation between the variables X and Y. It has a value between +1 and −1, where 1 is a total positive linear correlation, 0 is no linear correlation, and −1 is a total negative linear correlation. Widely used in the sciences, PCC was developed by Karl Pearson in the 1880s from a related idea introduced by Francis Galton [46–48].Correlation analysis is a method of analyzing the linear relationship between two variables in probability theory and statistics. The two variables can be correlated with each other from an independent relationship, and the strength of the relationship between them is the correlation coefficient, as defined in(6)rxy=∑xi−x¯yi−y¯∑i=1nxi−x¯2∑i=1nyi−y¯2,where x¯ and y¯ are mean values of x and y.The PCC (r) is also used to determine the relationship between two variables with a coefficient of determination (r2). The PCC is used to explain the linear relationship between the two variables. However, it may prove insufficient to determine the accuracy between the target and simulated values.Though the results of the two variables are positively correlated, the linearity of the linear equation cannot be accurately estimated according to its slope and constant value, so an error rate analysis is performed when evaluating accuracy. If the linear relationship between the comparison variables is shown, it is concluded that the relationship between the two variables is closely related to the tendency of the trend line and the degree of convergence of the values [45].


## 3. Data Collection of Climate Change Scenarios and Model Building

### 3.1. Precipitation Data Collection Generated by RCMs

#### 3.1.1. Provision of RCM for Generating Precipitation Data

The RCP scenarios in the IPCC’s fifth assessment report are input into climate models and are provided through the IPCC’s data distribution center. The development of global climate change scenarios (CMIP5) produces global data based on RCP scenarios. Based on global data, RCM regionally generated climate change scenarios are available in the local climate specification experiment (CORDEX) [49].CORDEX calculates regional scenarios of climate change by dividing the world into 14 regions, including North America, Europe, Africa, and East Asia, at a resolution of 0.44° (about 50 km). With regard to Korea, data on climate change forecasts by the RCM can be used in CORDEX-EA for the East Asia region by HadGEM3-RA, RegCM4, SNU-MM5, SNU-WRF, and YSU-RSM [49].Each website providing climate change scenarios provides data through various models, data periods, variables, and resolutions. This paper collected data from the CORDEX-EA and the websites of the Korea Global Atmosphere Watch Center (KGAWC) [50], which provides RCM results of several variables in RCP climate change scenarios.The World Climate Research Program (WCRP) initiated the CORDEX framework in order to improve worldwide regional climate change projections and provide a framework for better coordination of regional climate downscaling [51].The CORDEX experiments provided an opportunity to evaluate the relative and absolute performances of various RCMs over predefined regions. As one of 14 branch domains within the CORDEX framework, CORDEX-East Asia (CORDEX-EA) covers a large area of East Asia with a horizontal resolution of approximately 50 km [52].In the framework of the CORDEX project, five RCMs driven by ERA-Interim reanalysis were evaluated in terms of their ability to simulate the climatology of the East Asia domain (CORDEX-EA) in comparison with the Aphrodite observational data set [53]. The models are affected by a temperature bias ranging from 1.1 (RegCM4) to 0.2°C (SNU-MM5) from June to August and from 5.5 (SNU-MM5) to 0.9°C (RegCM4) from December to February [53].


#### 3.1.2. Collection of Precipitation Data from RCM

To compare results of the RCM and actual data in order to determine climate model forecasting accuracy, data were selected for use based on the status of the climate change scenario. The climate change scenarios collected are shown in Table1.Table 1
Data collection from RCM [49].
ClassificationContentsData sourceCORDEX-EAClimate change scenarioRCP 8.5Regional climate modelHadGEM3-RA, SNU-WRF, RegCM4, SNU-MM5, and YSU-RSMSpatial rangeEast AsiaTime range (year)2006∼2015Spatial and time resolution50 km/monthlyVariablePrecipitationData formatNetCDF(.nc)Data for the same period are required to compare climate change scenarios and observations. Thus, 50 km resolution precipitation data for 2006∼15 overlapping with the previous period’s data were collected from the HadGEM3-RA, RegCM4, SNU-WRF, SNU-MM5, and CORDEX-EA YSU-RS prediction data. Only monthly SUN-WRF data were available from 2006 to 2010. Daily data of SNU-WRF were thus collected and then converted into monthly data from 2011 to 2015. So monthly data by all RCMs were collected for ANN simulation.The collected RCM data were interpolated using Universal Kriging, the optimal spatial interpolation method available to collect data at the same location as the actual observatory site [49]. The interpolated precipitation data are used in the study as shown in Section 3.3.


### 3.2. Observation Data Collection for Comparison with RCM

Precipitation data are available through the KMA website and Korea Global Atmosphere Water Center [54]. Ground observation data are classified into the automated synoptic observation system (ASOS), automatic weather system (AWS), and automated agricultural observing system (AAOS). Data are provided in CSV and PDF formats.ASOS comprises ground observations performed at the same time on all observatories in order to determine the atmospheric conditions at any given time. Air pressure, temperature, wind direction, wind speed, relative humidity, precipitation, solar radiation, daylight time, ground temperature, vertical temperature, and ground temperature are all automatically observed. Each observation station is measuring the precipitation by one-minute time unit. It is currently being observed at more than 80 sites in Korea [54].Observed precipitation data for 2006∼2015, consistent with the RCM data’s period, were collected [46]. Observation points were selected from the synoptic meteorological network, where no data were missing. No movement of precipitation observation equipment was seen over the measurement period.Location information on 4 selected observation points of precipitation data collected through the Korea Global Atmosphere Water Center is shown in Figure2.Figure 2
Locations of 4 major cities in Korea.


### 3.3. Status of Collected Data

In all four cities, monthly precipitation by RCMs and observed precipitation patterns were similar. In the case of Seoul, observed precipitation in the summer season was higher than the RCM simulation. In the case of Busan, RCM simulated precipitation was frequently higher than the observed precipitation.Park (2017) analyzed five RCMs in Korea and found a similar correlation between observed and RCM precipitation, whether located inland or in northern parts [49]. Figure 3 shows that precipitation forecasting patterns for Seoul and Daejeon, both situated further north than Daegu and Busan, are more stable. As shown in the precipitation comparison graph, the monthly precipitation of five RCMs demonstrates a differing pattern.Figure 3
Monthly precipitation graph from RCMs and observatory of 4 major cities: (a) Seoul, (b) Daejeon, (c) Daegu, and (d) Busan.
(a)(b)(c)(d)Table2 shows the Pearson coefficients for the actual precipitation and the RCM simulated precipitation in four major Korean cities. As can be seen in the table, all have a positive correlation of 0.2∼0.6 for each model.Table 2
Pearson coefficient of precipitation between RCMs and observatories.
ClassificationHadGEM3-RARegCMSNU-WRFSNU-MM5YSU-RSMMonthly precipitationSeoul0.530.460.350.320.31Daejeon0.510.480.410.490.60Daegu0.420.520.340.470.56Busan0.410.290.200.280.32Annual precipitationSeoul0.04−0.280.04−0.02−0.26Daejeon0.32−0.49−0.57−0.47−0.36Daegu0.08−0.12−0.180.07−0.11Busan0.23−0.060.150.040.20Seasonal precipitation (Jul.–Sep.)Seoul0.560.030.100.200.28Daejeon0.470.200.160.250.45Daegu0.400.290.350.340.35Busan0.490.330.440.390.31Seasonal precipitation (Oct.–Jun.)Seoul0.320.220.11−0.03−0.08Daejeon0.330.170.190.230.45Daegu0.240.20−0.020.170.48Busan0.31−0.02−0.13−0.080.12In particular, the Busan area has a low Pearson coefficient except for the HadGEM3-RA model. Except for the HadGEM3-RA model, in Busan, it is found that the correlation with observed precipitation is low. Correlation between the three cities also varies depending on the model used. Of the five RCMs, the HadGEM3-RA model generally shows a high correlation, while the YSU-RSM shows a high correlation in Daejeon and Daegu.


## 3.1. Precipitation Data Collection Generated by RCMs

### 3.1.1. Provision of RCM for Generating Precipitation Data

The RCP scenarios in the IPCC’s fifth assessment report are input into climate models and are provided through the IPCC’s data distribution center. The development of global climate change scenarios (CMIP5) produces global data based on RCP scenarios. Based on global data, RCM regionally generated climate change scenarios are available in the local climate specification experiment (CORDEX) [49].CORDEX calculates regional scenarios of climate change by dividing the world into 14 regions, including North America, Europe, Africa, and East Asia, at a resolution of 0.44° (about 50 km). With regard to Korea, data on climate change forecasts by the RCM can be used in CORDEX-EA for the East Asia region by HadGEM3-RA, RegCM4, SNU-MM5, SNU-WRF, and YSU-RSM [49].Each website providing climate change scenarios provides data through various models, data periods, variables, and resolutions. This paper collected data from the CORDEX-EA and the websites of the Korea Global Atmosphere Watch Center (KGAWC) [50], which provides RCM results of several variables in RCP climate change scenarios.The World Climate Research Program (WCRP) initiated the CORDEX framework in order to improve worldwide regional climate change projections and provide a framework for better coordination of regional climate downscaling [51].The CORDEX experiments provided an opportunity to evaluate the relative and absolute performances of various RCMs over predefined regions. As one of 14 branch domains within the CORDEX framework, CORDEX-East Asia (CORDEX-EA) covers a large area of East Asia with a horizontal resolution of approximately 50 km [52].In the framework of the CORDEX project, five RCMs driven by ERA-Interim reanalysis were evaluated in terms of their ability to simulate the climatology of the East Asia domain (CORDEX-EA) in comparison with the Aphrodite observational data set [53]. The models are affected by a temperature bias ranging from 1.1 (RegCM4) to 0.2°C (SNU-MM5) from June to August and from 5.5 (SNU-MM5) to 0.9°C (RegCM4) from December to February [53].


### 3.1.2. Collection of Precipitation Data from RCM

To compare results of the RCM and actual data in order to determine climate model forecasting accuracy, data were selected for use based on the status of the climate change scenario. The climate change scenarios collected are shown in Table1.Table 1
Data collection from RCM [49].
ClassificationContentsData sourceCORDEX-EAClimate change scenarioRCP 8.5Regional climate modelHadGEM3-RA, SNU-WRF, RegCM4, SNU-MM5, and YSU-RSMSpatial rangeEast AsiaTime range (year)2006∼2015Spatial and time resolution50 km/monthlyVariablePrecipitationData formatNetCDF(.nc)Data for the same period are required to compare climate change scenarios and observations. Thus, 50 km resolution precipitation data for 2006∼15 overlapping with the previous period’s data were collected from the HadGEM3-RA, RegCM4, SNU-WRF, SNU-MM5, and CORDEX-EA YSU-RS prediction data. Only monthly SUN-WRF data were available from 2006 to 2010. Daily data of SNU-WRF were thus collected and then converted into monthly data from 2011 to 2015. So monthly data by all RCMs were collected for ANN simulation.The collected RCM data were interpolated using Universal Kriging, the optimal spatial interpolation method available to collect data at the same location as the actual observatory site [49]. The interpolated precipitation data are used in the study as shown in Section 3.3.


## 3.1.1. Provision of RCM for Generating Precipitation Data

The RCP scenarios in the IPCC’s fifth assessment report are input into climate models and are provided through the IPCC’s data distribution center. The development of global climate change scenarios (CMIP5) produces global data based on RCP scenarios. Based on global data, RCM regionally generated climate change scenarios are available in the local climate specification experiment (CORDEX) [49].CORDEX calculates regional scenarios of climate change by dividing the world into 14 regions, including North America, Europe, Africa, and East Asia, at a resolution of 0.44° (about 50 km). With regard to Korea, data on climate change forecasts by the RCM can be used in CORDEX-EA for the East Asia region by HadGEM3-RA, RegCM4, SNU-MM5, SNU-WRF, and YSU-RSM [49].Each website providing climate change scenarios provides data through various models, data periods, variables, and resolutions. This paper collected data from the CORDEX-EA and the websites of the Korea Global Atmosphere Watch Center (KGAWC) [50], which provides RCM results of several variables in RCP climate change scenarios.The World Climate Research Program (WCRP) initiated the CORDEX framework in order to improve worldwide regional climate change projections and provide a framework for better coordination of regional climate downscaling [51].The CORDEX experiments provided an opportunity to evaluate the relative and absolute performances of various RCMs over predefined regions. As one of 14 branch domains within the CORDEX framework, CORDEX-East Asia (CORDEX-EA) covers a large area of East Asia with a horizontal resolution of approximately 50 km [52].In the framework of the CORDEX project, five RCMs driven by ERA-Interim reanalysis were evaluated in terms of their ability to simulate the climatology of the East Asia domain (CORDEX-EA) in comparison with the Aphrodite observational data set [53]. The models are affected by a temperature bias ranging from 1.1 (RegCM4) to 0.2°C (SNU-MM5) from June to August and from 5.5 (SNU-MM5) to 0.9°C (RegCM4) from December to February [53].


## 3.1.2. Collection of Precipitation Data from RCM

To compare results of the RCM and actual data in order to determine climate model forecasting accuracy, data were selected for use based on the status of the climate change scenario. The climate change scenarios collected are shown in Table1.Table 1
Data collection from RCM [49].
ClassificationContentsData sourceCORDEX-EAClimate change scenarioRCP 8.5Regional climate modelHadGEM3-RA, SNU-WRF, RegCM4, SNU-MM5, and YSU-RSMSpatial rangeEast AsiaTime range (year)2006∼2015Spatial and time resolution50 km/monthlyVariablePrecipitationData formatNetCDF(.nc)Data for the same period are required to compare climate change scenarios and observations. Thus, 50 km resolution precipitation data for 2006∼15 overlapping with the previous period’s data were collected from the HadGEM3-RA, RegCM4, SNU-WRF, SNU-MM5, and CORDEX-EA YSU-RS prediction data. Only monthly SUN-WRF data were available from 2006 to 2010. Daily data of SNU-WRF were thus collected and then converted into monthly data from 2011 to 2015. So monthly data by all RCMs were collected for ANN simulation.The collected RCM data were interpolated using Universal Kriging, the optimal spatial interpolation method available to collect data at the same location as the actual observatory site [49]. The interpolated precipitation data are used in the study as shown in Section 3.3.


## 3.2. Observation Data Collection for Comparison with RCM

Precipitation data are available through the KMA website and Korea Global Atmosphere Water Center [54]. Ground observation data are classified into the automated synoptic observation system (ASOS), automatic weather system (AWS), and automated agricultural observing system (AAOS). Data are provided in CSV and PDF formats.ASOS comprises ground observations performed at the same time on all observatories in order to determine the atmospheric conditions at any given time. Air pressure, temperature, wind direction, wind speed, relative humidity, precipitation, solar radiation, daylight time, ground temperature, vertical temperature, and ground temperature are all automatically observed. Each observation station is measuring the precipitation by one-minute time unit. It is currently being observed at more than 80 sites in Korea [54].Observed precipitation data for 2006∼2015, consistent with the RCM data’s period, were collected [46]. Observation points were selected from the synoptic meteorological network, where no data were missing. No movement of precipitation observation equipment was seen over the measurement period.Location information on 4 selected observation points of precipitation data collected through the Korea Global Atmosphere Water Center is shown in Figure2.Figure 2
Locations of 4 major cities in Korea.


## 3.3. Status of Collected Data

In all four cities, monthly precipitation by RCMs and observed precipitation patterns were similar. In the case of Seoul, observed precipitation in the summer season was higher than the RCM simulation. In the case of Busan, RCM simulated precipitation was frequently higher than the observed precipitation.Park (2017) analyzed five RCMs in Korea and found a similar correlation between observed and RCM precipitation, whether located inland or in northern parts [49]. Figure 3 shows that precipitation forecasting patterns for Seoul and Daejeon, both situated further north than Daegu and Busan, are more stable. As shown in the precipitation comparison graph, the monthly precipitation of five RCMs demonstrates a differing pattern.Figure 3
Monthly precipitation graph from RCMs and observatory of 4 major cities: (a) Seoul, (b) Daejeon, (c) Daegu, and (d) Busan.
(a)(b)(c)(d)Table2 shows the Pearson coefficients for the actual precipitation and the RCM simulated precipitation in four major Korean cities. As can be seen in the table, all have a positive correlation of 0.2∼0.6 for each model.Table 2
Pearson coefficient of precipitation between RCMs and observatories.
ClassificationHadGEM3-RARegCMSNU-WRFSNU-MM5YSU-RSMMonthly precipitationSeoul0.530.460.350.320.31Daejeon0.510.480.410.490.60Daegu0.420.520.340.470.56Busan0.410.290.200.280.32Annual precipitationSeoul0.04−0.280.04−0.02−0.26Daejeon0.32−0.49−0.57−0.47−0.36Daegu0.08−0.12−0.180.07−0.11Busan0.23−0.060.150.040.20Seasonal precipitation (Jul.–Sep.)Seoul0.560.030.100.200.28Daejeon0.470.200.160.250.45Daegu0.400.290.350.340.35Busan0.490.330.440.390.31Seasonal precipitation (Oct.–Jun.)Seoul0.320.220.11−0.03−0.08Daejeon0.330.170.190.230.45Daegu0.240.20−0.020.170.48Busan0.31−0.02−0.13−0.080.12In particular, the Busan area has a low Pearson coefficient except for the HadGEM3-RA model. Except for the HadGEM3-RA model, in Busan, it is found that the correlation with observed precipitation is low. Correlation between the three cities also varies depending on the model used. Of the five RCMs, the HadGEM3-RA model generally shows a high correlation, while the YSU-RSM shows a high correlation in Daejeon and Daegu.


## 4. ANN Ensemble Analysis

### 4.1. Simulation Model Construction

ANN simulation cases were selected to produce ensemble precipitation forecasting using five RCMs in four major cities in Korea. Observed precipitation was used as a dependent variable while 5 RCM precipitation data were used as independent variables.In constructing an ANN simulation, 80% of total data are trained and 20% verified to simulate the constructed neural network. The hyperbolic tangent function is used for the activation function and the conjugate gradient method is used as the optimization algorithm. The conjugate gradient method is an algorithm for finding the nearest local minimum of a function ofn variables which presupposes that the gradient of the function can be computed [55]. The minimum variation of the learning error is 0.0001 in the multilayer perceptron of ANN, and the minimum relative variation of the learning error is 0.001. The results are repeatedly calculated until the error is minimized.ANN simulation cases were selected according to the coefficient of determination calculated in Table2, using five RCMs. The RCM independent variables were removed one by one in descending order of the coefficient of determination. Five cases were then selected as shown in Table 3 for each city.Table 3
ANN simulation cases.
LocationRCMsNumber of neuronsModeling caseSeoulHadGEM3-RA, RegCM4, SNU-WRF, SNU-MM5, and YSU-RSM10Case 1-1HadGEM3-RA, RegCM4, SNU-WRF, and SNU-MM58Case 1-2HadGEM3-RA, RegCM4, and SNU-WRF6Case 1-3HadGEM3-RA and RegCM44Case 1-4HadGEM3-RA2Case 1-5DaejeonHadGEM3-RA, RegCM4, SNU-WRF, SNU-MM5, and YSU-RSM10Case 2-1HadGEM3-RA, RegCM4, SNU-WRF, and YSU-RSM8Case 2-2HadGEM3-RA, SNU-WRF, and YSU-RSM6Case 2-3HadGEM3-RA and YSU-RSM4Case 2-4YSU-RSM2Case 2-5DaeguHadGEM3-RA, RegCM4, SNU-WRF, SNU-MM5, and YSU-RSM10Case 3-1HadGEM3-RA, RegCM4, SNU-MM5, and YSU-RSM8Case 3-2RegCM4, SNU-MM5, YSU-RSM6Case 3-3RegCM4, and YSU-RSM4Case 3-4YSU-RSM2Case 3-5BusanHadGEM3-RA, RegCM4, SNU-WRF, SNU-MM5, and YSU-RSM10Case 4-1HadGEM3-RA, SNU-WRF, SNU-MM5, and YSU-RSM8Case 4-2HadGEM3-RA, SNU-WRF, and YSU-RSM6Case 4-3HadGEM3-RA and SNU-WRF4Case 4-4HadGEM3-RA2Case 4-5The number of neurons in the ANN model was set to twice the independent variable, that is, the optimal stimulation condition, while the number of independent variables was taken into account [44, 45].


### 4.2. ANN Ensemble Results Using RCMs

#### 4.2.1. Accuracy Assessment Results

Using ANN, the most reliable case for estimating the precipitation of 4 major cities was selected. Accuracy assessment was determined by using the MAE, MSE, BIAS, and G value (goodness of prediction). Figure4 shows the accuracy assessment results for all simulation cases.Where the calculated value of the MAE and MSE is small and the G value close to 100, the simulated value is therefore corrected to the measured value.Figure 4
Results of accuracy assessment between ANN simulation and RCMs: (a) MAE, (b) MSE, (c) G value, and (d) BIAS.
(a)(b)(c)(d)The results of the ANN ensemble are more accurate than the results of RCM estimated precipitation. In all cities, the precipitation calculated by ANN is lower in the MAE and MSE values than the RCM results.Additionally, since ANN simulation results show that the G value is closer to 100 than the conventional RCM, the ANN can be considered to be effective in precipitation prediction.However, it is difficult to use as an estimation model if the G value is negative. All ANN simulation results show positive values, while some RCMs show negative values.In the case of BIAS, RCMs demonstrate bias according to location and model, whereas the ANN ensemble model is relatively unbiased. Detailed results of the ANN ensemble accuracy assessment are shown in Table4. Pearson co. (avr.) is averaged PCC for selected RCMs.Table 4
Accuracy assessment results by ANN simulation.
LocationSelected RCMsCasesMAEMSEBIASGPearson co. (ANN)Pearson co. (avr.)Optimal modelSeoulHadGEM3-RA, RegCM4, SNU-WRF, SNU-MM5, and YSU-RSMCase 1-181.722,37311.235.80.600.36HadGEM3-RA, RegCM4, SNU-WRF, and SNU-MM5Case 1-270.117,288−0.150.40.720.35OHadGEM3-RA, RegCM4, and SNU-WRFCase 1-379.521,2901.138.90.620.38HadGEM3-RA and RegCM4Case 1-476.126,45634.524.10.570.46HadGEM3-RACase 1-585.224,65613.829.30.550.56DaejeonHadGEM3-RA, RegCM4, SNU-WRF, SNU-MM5, and YSU-RSMCase 2-157.28,0486.345.50.680.39HadGEM3-RA, RegCM4, SNU-WRF, and YSU-RSMCase 2-258.77,871−2.646.70.690.41OHadGEM3-RA, SNU-WRF, and YSU-RSMCase 2-361.98,579−2.441.90.650.44HadGEM3-RA and YSU-RSMCase 2-460.78,172−1.944.70.670.51YSU-RSMCase 2-563.09,1601.238.00.620.45DaeguHadGEM3-RA, RegCM4, SNU-WRF, SNU-MM5, and YSU-RSMCase 3-152.05,5270.946.80.690.41OHadGEM3-RA, RegCM4, SNU-MM5, and YSU-RSMCase 3-253.46,845−2.134.20.590.41RegCM4, SNU-MM5, and YSU-RSMCase 3-352.26,9334.633.30.580.37RegCM4 and YSU-RSMCase 3-453.56,9481.133.20.580.36YSU-RSMCase 3-556.67,1502.931.20.560.35BusanHadGEM3-RA, RegCM4, SNU-WRF, SNU-MM5, and YSU-RSMCase 4-166.412,0386.829.30.540.48HadGEM3-RA, SNU-WRF, SNU-MM5, and YSU-RSMCase 4-265.410,3607.839.10.630.49OHadGEM3-RA, SNU-WRF, and YSU-RSMCase 4-364.710,4463.538.60.630.49HadGEM3-RA and SNU-WRFCase 4-469.912,197−5.028.30.530.51HadGEM3-RACase 4-571.213,7010.819.50.440.49As shown in Figure4, HadGEM3-RA for Seoul and Busan, YSU-RSM for Daejeon, and RegCM4 for Daegu were selected as the most accurate RCMs. Therefore, various RCMs are required for accurate model selection in each region. Pearson coefficient by ANN is higher than that by averaged precipitation by RCMs in all cases.In Table4, the most accurate ANN simulation cases were selected by analyzing MAE, MSE, and G values, which are the comparison indices of measured and predicted values. When analyzing ANN using 4 or 5 RCMs, results using ANN showed the highest accuracy.In the case of Seoul, accuracy was high except when using the YSU-RSM model, which had the lowest Pearson coefficient of 0.31 in Table2. In the case of Daejeon, accuracy was high except when using the SNU-MM5 model, and in the case of Busan, accuracy was high except when using the RegCM4 model which had the lowest Pearson coefficient. In the case of Daegu, cases using all RCMs were the most accurate.


#### 4.2.2. Graph Analysis

Figure5 shows ANN ensemble simulation results for the optimal cases selected in Table 4. And, Table 5 is a basic statistical analysis for precipitations from Figure 5. ANN ensemble was determined through the results such as MAE, MSE, and G values. It was compared with a case where the correlation was high among the 5 RCMs.Figure 5
Precipitation comparison between ANN simulation and RCM with observed value: (a) Seoul, (b) Daejeon, (c) Daegu, and (d) Busan.
(a)(b)(c)(d)Table 5
Basic precipitation comparison (unit: mm).
LocationSelected RCMsMax.Min.Ave.SeoulObservation1,131.00.8121.2HadGEM3-RA457.12.385.7ANN ensemble530.80.0121.3DaejeonObservation587.32.5107.6HadGEM3-RA527.52.978.3ANN ensemble391.630.6110.1DaeguObservation568.1087.5HadGEM3-RA359.80.481.7ANN ensemble379.87.086.6BusanObservation886.10123.9HadGEM3-RA452.51.697.8ANN ensemble316.910.4116.1In Seoul, Case 1-2 is the most accurate. It uses 4 RCMs, not including the YSU-RSM model. Certain monthly estimated precipitation in Seoul is higher than 1,000 mm. The amount of precipitation corresponding to annual rainfall is simulated. The HadGEM3-RA had the highest accuracy among the five RCMs used. However, it was not reliable in predicting extreme precipitation. Instead, predictions of monthly rainfall patterns proved better than in other cities.As a result of the ANN ensemble, there was a pattern similar to the HadGEM3-RA precipitation model which was the optimal RCM for Seoul. In the case of Daejeon and Daegu, the YSU-RSM model achieved the results closest to actually observed precipitation. In certain periods, precipitation by RCM was overestimated. The model selected through the ANN ensemble was adjusted to be close to the observed precipitation.In Busan, the ANN ensemble shows a similar pattern to the optimal RCM. However, it was overestimated in certain periods. The average monthly precipitation in Busan is 124 mm, and the average monthly precipitation by HadGEM3-RA is 98 mm, which is underestimated compared with observed data in Table5. The ANN ensemble is 116 mm, which is closer to the observed rainfall than HadGEM3-RA. However, HadGEM3-RA was 452 mm and ANN ensemble was 316 mm compared with the maximum monthly rainfall of 886 mm. Even during periods when no precipitation was observed, a certain amount of precipitation was predicted in the model.The results showed that the ANN ensemble predicted precipitation results closer to the observed precipitation than the single used RCM. In particular, the ANN ensemble produced more reliable patterns than overestimating RCM results in inland cities such as Daejeon and Daegu.


## 4.1. Simulation Model Construction

ANN simulation cases were selected to produce ensemble precipitation forecasting using five RCMs in four major cities in Korea. Observed precipitation was used as a dependent variable while 5 RCM precipitation data were used as independent variables.In constructing an ANN simulation, 80% of total data are trained and 20% verified to simulate the constructed neural network. The hyperbolic tangent function is used for the activation function and the conjugate gradient method is used as the optimization algorithm. The conjugate gradient method is an algorithm for finding the nearest local minimum of a function ofn variables which presupposes that the gradient of the function can be computed [55]. The minimum variation of the learning error is 0.0001 in the multilayer perceptron of ANN, and the minimum relative variation of the learning error is 0.001. The results are repeatedly calculated until the error is minimized.ANN simulation cases were selected according to the coefficient of determination calculated in Table2, using five RCMs. The RCM independent variables were removed one by one in descending order of the coefficient of determination. Five cases were then selected as shown in Table 3 for each city.Table 3
ANN simulation cases.
LocationRCMsNumber of neuronsModeling caseSeoulHadGEM3-RA, RegCM4, SNU-WRF, SNU-MM5, and YSU-RSM10Case 1-1HadGEM3-RA, RegCM4, SNU-WRF, and SNU-MM58Case 1-2HadGEM3-RA, RegCM4, and SNU-WRF6Case 1-3HadGEM3-RA and RegCM44Case 1-4HadGEM3-RA2Case 1-5DaejeonHadGEM3-RA, RegCM4, SNU-WRF, SNU-MM5, and YSU-RSM10Case 2-1HadGEM3-RA, RegCM4, SNU-WRF, and YSU-RSM8Case 2-2HadGEM3-RA, SNU-WRF, and YSU-RSM6Case 2-3HadGEM3-RA and YSU-RSM4Case 2-4YSU-RSM2Case 2-5DaeguHadGEM3-RA, RegCM4, SNU-WRF, SNU-MM5, and YSU-RSM10Case 3-1HadGEM3-RA, RegCM4, SNU-MM5, and YSU-RSM8Case 3-2RegCM4, SNU-MM5, YSU-RSM6Case 3-3RegCM4, and YSU-RSM4Case 3-4YSU-RSM2Case 3-5BusanHadGEM3-RA, RegCM4, SNU-WRF, SNU-MM5, and YSU-RSM10Case 4-1HadGEM3-RA, SNU-WRF, SNU-MM5, and YSU-RSM8Case 4-2HadGEM3-RA, SNU-WRF, and YSU-RSM6Case 4-3HadGEM3-RA and SNU-WRF4Case 4-4HadGEM3-RA2Case 4-5The number of neurons in the ANN model was set to twice the independent variable, that is, the optimal stimulation condition, while the number of independent variables was taken into account [44, 45].


## 4.2. ANN Ensemble Results Using RCMs

### 4.2.1. Accuracy Assessment Results

Using ANN, the most reliable case for estimating the precipitation of 4 major cities was selected. Accuracy assessment was determined by using the MAE, MSE, BIAS, and G value (goodness of prediction). Figure4 shows the accuracy assessment results for all simulation cases.Where the calculated value of the MAE and MSE is small and the G value close to 100, the simulated value is therefore corrected to the measured value.Figure 4
Results of accuracy assessment between ANN simulation and RCMs: (a) MAE, (b) MSE, (c) G value, and (d) BIAS.
(a)(b)(c)(d)The results of the ANN ensemble are more accurate than the results of RCM estimated precipitation. In all cities, the precipitation calculated by ANN is lower in the MAE and MSE values than the RCM results.Additionally, since ANN simulation results show that the G value is closer to 100 than the conventional RCM, the ANN can be considered to be effective in precipitation prediction.However, it is difficult to use as an estimation model if the G value is negative. All ANN simulation results show positive values, while some RCMs show negative values.In the case of BIAS, RCMs demonstrate bias according to location and model, whereas the ANN ensemble model is relatively unbiased. Detailed results of the ANN ensemble accuracy assessment are shown in Table4. Pearson co. (avr.) is averaged PCC for selected RCMs.Table 4
Accuracy assessment results by ANN simulation.
LocationSelected RCMsCasesMAEMSEBIASGPearson co. (ANN)Pearson co. (avr.)Optimal modelSeoulHadGEM3-RA, RegCM4, SNU-WRF, SNU-MM5, and YSU-RSMCase 1-181.722,37311.235.80.600.36HadGEM3-RA, RegCM4, SNU-WRF, and SNU-MM5Case 1-270.117,288−0.150.40.720.35OHadGEM3-RA, RegCM4, and SNU-WRFCase 1-379.521,2901.138.90.620.38HadGEM3-RA and RegCM4Case 1-476.126,45634.524.10.570.46HadGEM3-RACase 1-585.224,65613.829.30.550.56DaejeonHadGEM3-RA, RegCM4, SNU-WRF, SNU-MM5, and YSU-RSMCase 2-157.28,0486.345.50.680.39HadGEM3-RA, RegCM4, SNU-WRF, and YSU-RSMCase 2-258.77,871−2.646.70.690.41OHadGEM3-RA, SNU-WRF, and YSU-RSMCase 2-361.98,579−2.441.90.650.44HadGEM3-RA and YSU-RSMCase 2-460.78,172−1.944.70.670.51YSU-RSMCase 2-563.09,1601.238.00.620.45DaeguHadGEM3-RA, RegCM4, SNU-WRF, SNU-MM5, and YSU-RSMCase 3-152.05,5270.946.80.690.41OHadGEM3-RA, RegCM4, SNU-MM5, and YSU-RSMCase 3-253.46,845−2.134.20.590.41RegCM4, SNU-MM5, and YSU-RSMCase 3-352.26,9334.633.30.580.37RegCM4 and YSU-RSMCase 3-453.56,9481.133.20.580.36YSU-RSMCase 3-556.67,1502.931.20.560.35BusanHadGEM3-RA, RegCM4, SNU-WRF, SNU-MM5, and YSU-RSMCase 4-166.412,0386.829.30.540.48HadGEM3-RA, SNU-WRF, SNU-MM5, and YSU-RSMCase 4-265.410,3607.839.10.630.49OHadGEM3-RA, SNU-WRF, and YSU-RSMCase 4-364.710,4463.538.60.630.49HadGEM3-RA and SNU-WRFCase 4-469.912,197−5.028.30.530.51HadGEM3-RACase 4-571.213,7010.819.50.440.49As shown in Figure4, HadGEM3-RA for Seoul and Busan, YSU-RSM for Daejeon, and RegCM4 for Daegu were selected as the most accurate RCMs. Therefore, various RCMs are required for accurate model selection in each region. Pearson coefficient by ANN is higher than that by averaged precipitation by RCMs in all cases.In Table4, the most accurate ANN simulation cases were selected by analyzing MAE, MSE, and G values, which are the comparison indices of measured and predicted values. When analyzing ANN using 4 or 5 RCMs, results using ANN showed the highest accuracy.In the case of Seoul, accuracy was high except when using the YSU-RSM model, which had the lowest Pearson coefficient of 0.31 in Table2. In the case of Daejeon, accuracy was high except when using the SNU-MM5 model, and in the case of Busan, accuracy was high except when using the RegCM4 model which had the lowest Pearson coefficient. In the case of Daegu, cases using all RCMs were the most accurate.


### 4.2.2. Graph Analysis

Figure5 shows ANN ensemble simulation results for the optimal cases selected in Table 4. And, Table 5 is a basic statistical analysis for precipitations from Figure 5. ANN ensemble was determined through the results such as MAE, MSE, and G values. It was compared with a case where the correlation was high among the 5 RCMs.Figure 5
Precipitation comparison between ANN simulation and RCM with observed value: (a) Seoul, (b) Daejeon, (c) Daegu, and (d) Busan.
(a)(b)(c)(d)Table 5
Basic precipitation comparison (unit: mm).
LocationSelected RCMsMax.Min.Ave.SeoulObservation1,131.00.8121.2HadGEM3-RA457.12.385.7ANN ensemble530.80.0121.3DaejeonObservation587.32.5107.6HadGEM3-RA527.52.978.3ANN ensemble391.630.6110.1DaeguObservation568.1087.5HadGEM3-RA359.80.481.7ANN ensemble379.87.086.6BusanObservation886.10123.9HadGEM3-RA452.51.697.8ANN ensemble316.910.4116.1In Seoul, Case 1-2 is the most accurate. It uses 4 RCMs, not including the YSU-RSM model. Certain monthly estimated precipitation in Seoul is higher than 1,000 mm. The amount of precipitation corresponding to annual rainfall is simulated. The HadGEM3-RA had the highest accuracy among the five RCMs used. However, it was not reliable in predicting extreme precipitation. Instead, predictions of monthly rainfall patterns proved better than in other cities.As a result of the ANN ensemble, there was a pattern similar to the HadGEM3-RA precipitation model which was the optimal RCM for Seoul. In the case of Daejeon and Daegu, the YSU-RSM model achieved the results closest to actually observed precipitation. In certain periods, precipitation by RCM was overestimated. The model selected through the ANN ensemble was adjusted to be close to the observed precipitation.In Busan, the ANN ensemble shows a similar pattern to the optimal RCM. However, it was overestimated in certain periods. The average monthly precipitation in Busan is 124 mm, and the average monthly precipitation by HadGEM3-RA is 98 mm, which is underestimated compared with observed data in Table5. The ANN ensemble is 116 mm, which is closer to the observed rainfall than HadGEM3-RA. However, HadGEM3-RA was 452 mm and ANN ensemble was 316 mm compared with the maximum monthly rainfall of 886 mm. Even during periods when no precipitation was observed, a certain amount of precipitation was predicted in the model.The results showed that the ANN ensemble predicted precipitation results closer to the observed precipitation than the single used RCM. In particular, the ANN ensemble produced more reliable patterns than overestimating RCM results in inland cities such as Daejeon and Daegu.


## 4.2.1. Accuracy Assessment Results

Using ANN, the most reliable case for estimating the precipitation of 4 major cities was selected. Accuracy assessment was determined by using the MAE, MSE, BIAS, and G value (goodness of prediction). Figure4 shows the accuracy assessment results for all simulation cases.Where the calculated value of the MAE and MSE is small and the G value close to 100, the simulated value is therefore corrected to the measured value.Figure 4
Results of accuracy assessment between ANN simulation and RCMs: (a) MAE, (b) MSE, (c) G value, and (d) BIAS.
(a)(b)(c)(d)The results of the ANN ensemble are more accurate than the results of RCM estimated precipitation. In all cities, the precipitation calculated by ANN is lower in the MAE and MSE values than the RCM results.Additionally, since ANN simulation results show that the G value is closer to 100 than the conventional RCM, the ANN can be considered to be effective in precipitation prediction.However, it is difficult to use as an estimation model if the G value is negative. All ANN simulation results show positive values, while some RCMs show negative values.In the case of BIAS, RCMs demonstrate bias according to location and model, whereas the ANN ensemble model is relatively unbiased. Detailed results of the ANN ensemble accuracy assessment are shown in Table4. Pearson co. (avr.) is averaged PCC for selected RCMs.Table 4
Accuracy assessment results by ANN simulation.
LocationSelected RCMsCasesMAEMSEBIASGPearson co. (ANN)Pearson co. (avr.)Optimal modelSeoulHadGEM3-RA, RegCM4, SNU-WRF, SNU-MM5, and YSU-RSMCase 1-181.722,37311.235.80.600.36HadGEM3-RA, RegCM4, SNU-WRF, and SNU-MM5Case 1-270.117,288−0.150.40.720.35OHadGEM3-RA, RegCM4, and SNU-WRFCase 1-379.521,2901.138.90.620.38HadGEM3-RA and RegCM4Case 1-476.126,45634.524.10.570.46HadGEM3-RACase 1-585.224,65613.829.30.550.56DaejeonHadGEM3-RA, RegCM4, SNU-WRF, SNU-MM5, and YSU-RSMCase 2-157.28,0486.345.50.680.39HadGEM3-RA, RegCM4, SNU-WRF, and YSU-RSMCase 2-258.77,871−2.646.70.690.41OHadGEM3-RA, SNU-WRF, and YSU-RSMCase 2-361.98,579−2.441.90.650.44HadGEM3-RA and YSU-RSMCase 2-460.78,172−1.944.70.670.51YSU-RSMCase 2-563.09,1601.238.00.620.45DaeguHadGEM3-RA, RegCM4, SNU-WRF, SNU-MM5, and YSU-RSMCase 3-152.05,5270.946.80.690.41OHadGEM3-RA, RegCM4, SNU-MM5, and YSU-RSMCase 3-253.46,845−2.134.20.590.41RegCM4, SNU-MM5, and YSU-RSMCase 3-352.26,9334.633.30.580.37RegCM4 and YSU-RSMCase 3-453.56,9481.133.20.580.36YSU-RSMCase 3-556.67,1502.931.20.560.35BusanHadGEM3-RA, RegCM4, SNU-WRF, SNU-MM5, and YSU-RSMCase 4-166.412,0386.829.30.540.48HadGEM3-RA, SNU-WRF, SNU-MM5, and YSU-RSMCase 4-265.410,3607.839.10.630.49OHadGEM3-RA, SNU-WRF, and YSU-RSMCase 4-364.710,4463.538.60.630.49HadGEM3-RA and SNU-WRFCase 4-469.912,197−5.028.30.530.51HadGEM3-RACase 4-571.213,7010.819.50.440.49As shown in Figure4, HadGEM3-RA for Seoul and Busan, YSU-RSM for Daejeon, and RegCM4 for Daegu were selected as the most accurate RCMs. Therefore, various RCMs are required for accurate model selection in each region. Pearson coefficient by ANN is higher than that by averaged precipitation by RCMs in all cases.In Table4, the most accurate ANN simulation cases were selected by analyzing MAE, MSE, and G values, which are the comparison indices of measured and predicted values. When analyzing ANN using 4 or 5 RCMs, results using ANN showed the highest accuracy.In the case of Seoul, accuracy was high except when using the YSU-RSM model, which had the lowest Pearson coefficient of 0.31 in Table2. In the case of Daejeon, accuracy was high except when using the SNU-MM5 model, and in the case of Busan, accuracy was high except when using the RegCM4 model which had the lowest Pearson coefficient. In the case of Daegu, cases using all RCMs were the most accurate.


## 4.2.2. Graph Analysis

Figure5 shows ANN ensemble simulation results for the optimal cases selected in Table 4. And, Table 5 is a basic statistical analysis for precipitations from Figure 5. ANN ensemble was determined through the results such as MAE, MSE, and G values. It was compared with a case where the correlation was high among the 5 RCMs.Figure 5
Precipitation comparison between ANN simulation and RCM with observed value: (a) Seoul, (b) Daejeon, (c) Daegu, and (d) Busan.
(a)(b)(c)(d)Table 5
Basic precipitation comparison (unit: mm).
LocationSelected RCMsMax.Min.Ave.SeoulObservation1,131.00.8121.2HadGEM3-RA457.12.385.7ANN ensemble530.80.0121.3DaejeonObservation587.32.5107.6HadGEM3-RA527.52.978.3ANN ensemble391.630.6110.1DaeguObservation568.1087.5HadGEM3-RA359.80.481.7ANN ensemble379.87.086.6BusanObservation886.10123.9HadGEM3-RA452.51.697.8ANN ensemble316.910.4116.1In Seoul, Case 1-2 is the most accurate. It uses 4 RCMs, not including the YSU-RSM model. Certain monthly estimated precipitation in Seoul is higher than 1,000 mm. The amount of precipitation corresponding to annual rainfall is simulated. The HadGEM3-RA had the highest accuracy among the five RCMs used. However, it was not reliable in predicting extreme precipitation. Instead, predictions of monthly rainfall patterns proved better than in other cities.As a result of the ANN ensemble, there was a pattern similar to the HadGEM3-RA precipitation model which was the optimal RCM for Seoul. In the case of Daejeon and Daegu, the YSU-RSM model achieved the results closest to actually observed precipitation. In certain periods, precipitation by RCM was overestimated. The model selected through the ANN ensemble was adjusted to be close to the observed precipitation.In Busan, the ANN ensemble shows a similar pattern to the optimal RCM. However, it was overestimated in certain periods. The average monthly precipitation in Busan is 124 mm, and the average monthly precipitation by HadGEM3-RA is 98 mm, which is underestimated compared with observed data in Table5. The ANN ensemble is 116 mm, which is closer to the observed rainfall than HadGEM3-RA. However, HadGEM3-RA was 452 mm and ANN ensemble was 316 mm compared with the maximum monthly rainfall of 886 mm. Even during periods when no precipitation was observed, a certain amount of precipitation was predicted in the model.The results showed that the ANN ensemble predicted precipitation results closer to the observed precipitation than the single used RCM. In particular, the ANN ensemble produced more reliable patterns than overestimating RCM results in inland cities such as Daejeon and Daegu.


## 5. Comparison between ANN and Multiple Regression Analysis

In Section5, the multiple regression analysis (MRA) was calculated and compared with the ANN results. MRA is one of the most widely used statistical techniques. The ANN simulation results of target cities were compared using the various statistical analysis methods.


### 5.1. Generation of Equation by Multiple Regression Analysis

Multiple regression analysis (MRA) is used for identifying relationships between variables with high correlations and predict the value of variables. In the regression model, one independent variable is called simple regression analysis, and one independent variable does not sufficiently explain the dependent variable.The multiple linear regression model with independent variables is expressed as [56, 57](7)y=β0+β1x1+β2x2+…+βkxk,where xii=1,…,k is the independent variable (precipitation by RCMs), y is the dependent variable (observed precipitation), βii=1,…,k is the regression coefficient, and β0 is the intercept of y.MRA was performed on selected appropriate ANN simulation cases of each city. Selected cases are Case 1-2, Case 2-2, Case 3-1, and Case 4-2. Using the 4 cases, generated MRA equations are shown in equations (8)–(11). The precipitation unit is mm per month for 4 cities.(8)Prec.Seoul=−30.6+0.94HadGEM+1.26RegCM4+0.26SNUWRF−0.54SNUMM5,(9)Prec.Daejeon=11.2+0.39HadGEM+0.44RegCM4−0.27SNUWRF+0.50YSURSM,(10)Prec.Daegu=5.24+0.23HadGEM+0.65RegCM4−0.41SNUWRF+0.002SNUMM5+0.36YSURSM,(11)Prec.Busan=50.3+0.53HadGEM−0.21SNUWRF+0.21SNUMM5+0.17YSURSM.


### 5.2. Comparison between ANN and Multiple Regression Analysis

Multiple regression equations from (8) to (11) were used to generate precipitation; then these results were compared with ANN’s precipitation as shown in Table 6 through the accuracy assessment indices.Table 6
Accuracy assessment results between ANN and MRA.
ClassificationSeoulDaejeonDaeguBusanMRAANNMRAANNMRAANNMRAANNBIAS0.0−0.10.0−2.60.00.90.06.8MAE79.570.158.558.753.252.071.766.4MSE21,86317,2888165.67,8715,9715,52713,52812,038G37.350.444.746.742.646.820.529.3Pearson co.0.610.720.670.690.700.690.500.54Optimal modelOOOOAs shown in Table6, the prediction estimation accuracy of ANN was higher than MRA in all analyzed cities. MAE and MSE values are low and the G value is close to 100, which is the result of judging that the estimation accuracy is high.In the case of MRA, the BIAS value is 0 in all cities, and the estimated value is not biased compared to ANN. As a result of comparing ANN and MRA with MAE, MSE, and G values, ANN ensembles in all cities showed appropriate results to observations in all cities.Figure6 is a graph comparing ANN and MRA to actual precipitation. Estimated precipitation was underestimated in all four cities. Overall ANN and MRA precipitation distributions were similar. The results show that the accuracy of precipitation estimation using ANN and MRA is higher than that of RCM’s precipitation, but the accuracy is still lower than the observed precipitation in all analyzed cities.Figure 6
Precipitation comparison between ANN and MRA with observed value: (a) Seoul, (b) Daejeon, (c) Daegu, and (d) Busan.
(a)(b)(c)(d)Table7 shows the results of basic statistical analysis for observation, ANN, and MRA precipitations. The mean precipitation values of the estimated by ANN and MRA are almost similar to the observed value. Particularly, MRA is more similar to the average of the observed values than ANN.Table 7
Basic statistical analysis between ANN and MRA.
ClassificationSeoulDaejeonDaeguBusanObs.ANNMRAObs.ANNMRAObs.ANNMRAObs.ANNMRAAvr. prep.121.2121.3121.2107.6110.1107.687.586.687.5123.9116.1123.9Min. prep.0.8−1.5−9.72.530.611.60.07.0−5.40.010.443.9Max. prep.1,131530.8556.3587.3391.6366.8568.1379.8363.9886.1316.9368.8SD179.0191.2143.4147.1113.5110.7148.4170.0In the case of minimum precipitation, there was no monthly precipitation in case of observed value, but precipitation by RCM did not predict monthly precipitation well and it is difficult to judge which method of ANN and RMA is suitable. In the case of maximum precipitation, the observed precipitation tended to be higher in each city, and ANN and MRA did not show a similar range of precipitation.Since the predicted precipitation by ANN and MRA follows the results of RCMs, precipitation by reliable RCM should be prioritized. In particular, ANN ensembles did not estimate extreme precipitation appropriately because RCM precipitation did not predict actual extreme precipitation well.According to previous studies, the uncertainty of precipitation occurring in RCM in East Asia is 1 to 1.5 mm/day [58]. In case of monthly conversion, a difference of less than 50 mm may occur, and the monthly precipitation of this study exceeds the range. In four cities, monthly precipitation amounting to 50% of the annual average precipitation was observed and it is a reason for the bigger difference between observed and estimated precipitation.


## 5.1. Generation of Equation by Multiple Regression Analysis

Multiple regression analysis (MRA) is used for identifying relationships between variables with high correlations and predict the value of variables. In the regression model, one independent variable is called simple regression analysis, and one independent variable does not sufficiently explain the dependent variable.The multiple linear regression model with independent variables is expressed as [56, 57](7)y=β0+β1x1+β2x2+…+βkxk,where xii=1,…,k is the independent variable (precipitation by RCMs), y is the dependent variable (observed precipitation), βii=1,…,k is the regression coefficient, and β0 is the intercept of y.MRA was performed on selected appropriate ANN simulation cases of each city. Selected cases are Case 1-2, Case 2-2, Case 3-1, and Case 4-2. Using the 4 cases, generated MRA equations are shown in equations (8)–(11). The precipitation unit is mm per month for 4 cities.(8)Prec.Seoul=−30.6+0.94HadGEM+1.26RegCM4+0.26SNUWRF−0.54SNUMM5,(9)Prec.Daejeon=11.2+0.39HadGEM+0.44RegCM4−0.27SNUWRF+0.50YSURSM,(10)Prec.Daegu=5.24+0.23HadGEM+0.65RegCM4−0.41SNUWRF+0.002SNUMM5+0.36YSURSM,(11)Prec.Busan=50.3+0.53HadGEM−0.21SNUWRF+0.21SNUMM5+0.17YSURSM.


## 5.2. Comparison between ANN and Multiple Regression Analysis

Multiple regression equations from (8) to (11) were used to generate precipitation; then these results were compared with ANN’s precipitation as shown in Table 6 through the accuracy assessment indices.Table 6
Accuracy assessment results between ANN and MRA.
ClassificationSeoulDaejeonDaeguBusanMRAANNMRAANNMRAANNMRAANNBIAS0.0−0.10.0−2.60.00.90.06.8MAE79.570.158.558.753.252.071.766.4MSE21,86317,2888165.67,8715,9715,52713,52812,038G37.350.444.746.742.646.820.529.3Pearson co.0.610.720.670.690.700.690.500.54Optimal modelOOOOAs shown in Table6, the prediction estimation accuracy of ANN was higher than MRA in all analyzed cities. MAE and MSE values are low and the G value is close to 100, which is the result of judging that the estimation accuracy is high.In the case of MRA, the BIAS value is 0 in all cities, and the estimated value is not biased compared to ANN. As a result of comparing ANN and MRA with MAE, MSE, and G values, ANN ensembles in all cities showed appropriate results to observations in all cities.Figure6 is a graph comparing ANN and MRA to actual precipitation. Estimated precipitation was underestimated in all four cities. Overall ANN and MRA precipitation distributions were similar. The results show that the accuracy of precipitation estimation using ANN and MRA is higher than that of RCM’s precipitation, but the accuracy is still lower than the observed precipitation in all analyzed cities.Figure 6
Precipitation comparison between ANN and MRA with observed value: (a) Seoul, (b) Daejeon, (c) Daegu, and (d) Busan.
(a)(b)(c)(d)Table7 shows the results of basic statistical analysis for observation, ANN, and MRA precipitations. The mean precipitation values of the estimated by ANN and MRA are almost similar to the observed value. Particularly, MRA is more similar to the average of the observed values than ANN.Table 7
Basic statistical analysis between ANN and MRA.
ClassificationSeoulDaejeonDaeguBusanObs.ANNMRAObs.ANNMRAObs.ANNMRAObs.ANNMRAAvr. prep.121.2121.3121.2107.6110.1107.687.586.687.5123.9116.1123.9Min. prep.0.8−1.5−9.72.530.611.60.07.0−5.40.010.443.9Max. prep.1,131530.8556.3587.3391.6366.8568.1379.8363.9886.1316.9368.8SD179.0191.2143.4147.1113.5110.7148.4170.0In the case of minimum precipitation, there was no monthly precipitation in case of observed value, but precipitation by RCM did not predict monthly precipitation well and it is difficult to judge which method of ANN and RMA is suitable. In the case of maximum precipitation, the observed precipitation tended to be higher in each city, and ANN and MRA did not show a similar range of precipitation.Since the predicted precipitation by ANN and MRA follows the results of RCMs, precipitation by reliable RCM should be prioritized. In particular, ANN ensembles did not estimate extreme precipitation appropriately because RCM precipitation did not predict actual extreme precipitation well.According to previous studies, the uncertainty of precipitation occurring in RCM in East Asia is 1 to 1.5 mm/day [58]. In case of monthly conversion, a difference of less than 50 mm may occur, and the monthly precipitation of this study exceeds the range. In four cities, monthly precipitation amounting to 50% of the annual average precipitation was observed and it is a reason for the bigger difference between observed and estimated precipitation.


## 6. Conclusions

This study is an application of ANN in order to improve the accuracy of estimated precipitation according to climate change scenarios in Korea. For this purpose, RCM precipitation data were collected from CORDEX and the Korea Meteorological Administration. ANN was applied to major cities in Korea and the following conclusions were drawn.The five RCMs provided in Korea were compared with the observed precipitation, and the ANN ensemble model was applied to combine the advantages of each model. The results show that using the ANN ensemble simulation estimates observed precipitation appropriately compared to a single RCM. In addition, irregular precipitation patterns in single RCM results have steadily changed by using the ensemble model.In applying ANN to predicting the precipitation of major cities in Korea, it is most accurate to apply 4 or 5 RCMs as independent variables. When ANN is applied, optimal weighting factors should be adjusted for low and high correlation models.In the time series analysis, the ANN ensembles are closer to the observed precipitation pattern than the single used RCM. In particular, it is shown that precipitation pattern forecasting is more effective in inland areas such as Daejeon and Daegu than coastal areas such as Busan, where the precipitation pattern is more irregular.ANN and MRA were compared by the RCM ensemble. As a result, ANN was selected as the accurate model in representative cities in Korea. However, it is necessary to improve the accuracy in quantitatively evaluating the precipitation.An RCM of 50 km resolution was extracted by the spatial interpolation method and used for this study. Estimated precipitation may vary according to the statistical downscaling method using RCM, so that different optimal ANN models may be selected by different downscaling results. A more detailed analysis will be possible if applied to other regions in the future, and the applicability of the ANN model will see growth following the collection of more observation data. To increase model accuracy, M5Tree, MARS, LSSVR, and GEP models can be applied in future research. Also, this study was conducted on four major cities and needs to be supplemented through research in various domestic or overseas regions with different city characteristics in the future.Since it is not possible to predict optimal precipitation using all RCMs, an additional RCM selection methodology is required for ANN application. The study classifies simulation cases using Pearson correlation coefficients, but further studies are needed to select the optimal ANN cases.



---

*Source: 7363471-2021-03-09.xml*