# Methodology for Developing Hydrological Models Based on an Artificial Neural Network to Establish an Early Warning System in Small Catchments

Authors: Ivana Sušanj; Nevenka Ožanić; Ivan Marović
Journal: Advances in Meteorology
 (2016)

---

## Abstract

In some situations, there is no possibility of hazard mitigation, especially if the hazard is induced by water. Thus, it is important to prevent consequences via an early warning system (EWS) to announce the possible occurrence of a hazard. The aim and objective of this paper are to investigate the possibility of implementing an EWS in a small-scale catchment and to develop a methodology for developing a hydrological prediction model based on an artificial neural network (ANN) as an essential part of the EWS. The methodology is implemented in the case study of the Slani Potok catchment, which is historically recognized as a hazard-prone area, by establishing continuous monitoring of meteorological and hydrological parameters to collect data for the training, validation, and evaluation of the prediction capabilities of the ANN model. The model is validated and evaluated by visual and common calculation approaches and a new evaluation for the assessment. This new evaluation is proposed based on the separation of the observed data into classes based on the mean data value and the percentages of classes above or below the mean data value as well as on the performance of the mean absolute error.


---

## Body

## 1. Introduction

Natural events, phenomena that occur in urban areas, with consequences such as loss of human life and/or significant material and infrastructure damage, are considered hazards. The same events in uninhabited areas and areas of no interest to people are not considered disasters, and they are rarely of interest in terms of detailed research and the implementation of hazard mitigation processes, such as early warning systems (EWSs) [1]. In populated areas, it is difficult to separate events as solely natural events in a manner that excludes the impact of human activities. The occurrence of hazard phenomena cannot be prevented by humans, but its consequences can be minimized or even intensified depending on the human activities in the hazard-prone area. Debris flow, expansive soils, landslides, rock falls, drought, erosion, sedimentation, river flooding, flash floods, and mud flows are all considered hazard events.This paper focuses on hazards that are caused by the activity of water, such as flash floods, mud flows, and debris flows.Flash floods can be described as floods caused by a storm event in a short period of time. The term “flash” reflects a fast response, with water levels in the water bed reaching a peak within minutes to a few hours after the onset of the rain event, leaving an extremely short time for warning [2]. Flash floods can also become filled with small particles from terrestrial deposits that were saturated with rain; in that case, they are defined as mud flows [3]. A debris flow is a flow, typically torrential, that is a mixture of mud flows and debris that suddenly comes down the slope, preceded by huge boulders that pose a severe hazard [4].Prediction of flash floods, mud flows, and debris flows, as a part of the EWS in areas where there is no possibility of minimizing human activities or mitigating risk, becomes a crucial tool for preventing the consequences caused by the aforementioned hazards. As a result, there are currently many projects aimed at the development and implementation of EWSs. One such project is the bilateral Croatian-Japanese project “Risk Identification and Land-Use Planning for Disaster Mitigation of Landslides and Floods in Croatia,” in which Japanese scientists transferred their knowledge of the development of EWSs to Croatian researchers because EWSs are still in the development stage in the Republic of Croatia.As the aforementioned hazards are initiated by many natural and anthropogenic factors, which can become triggering factors when combined, it is critical to establish the monitoring of areas that are known as existing or potentially hazardous areas. Natural triggering factors can be extreme meteorological events (e.g., rainfall, snow melt, or wind) or hydrogeological conditions, such as high water levels and poor soil.According to the United Nations International Strategy for Disaster Reduction (UN/ISDR, 2009), a complete and effective EWS includes four related elements: (i) risk knowledge, (ii) a monitoring and warning system service, (iii) dissemination and communication, and (iv) response capability. The hazard prediction model is developed under the monitoring and warning system service. It requires a number of technologies and areas of expertise that consist of several elements, such as long-term monitoring and collection of existing data on the potential hazard area, real-time and remote monitoring of triggering factors, data analysis, development, validation, and evaluation of the predictive hydrological model, and development of a decision support system that will assist public authorities and citizens in choosing the appropriate protection measures [5].In the last few decades, predictive hydrological models for establishing EWSs have been developed with the growth of computational capabilities. Most of the prediction models are formed as rainfall-runoff models that can be assigned to one of three broad categories: (i) deterministic (physical), (ii) conceptual, or (iii) parametric (also known as analytic or empirical). Deterministic models use physical laws of mass and energy transfer to describe rainfall-runoff processes, whereas conceptual models use perceived systems to simplify the processes, and parametric models use mathematical transfer functions to connect meteorological parameters to runoff. Hydrological models can also be classified as lumped, which means that the model treats a catchment as a single unit or as distributed, where the catchment is divided into connected subsystems [6].Hydrological prediction models are typically extremely complex, which inhibits their widespread implementation. Furthermore, there is a lack of objectivity and consistency in the way that models are assessed, evaluated, and compared [7]. The models are typically prepared for specific large catchments, and they cannot be used anywhere else. Such models cannot be applied to small catchments, whose resolution and time of prediction are more sensitive.Therefore, in this paper, the methodology for developing data-driven predictive models, as well as its application and predictive ability as a function of the time step, is based on an artificial neural network (ANN) and is developed for small catchments (less than 5 km2) as a basis for the establishment of an EWS.An ANN can be classified as a parametric model that is generally lumped because rainfall-runoff processes are treated as a “black box” with inputs and outputs [6, 8, 9]. Additionally, ANNs are often less expensive and simpler to implement than other types of models [6, 8].Recently, many studies have been conducted with the aim of predicting hydrogeological parameters with the help of an ANN, such as river discharge [6, 10, 11], flood prediction [12], pore-water pressure [13], lake water levels [14], ground water levels [15], water resources prediction [16], peak flow estimates [17], evaporation estimation [18], river water temperature [19], and water quality modelling [20].All of these studies were prepared for large catchments, whereas few studies consider small catchments, perhaps because they do not represent an enormous hazard risk compared to large ones or because it is widely accepted that it is difficult to predict flash floods, mud flows, or debris flows for catchments that are small and have short rainfall response periods [21]. However, although hazards associated with small catchments do not seem intimidating, they still exist and can cause the same hazards as large areas.


## 2. Methodology for the Development of Data-Driven ANN Predictive Models for Small Catchments

There are already many existing guidelines and methodologies for the development of rainfall-runoff data-driven models [22, 23], and all of them are generally based on three main steps: (i) monitoring, (ii) modelling, and (iii) evaluation. Those steps can also be scaled for predictive ANN modelling in small catchments, whose development flowchart is shown in Figure 1.Figure 1
ANN predictive model development flowchart for the small catchments (green direction: forward movement in procedure; red direction: backward movement in procedure).


### 2.1. Monitoring

As shown in Figure1, before monitoring points in the research area are established, it is important to collect all of the available historical data, such as information on constructed hydraulic structures (e.g., river network, river regulation), geology (e.g., soil type, erosion, and landslide-affected areas), land use (e.g., types of vegetation coverage, areas used for agriculture), and anthropology (e.g., urban areas, traffic infrastructures, and illegal waste disposals), as well as historical data (e.g., affected areas in the past, implemented structural and nonstructural measures).After the available data are collected, continuous monitoring of meteorological and hydrological parameters should be set to recognize triggering factors that can lead to the hazard events and to represent the basis of every model. The establishment of monitoring with at least one metrological station and water level monitoring point is highly recommended.Before developing the model, small catchments should be monitored for a sufficiently long period to have a range of several heavy rain events in different periods of the year, with a minimum period of two years. Additionally, the time step of meteorological and hydrological measurements should not be longer than five minutes.


### 2.2. Modelling

After collecting a sufficient amount of measured data, the model development can begin by identifying model inputs and outputs. For the purpose of modelling the small catchment using the ANN model with a small time step, the measured data must be processed to remove data noise and to identify possible systematic errors because they can lead to appreciable model prediction errors. If data processing did not remove all errors, data collection procedure must be verified until the problem is resolved. The entire procedure of data processing is shown in Figure1.The ANN model is chosen to apply the predictive model to the small catchment because it is a fast and efficient model that can rapidly predict hazards caused by the activity of water, thus leaving sufficient time to announce a hazard notification.An ANN is a massively parallel distributed processor that has a natural propensity for storing experiential knowledge and making it available for use. It resembles the brain in two respects: (i) knowledge is acquired by the network through a learning process and (ii) interneuron connection strengths, known as synaptic weights, are used to store the knowledge [24].The main microstructural component of the ANN is the artificial neuron node, whose model is shown in Figure2.Figure 2
Artificial neuron node model (x1,x2,…,xm: input data; w1,w2,…,wm: weight coefficient; vk is the sum of products of the weight coefficients; φ is the activation function of the neuron node; ok is the response of the neuron node in the kth epoch of the calculation) [8].An artificial neuron node can also be defined by the following mathematical expressions:(1)ok=φvk,vk=∑n=0mwk×xkn,where ok is the response of the neuron node in the kth epoch of the calculation, vk is the sum of products of the weight coefficients wk, xk is the input data in the kth epoch of the calculation, and φ is the activation function of the neuron node.As shown in Figure1, ANN implementation consists of (i) selection of the adequate ANN architecture and training algorithm and (ii) ANN training procedure. ANN implementation procedure is shown in Figure 3.Figure 3
ANN implementation procedure.ANN implementation in prediction model according to Figure3 starts with the selection of the ANN mesostructure, which refer to the type of network (architecture) with which the model will be built. This structure can in turn be generally divided into static and dynamic ones; this is followed by the selection of the activation function [8, 9]. The most common types of networks used in the development of rainfall-runoff models are (i) multilayer perceptron (MLP), (ii) radial basis function (RBF), (iii) self-organizing map (SOP), and (iv) support vector machines (SVMs) [9]. The MLP architecture is the best choice for data-driven prediction model development [8]. The MLP architecture can be described as a static feed forward neuron network that consists of a minimum of three layers: (i) input, (ii) hidden, and (iii) output, as shown in Figure 4. Every layer consists of neurons that are connected by activation functions. Activation functions can be (i) linear, (ii) limited linear, (iii) unipolar sigmoid, (iv) bipolar sigmoid, or (v) hyperbolic tangent, among others [9, 12]. Their purpose is to direct data through the layers of the network from the input layer to the output layer. The numbers of neurons in the input and output layers are defined by the number of selected data, whereas the number of neurons in the hidden layer should be optimized to avoid overfitting the model, defined as the loss of predictive ability [9].Figure 4
Multilayer perceptron (MLP) model [8].The MLP architecture was introduced by Werbos in 1974 in his Ph.D. thesis [25]. Its final form was introduced by Rumelhart, Hinton, and Williams in 1986 [26], who also presented applications of the MLP architecture and a description of its success in prediction, classification, and association related to real problems.For the purpose of predictive hydrological ANN model development it is important that input layer consist of the data with minimum ten delay steps and output layer with prediction time step as presented in Figure3. Delay steps can be defined as input data from previous time steps.Because the output data from the network in one epoch of calculation will have errors, which are a function of the target output and model response in the output layer, an algorithm for determining the changeΔwk of the weight coefficient wk is needed. These algorithms are known as training algorithms because they optimize input data in each following epoch, which reduces the error in the output layer with respect to the target output. The optimization of the weight coefficient can be defined as(2)wk+1=wk+Δwk,where wk+1 is the weight coefficient in the k+1th epoch and Δwk is the change determined by the training algorithm.Training algorithms can be divided into three groups: (i) first-order local algorithms (error backpropagation, generalized delta rule), (ii) second-order local algorithms (Newton algorithm, quasi-Newton algorithm, and Levenberg-Marquardt (LM) algorithm), and (iii) global algorithms (genetic algorithm, simulated annealing, and evolutionary programming) [9, 27].The LM algorithm is the fastest and most appropriate for training simpler structures [28] under the MLP architecture, and it was specially developed for the training of ANNs. Because of those characteristics, this algorithm is proposed for the development of data-driven ANN models for small catchments as shown in ANN implementation procedure flowchart (Figure 3).Using the second-order local algorithms, the change measureΔwk is obtained from the squared approximation of the error function, which is represented by the Hessian matrix. Because the Hessian matrix typically cannot be used in ANN training and because it is not in compliance with appropriate conditions and is thus unsolvable, algorithms that avoid solving the Hessian matrix, such as the LM algorithm, are used.The LM algorithm [29], which is a special combination of the Gauss-Newton and error backpropagation algorithms, uses a conjugate gradient method by introducing the Jacobian matrix instead of the Hessian matrix. The change measure Δwk can be defined as(3)Δwk=-JT∗J+μ∗I-1×JT∗e,where J is the Jacobian matrix of the error vector e with respect to the weight coefficients in the kth epoch of the calculation, JT is the transpose of the Jacobian matrix, and μ is a scalar representing the learning rate.The Jacobian matrix of networks errors can be written as(4)J=∂e∂w=∂e1∂w1∂e1∂w2⋯∂e1∂wn∂e2∂w1∂e2∂w2⋯∂e2∂wn⋮⋮⋮∂en∂w1∂en∂w2⋯∂en∂wn,where J is the Jacobian matrix of the network errors, e1,e2,…,en are the errors, and w1,w2,…,wn are the weight coefficients.At the end of every calculating epoch, the sum squared errorE(e) is calculated as follows: (5)Ee=∑k=1nek2=∑k=1ndk-ok2,where ek is the error in kth epoch of the calculation, dk is the target value, and ok is the response model value in kth epoch of the calculation.Depending on the increase or decrease in the sum squared errorE(e), the learning rate scalar μ changes through every epoch of the calculation by dividing or multiplying by a constant factor (e.g., β in the range [0,1]) to control the LM algorithm to be more similar to the Gauss-Newton error backpropagation algorithm and also to increase the training speed. If the sum of squared errors increases, the learning rate scalar μ will be multiplied by a constant amount β, and the LM algorithm will be more similar to the Gauss-Newton algorithm; otherwise, it will be more similar to the backpropagation algorithm.After the architecture of the ANN and the training algorithm are determined, the software should be chosen in order to conduct ANN training process as shown in Figure3. There is a variety of prepared software programs available for ANN modelling, such as Brainmaker Professional, NeuralWorks Professional II/Plus, Explorer from Neural Ware Inc., WEKA, MATLAB Neural Network Toolbox, and Statistica [8, 9]. For the purpose of this study, MATLAB Neural Network Toolbox is proposed because it provides built-in training process that stops when the ANN is adequately trained. ANN model should be trained for every time prediction step and, after training process, validated and evaluated as presented in model development flowchart (Figure 1).


### 2.3. Validation and Evaluation

Assessment of the model during the training period is considered the model validation, and it cannot be used as criteria with which to evaluate the predictive abilities of the ANN model. Validation is defined as an assessment of the errors between the ANN model response and the target training data, and it can be represented by the same measures as the evaluation, the most common being the mean square error (MSE) and the coefficient of determination (r2), which are defined by (6) and (9), respectively.Validation boundary and scale criteria according to validation measures are presented in Table1. If validation process has indicated that the model is “poor” or “very poor,” the model should be improved. Figure 1 shows four possible steps for the model improvement: (i) reduction of the prediction time step, (ii) increase of the data monitoring collection period, (iii) selection of the different ANN architecture and/or training algorithm, or (iv) identification of the error in modelling process. If model improvement did not result in problem solving, then ANN is not appropriate for predictive purposes of small catchments.Table 1
Boundary scale for validation and evaluation criteria.
Validation/evaluation boundary and scale criteria
CE
r
2
-∞, +1
[−1, +1]
Very good
0.75, 1.00
|0.75|, |1|
Good
0.65, 0.75
/
Poor
0.5, 0.65
|0.50|, |0.75|
Very poor
≤0.5
<|0.5|The evaluation of the model, as shown in Figure1, is considered to be an assessment of the predictive ability of the time step of the ANN model. As mentioned before, evaluation of ANN models and of predictive models in general is problematic. There are a large number of evaluative measures that are widely used, and they can be divided into visual and quantitative measures. Visual evaluation measures are considered to be graphical representations of the ANN model response and target data in the form of the graph, which provides insight into errors in the model output. The most commonly used calculation evaluation measures are the MSE, the mean square relative error (MSRE), the Nash-Sutcliffe coefficient (CE), and coefficient of determination (r2) [8, 30]. The MSE and MSRE are measures that indicate error in the units (or squared units) of the model, and CE and r2 describe the degree of collinearity between modelled and measured data [29]. The described measures can be defined by the following equations: (6)MSE=1n∑k=1ndk-ok2,(7)MSRE=1n∑k=1ndk-okdk2,(8)CE=1-∑k=1ndk-ok2∑k=1ndk-d-2,(9)r2=∑k=1ndk-d-ok-o-∑k=1ndk-d-2∑k=1nok-o-22,where n is the number of data points in the input layer, dk is the target value, ok is the model response value in the kth epoch of the calculation, d- is the mean value of the target data, and o- is the mean value of the network response data.These measures provide insight into the global model errors, but it is impossible to determine the distribution of the errors from those measures. Many studies have been published on classification approaches to model evaluation, such as seasonal weather data classification [31], classification of the predictions according to the percentage of observed data, or measurement of the mean absolute error (MAE) and root mean squared error (RMSE) for all predicted peak flood events in a data set [32]. Thus, for ANN model evaluation in small catchments, the classification of the errors is proposed in this paper. This evaluation consists of separating the data into evaluation classes considering the mean value of the data and the percentage classes above or below the mean value in the range of −100% below the mean value to 100% above the mean value, as well as performance of the MAE of every class. This evaluation measure ensures the visibility of error clustering. The mean absolute error can be defined as(10)MAE=1n∑k=1ndk-ok,where dk is the target value and ok is the model response value in the kth epoch of calculation.The model quality boundary criteria of the validation and evaluation measures for the MSE, MSRE, and MAE are not strictly defined, but it is preferred that they be as small as possible, with a value of 0 indicating a perfect fit. Quality boundaries of the CE andr2 measures are shown in Table 1 [7, 8].Because ANN models operate as universal optimizers and are able to replicate any input data to output data, evaluations must be performed with data that are not used during the training process. In this manner, generalization properties can be evaluated. In other words, it is possible to determine whether the ANN model is able to produce good responses according to learned similar events from the training process.


## 2.1. Monitoring

As shown in Figure1, before monitoring points in the research area are established, it is important to collect all of the available historical data, such as information on constructed hydraulic structures (e.g., river network, river regulation), geology (e.g., soil type, erosion, and landslide-affected areas), land use (e.g., types of vegetation coverage, areas used for agriculture), and anthropology (e.g., urban areas, traffic infrastructures, and illegal waste disposals), as well as historical data (e.g., affected areas in the past, implemented structural and nonstructural measures).After the available data are collected, continuous monitoring of meteorological and hydrological parameters should be set to recognize triggering factors that can lead to the hazard events and to represent the basis of every model. The establishment of monitoring with at least one metrological station and water level monitoring point is highly recommended.Before developing the model, small catchments should be monitored for a sufficiently long period to have a range of several heavy rain events in different periods of the year, with a minimum period of two years. Additionally, the time step of meteorological and hydrological measurements should not be longer than five minutes.


## 2.2. Modelling

After collecting a sufficient amount of measured data, the model development can begin by identifying model inputs and outputs. For the purpose of modelling the small catchment using the ANN model with a small time step, the measured data must be processed to remove data noise and to identify possible systematic errors because they can lead to appreciable model prediction errors. If data processing did not remove all errors, data collection procedure must be verified until the problem is resolved. The entire procedure of data processing is shown in Figure1.The ANN model is chosen to apply the predictive model to the small catchment because it is a fast and efficient model that can rapidly predict hazards caused by the activity of water, thus leaving sufficient time to announce a hazard notification.An ANN is a massively parallel distributed processor that has a natural propensity for storing experiential knowledge and making it available for use. It resembles the brain in two respects: (i) knowledge is acquired by the network through a learning process and (ii) interneuron connection strengths, known as synaptic weights, are used to store the knowledge [24].The main microstructural component of the ANN is the artificial neuron node, whose model is shown in Figure2.Figure 2
Artificial neuron node model (x1,x2,…,xm: input data; w1,w2,…,wm: weight coefficient; vk is the sum of products of the weight coefficients; φ is the activation function of the neuron node; ok is the response of the neuron node in the kth epoch of the calculation) [8].An artificial neuron node can also be defined by the following mathematical expressions:(1)ok=φvk,vk=∑n=0mwk×xkn,where ok is the response of the neuron node in the kth epoch of the calculation, vk is the sum of products of the weight coefficients wk, xk is the input data in the kth epoch of the calculation, and φ is the activation function of the neuron node.As shown in Figure1, ANN implementation consists of (i) selection of the adequate ANN architecture and training algorithm and (ii) ANN training procedure. ANN implementation procedure is shown in Figure 3.Figure 3
ANN implementation procedure.ANN implementation in prediction model according to Figure3 starts with the selection of the ANN mesostructure, which refer to the type of network (architecture) with which the model will be built. This structure can in turn be generally divided into static and dynamic ones; this is followed by the selection of the activation function [8, 9]. The most common types of networks used in the development of rainfall-runoff models are (i) multilayer perceptron (MLP), (ii) radial basis function (RBF), (iii) self-organizing map (SOP), and (iv) support vector machines (SVMs) [9]. The MLP architecture is the best choice for data-driven prediction model development [8]. The MLP architecture can be described as a static feed forward neuron network that consists of a minimum of three layers: (i) input, (ii) hidden, and (iii) output, as shown in Figure 4. Every layer consists of neurons that are connected by activation functions. Activation functions can be (i) linear, (ii) limited linear, (iii) unipolar sigmoid, (iv) bipolar sigmoid, or (v) hyperbolic tangent, among others [9, 12]. Their purpose is to direct data through the layers of the network from the input layer to the output layer. The numbers of neurons in the input and output layers are defined by the number of selected data, whereas the number of neurons in the hidden layer should be optimized to avoid overfitting the model, defined as the loss of predictive ability [9].Figure 4
Multilayer perceptron (MLP) model [8].The MLP architecture was introduced by Werbos in 1974 in his Ph.D. thesis [25]. Its final form was introduced by Rumelhart, Hinton, and Williams in 1986 [26], who also presented applications of the MLP architecture and a description of its success in prediction, classification, and association related to real problems.For the purpose of predictive hydrological ANN model development it is important that input layer consist of the data with minimum ten delay steps and output layer with prediction time step as presented in Figure3. Delay steps can be defined as input data from previous time steps.Because the output data from the network in one epoch of calculation will have errors, which are a function of the target output and model response in the output layer, an algorithm for determining the changeΔwk of the weight coefficient wk is needed. These algorithms are known as training algorithms because they optimize input data in each following epoch, which reduces the error in the output layer with respect to the target output. The optimization of the weight coefficient can be defined as(2)wk+1=wk+Δwk,where wk+1 is the weight coefficient in the k+1th epoch and Δwk is the change determined by the training algorithm.Training algorithms can be divided into three groups: (i) first-order local algorithms (error backpropagation, generalized delta rule), (ii) second-order local algorithms (Newton algorithm, quasi-Newton algorithm, and Levenberg-Marquardt (LM) algorithm), and (iii) global algorithms (genetic algorithm, simulated annealing, and evolutionary programming) [9, 27].The LM algorithm is the fastest and most appropriate for training simpler structures [28] under the MLP architecture, and it was specially developed for the training of ANNs. Because of those characteristics, this algorithm is proposed for the development of data-driven ANN models for small catchments as shown in ANN implementation procedure flowchart (Figure 3).Using the second-order local algorithms, the change measureΔwk is obtained from the squared approximation of the error function, which is represented by the Hessian matrix. Because the Hessian matrix typically cannot be used in ANN training and because it is not in compliance with appropriate conditions and is thus unsolvable, algorithms that avoid solving the Hessian matrix, such as the LM algorithm, are used.The LM algorithm [29], which is a special combination of the Gauss-Newton and error backpropagation algorithms, uses a conjugate gradient method by introducing the Jacobian matrix instead of the Hessian matrix. The change measure Δwk can be defined as(3)Δwk=-JT∗J+μ∗I-1×JT∗e,where J is the Jacobian matrix of the error vector e with respect to the weight coefficients in the kth epoch of the calculation, JT is the transpose of the Jacobian matrix, and μ is a scalar representing the learning rate.The Jacobian matrix of networks errors can be written as(4)J=∂e∂w=∂e1∂w1∂e1∂w2⋯∂e1∂wn∂e2∂w1∂e2∂w2⋯∂e2∂wn⋮⋮⋮∂en∂w1∂en∂w2⋯∂en∂wn,where J is the Jacobian matrix of the network errors, e1,e2,…,en are the errors, and w1,w2,…,wn are the weight coefficients.At the end of every calculating epoch, the sum squared errorE(e) is calculated as follows: (5)Ee=∑k=1nek2=∑k=1ndk-ok2,where ek is the error in kth epoch of the calculation, dk is the target value, and ok is the response model value in kth epoch of the calculation.Depending on the increase or decrease in the sum squared errorE(e), the learning rate scalar μ changes through every epoch of the calculation by dividing or multiplying by a constant factor (e.g., β in the range [0,1]) to control the LM algorithm to be more similar to the Gauss-Newton error backpropagation algorithm and also to increase the training speed. If the sum of squared errors increases, the learning rate scalar μ will be multiplied by a constant amount β, and the LM algorithm will be more similar to the Gauss-Newton algorithm; otherwise, it will be more similar to the backpropagation algorithm.After the architecture of the ANN and the training algorithm are determined, the software should be chosen in order to conduct ANN training process as shown in Figure3. There is a variety of prepared software programs available for ANN modelling, such as Brainmaker Professional, NeuralWorks Professional II/Plus, Explorer from Neural Ware Inc., WEKA, MATLAB Neural Network Toolbox, and Statistica [8, 9]. For the purpose of this study, MATLAB Neural Network Toolbox is proposed because it provides built-in training process that stops when the ANN is adequately trained. ANN model should be trained for every time prediction step and, after training process, validated and evaluated as presented in model development flowchart (Figure 1).


## 2.3. Validation and Evaluation

Assessment of the model during the training period is considered the model validation, and it cannot be used as criteria with which to evaluate the predictive abilities of the ANN model. Validation is defined as an assessment of the errors between the ANN model response and the target training data, and it can be represented by the same measures as the evaluation, the most common being the mean square error (MSE) and the coefficient of determination (r2), which are defined by (6) and (9), respectively.Validation boundary and scale criteria according to validation measures are presented in Table1. If validation process has indicated that the model is “poor” or “very poor,” the model should be improved. Figure 1 shows four possible steps for the model improvement: (i) reduction of the prediction time step, (ii) increase of the data monitoring collection period, (iii) selection of the different ANN architecture and/or training algorithm, or (iv) identification of the error in modelling process. If model improvement did not result in problem solving, then ANN is not appropriate for predictive purposes of small catchments.Table 1
Boundary scale for validation and evaluation criteria.
Validation/evaluation boundary and scale criteria
CE
r
2
-∞, +1
[−1, +1]
Very good
0.75, 1.00
|0.75|, |1|
Good
0.65, 0.75
/
Poor
0.5, 0.65
|0.50|, |0.75|
Very poor
≤0.5
<|0.5|The evaluation of the model, as shown in Figure1, is considered to be an assessment of the predictive ability of the time step of the ANN model. As mentioned before, evaluation of ANN models and of predictive models in general is problematic. There are a large number of evaluative measures that are widely used, and they can be divided into visual and quantitative measures. Visual evaluation measures are considered to be graphical representations of the ANN model response and target data in the form of the graph, which provides insight into errors in the model output. The most commonly used calculation evaluation measures are the MSE, the mean square relative error (MSRE), the Nash-Sutcliffe coefficient (CE), and coefficient of determination (r2) [8, 30]. The MSE and MSRE are measures that indicate error in the units (or squared units) of the model, and CE and r2 describe the degree of collinearity between modelled and measured data [29]. The described measures can be defined by the following equations: (6)MSE=1n∑k=1ndk-ok2,(7)MSRE=1n∑k=1ndk-okdk2,(8)CE=1-∑k=1ndk-ok2∑k=1ndk-d-2,(9)r2=∑k=1ndk-d-ok-o-∑k=1ndk-d-2∑k=1nok-o-22,where n is the number of data points in the input layer, dk is the target value, ok is the model response value in the kth epoch of the calculation, d- is the mean value of the target data, and o- is the mean value of the network response data.These measures provide insight into the global model errors, but it is impossible to determine the distribution of the errors from those measures. Many studies have been published on classification approaches to model evaluation, such as seasonal weather data classification [31], classification of the predictions according to the percentage of observed data, or measurement of the mean absolute error (MAE) and root mean squared error (RMSE) for all predicted peak flood events in a data set [32]. Thus, for ANN model evaluation in small catchments, the classification of the errors is proposed in this paper. This evaluation consists of separating the data into evaluation classes considering the mean value of the data and the percentage classes above or below the mean value in the range of −100% below the mean value to 100% above the mean value, as well as performance of the MAE of every class. This evaluation measure ensures the visibility of error clustering. The mean absolute error can be defined as(10)MAE=1n∑k=1ndk-ok,where dk is the target value and ok is the model response value in the kth epoch of calculation.The model quality boundary criteria of the validation and evaluation measures for the MSE, MSRE, and MAE are not strictly defined, but it is preferred that they be as small as possible, with a value of 0 indicating a perfect fit. Quality boundaries of the CE andr2 measures are shown in Table 1 [7, 8].Because ANN models operate as universal optimizers and are able to replicate any input data to output data, evaluations must be performed with data that are not used during the training process. In this manner, generalization properties can be evaluated. In other words, it is possible to determine whether the ANN model is able to produce good responses according to learned similar events from the training process.


## 3. Implementation of the Model

### 3.1. Location of the Research Area and Geological and Hydrological Characteristics

The Slani Potok catchment is a part of the Dubračina River catchment area, located in the central part of the Vinodol valley, as shown in Figure5. The Vinodol valley is a separated geographical entity of the eastern Kvarner area in the Republic of Croatia, and it is a unique spatial unit between the Križišće village to the northwest, the city of Novi Vinodolski to southeast, and the Vinodol channel.Figure 5
Location of the investigated area according to the Republic of Croatia map, with an aerial photograph of the Slani Potok catchment area [33].The Slani Potok catchment area can be considered an example of combined erosion. Excessive surface erosion occurs in an area that is 600 m in length and 250 m in width. Side effects around the erosion centre include local landslides, which result from weathering of the flysch rock mass. This affected area is approximately 3 km2 large, and the surrounding settlements of Belgrade, Baretići, Grižane, and Kamenjak, as well as the surrounding roads, are at risk. Because of mentioned hazard risk, this area was chosen as the case study area under the bilateral Croatian-Japanese project “Risk Identification and Land-Use Planning for Disaster Mitigation of Landslides and Floods in Croatia” coordinated by the Research Centre for Natural Hazards and Disaster Recovery of the Niigata University in Japan. Within this project’s timeframe (from 2009 to 2014) monitoring of the meteorological and hydrological parameters was established. The same case study area research continued, financed by the University of Rijeka in the Republic of Croatia, as part of the scientific project “Water Resources Hydrology and Floods and Mud Flow Risks Identification in the Karstic Area.” Results of aforementioned research became the foundation for the hydrological model development based on ANN methodology.The Slani Potok catchment has an area of approximately 2 km2, and its altitude extends from 50 to 700 m a.s.l. The average slope of the catchment area is 22%, and the slopes range from 5% to 100%, as shown in Figure 6. Therefore, this catchment area is characterized as being very steep. The lower part of the catchment area (0.9 km2) is formed in flysch sediments (mainly siltstone), and it contributes the majority of the surface runoff. The upper part of the catchment area is a karstic plateau from which the runoff is insignificant. A schematized geologic map of the area is shown in Figure 7. In the karstic and flysch contact zone, several overflow springs are placed, contributing the majority of the water balance in the dry season.Figure 6
Slope map of the Slani Potok catchment area.Figure 7
Schematized geologic map of the Vinodol valley with the Slani Potok catchment area [33].As noted in Figure7, the Slani Potok catchment area is known as an example where erosion is combined with local landslides. Together with water activity, these landslides have resulted in an increasing occurrence of flash floods, mud flows, and debris flows in the last 100 years.The main problem with this surface erosion area is the impossibility of reconstruction or mitigation of erosion processes or human activity. Therefore, it is essential to establish EWSs to notify residents about the possibility of occurrence of a hazard in a timely manner. The study catchment is small, with a large coefficient of runoff, distinct steep slopes, and a short response time of the rain event, which means that the time period from the beginning of the rain event until the maximum hydrograph peak can be measured in minutes. Therefore, it is essential to develop a model with the capability for fast response, such as a data-driven ANN model.


### 3.2. Data Collection

Continuous data monitoring points of the hydrological and meteorological parameters have been established since 2012. Water levels in the Slani Potok creek waterbed are measured by a Mini Diver pressure probe (manufactured by Schlumberger Water Services) at the mouth of the Slani Potok creek as it enters the Dubračina River. Meteorological parameters were measured using a Vantage Pro 2 meteorological station (manufactured by Davis Instruments Corporation) near Belgrade with a measurement frequency interval of two minutes. The position of the installed equipment is presented in Figure8. After three years of data collection, rain events from 2013 were selected as the representative data set.Figure 8
Location of the monitoring points in the Slani Potok catchment.


### 3.3. Data Processing and Model Implementation

Because this area is known as a hazard area, the impact of the rainfall on the erosion base was recognized many years ago. An immediate hazard is possible when the rainfall starts to erode the surface, causing local landslides, which bring mud and debris mixed with water downstream.Selection of the input layer data and output layer data was conducted to develop the ANN model. In this case study, the following meteorological parameters were selected as input data: (i) rain, (ii) rain rate, (iii) air temperature, (iv) humidity, (v) air pressure, and (vi) solar radiation. River water levels were used as output data (target data), as shown in Figure9. Those meteorological parameters were selected because they directly or indirectly influence the prediction of the rain event or because they define the hydrometeorological conditions of the catchment.Figure 9
Schematized ANN model structure.Using the software MATLAB 2012a (MathWorks, Natick, Massachusetts, US), selected data were processed to recognize errors and then locally smoothed by using locally weighted polynomial regression (LOESS method) [34] to eliminate data noise, and then the time between input and output layers was synchronized. After data processing, data were divided into training, validation, and evaluation data in a proportion of 70% for training, 15% for validation, and 15% for evaluation. Statistics of the data used for the model are shown in Table 2.Table 2
Statistics of data used for training and evaluation of the ANN model.
Statistics∗
Input layer
Output layer
Rain
Rain rate
Air temperature
Humidity
Air pressure
Solar radiation
Water level
[mm]
[mm/h]
[°C]
[%]
[hPa]
[W/m²]
[cm]
Model training data
n
92948
92948
92948
92948
92948
92948
92948
Max.
7.68
230.4
33.3
96
773
1092
156.7
Min.
0
0
5.8
32
750.2
0
8.1
µ
0.0066
0.20
16.78
68.68
762.42
113.25
64.90
σ
0.0955
2.86
4.58
14.88
3.86
214.69
7.79
Model validation data
n
19912
19912
19912
19912
19912
19912
19912
Max.
2.85
85.4
27.5
96
772.1
860.0
104.0
Min.
0
0
6.20
47.0
753.7
0
63.3
µ
0.0053
0.158
12.78
76.03
762.43
60.20
70.88
σ
0.0558
1.6744
4.89
10.81
4.41
131.85
4.47
Model evaluation data
n
19912
19912
19912
19912
19912
19912
19912
Max.
10.11
303.2
29.8
95
764.80
938
210.54
Min.
0
0
14.1
38
752.6
0
61.47
µ
0.0166
0.499
21.14
69.80
760.66
151.77
66.99
Σ
0.218
6.525
3.37
12.76
2.25
229.973
5.86
∗
n: number of observation; Max.: maximum; Min.: minimum; µ: sample mean; σ: standard deviation.The training data included 92,948 samples, with over ten large rain events that caused a maximum water level of 156.7 cm. The validation data included 19,912 samples with six rain events, with a maximum water level of 104.0 cm. The evaluation data set included 19,912 samples with five rain events, one of which resulted in a water level of 210.54 cm and induced debris flow and infrastructure damage; thus, this data set is excellent for evaluating the predictive ability of the model.As described in the methodology of this paper, an MLP mesostructure is used to develop the data-driven ANN model for small catchments, with sigmoid and linear activation functions trained by the LM algorithm. The model is conducted with the help of the software MATLAB 2012a Neural Network Toolbox (MathWorks, Natick, Massachusetts, US).To test the predictive capability of the model, ten steps of delay were used in the input layer. In other words, meteorological parameters from the last ten measured parameters (twenty minutes) were used in every step of the calculation. In the output layer, future steps for prediction at the fifteenth step (S15;t+30 minutes), thirtieth step (S30; t+60 minutes), and sixtieth step (S60; t+120 minutes) were selected. The schematized structure of the prediction model is shown in Figure 10.Figure 10
Schematized ANN prediction model.Furthermore, 10 neurons are chosen to provide calculation in the hidden layer.Validation of the developed model, after the training process and according to the proposed measures, is presented visually by comparing the water level targets with the ANN model response in Figure11, as well as by calculations according to (6) and (9) for all prediction steps, as shown in Table 3.Table 3
Performance statistics of the ANN model during validation.
Validation prediction step
MSE
r
2
S15 (t + 30 minutes)
0.603
0.960
S30 (t + 60 minutes)
1.150
0.940
S60 (t + 120 minutes)
1.391
0.932Figure 11
Graphical presentation of the target water level data and response water level data for the ANN model during validation: (a) S15, (b) S30, and (c) S60.
(a)
(b)
(c)The validation results have shown that each of three prediction steps can be used, but their prediction quality must be evaluated. The validation measures presented in Table3 for the MSE are small in all prediction steps, which means that the models do not have many global errors. r2 indicates that the models can be categorized as “very good” according to the model quality criteria in Table 1. Additionally, by visual comparison of the target data with the response of the ANN model in Figure 11, a good match with the data is visible, with some deviations in the maximum water levels.


### 3.4. Water Level Prediction Capability: Model Results and Discussion

The predictive ability of the model is tested by visual and quantitative evaluation measures for prediction steps S15, S30, and S60. A graphical presentation of the water level target data and the ANN model response is shown in Figure12. The performance of the model is quantitatively evaluated according to (6), (7), (8), and (9), as shown in Table 4.Table 4
Performance statistics of the ANN model during evaluation.
Evaluation prediction step
MSE
MSRE
CE
r
2
S15
5.737
0.0003
0.833
0.902
S30
9.359
0.0005
0.728
0.849
S60
11.656
0.0007
0.661
0.809Figure 12
Graphical presentation of the target data and response of the ANN prediction model during evaluation: (a) S15, (b) S30, and (c) S60.
(a)
(b)
(c)Because the model was evaluated using a data set that consists of data that were not used in the training process and the data set included one large rain event that caused a hazard, it is visibly apparent in Figure12 that errors in the prediction of the maximum water levels increase at every prediction step. Additionally, prediction of the other water levels did not result in large errors at all of the prediction steps. Although the visual evaluation indicates errors in the high water level prediction, the models still have a good time response to increases in water level.The results of the quantitative evaluation measures (see Table4) indicate an increase in the errors at every step of the prediction, as expected. The MSE and MSRE measures indicate small global errors in the models. The evaluation measure CE, according to the model quality criteria boundaries presented in Table 1, categorized prediction model S15 as “very good” and models S30 and S60 as “good,” whereas evaluation measure r2 categorized all prediction steps in the models as “very good.” The calculation evaluation measures show that all prediction models are usable for the prediction and do not reproduce large global model errors. Visual evaluation errors are recognized in predictions of high water levels. Therefore, to evaluate the predictive models using target water level data, data must be categorized into classes to recognize error clustering.Error clustering of the prediction models was evaluated by categorizing the target data into classes and solving (10) on every class. The data classes are presented in Figure 13, and the results of the class evaluation are presented in Table 5.Table 5
Performance statistics of the ANN model during evaluation: mean absolute error (MAE) for the data classes.
Target data versus output data[cm]
S15
S30
S60
Maximum absolute error[cm]
98.24
111.34
117.37
Class
Percentage[%]
Water level class limits[cm]
MAE
1
100, 75
210.5, 174.65
82.54
100.47
101.32
2
75, 50
174.65, 138.76
38.7
53.30
59.80
3
50, 25
138.76, 102.88
13.55
24.94
34.39
4
25, 0
102.88, 66.99
0.56
0.74
0.84
5
0, −25
66.99, 65.61
0.29
0.44
0.71
6
-25, −50
65.61, 64.23
0.23
0.67
1.35
7
-50, −75
64.23, 62.85
0.23
0.41
0.44
8
[
-75, −100]
[62.85, 61.47]
0.21
0.28
0.45Figure 13
Evaluation classes of the target water level data.The error clustering evaluation performed by the MAE shows that, for all prediction steps, the majority of the errors are placed in classes 1, 2, and 3. In other words, the values of evaluation measure MAE are larger if the predicted water level is in the range above 25% of the mean water level. In the data range between −100% and 25% of the data set, the MAE value is small aside from model S60, which showed large errors for all visual and quantitative evaluation measures. Therefore, the S15 and S30 models can be used for prediction purposes.The conducted evaluation indicates that, for all prediction steps, errors in maximum water levels occurred and increased at each time prediction step. The majority of errors are clustered near maximum water level predictions, which can be explained by the use of a data set, for the training process, that did not have a sufficient variety in water levels to predict the maximum water level, which was not used in the training process.For the development of the EWS, the main objective is to obtain a model that is able to predict the time when the water level will start to increase according to meteorological parameters; this objective has been fulfilled. After the evaluation data set is implemented in the training process, the errors in maximum water level are expected to decrease, and, thus, the models will have better water level prediction performance.According to the visual and calculated evaluation measures, it is difficult to determine which prediction step is optimal for use because all of the measures (apart from MAE) categorized model S15 as “very good” and models S30 and S60 as “good.” As noted above, there is a significant problem in evaluation of the models. In this case, it is the best to exclude model S60 because visual evaluation and the MAE indicated large clustering errors.


## 3.1. Location of the Research Area and Geological and Hydrological Characteristics

The Slani Potok catchment is a part of the Dubračina River catchment area, located in the central part of the Vinodol valley, as shown in Figure5. The Vinodol valley is a separated geographical entity of the eastern Kvarner area in the Republic of Croatia, and it is a unique spatial unit between the Križišće village to the northwest, the city of Novi Vinodolski to southeast, and the Vinodol channel.Figure 5
Location of the investigated area according to the Republic of Croatia map, with an aerial photograph of the Slani Potok catchment area [33].The Slani Potok catchment area can be considered an example of combined erosion. Excessive surface erosion occurs in an area that is 600 m in length and 250 m in width. Side effects around the erosion centre include local landslides, which result from weathering of the flysch rock mass. This affected area is approximately 3 km2 large, and the surrounding settlements of Belgrade, Baretići, Grižane, and Kamenjak, as well as the surrounding roads, are at risk. Because of mentioned hazard risk, this area was chosen as the case study area under the bilateral Croatian-Japanese project “Risk Identification and Land-Use Planning for Disaster Mitigation of Landslides and Floods in Croatia” coordinated by the Research Centre for Natural Hazards and Disaster Recovery of the Niigata University in Japan. Within this project’s timeframe (from 2009 to 2014) monitoring of the meteorological and hydrological parameters was established. The same case study area research continued, financed by the University of Rijeka in the Republic of Croatia, as part of the scientific project “Water Resources Hydrology and Floods and Mud Flow Risks Identification in the Karstic Area.” Results of aforementioned research became the foundation for the hydrological model development based on ANN methodology.The Slani Potok catchment has an area of approximately 2 km2, and its altitude extends from 50 to 700 m a.s.l. The average slope of the catchment area is 22%, and the slopes range from 5% to 100%, as shown in Figure 6. Therefore, this catchment area is characterized as being very steep. The lower part of the catchment area (0.9 km2) is formed in flysch sediments (mainly siltstone), and it contributes the majority of the surface runoff. The upper part of the catchment area is a karstic plateau from which the runoff is insignificant. A schematized geologic map of the area is shown in Figure 7. In the karstic and flysch contact zone, several overflow springs are placed, contributing the majority of the water balance in the dry season.Figure 6
Slope map of the Slani Potok catchment area.Figure 7
Schematized geologic map of the Vinodol valley with the Slani Potok catchment area [33].As noted in Figure7, the Slani Potok catchment area is known as an example where erosion is combined with local landslides. Together with water activity, these landslides have resulted in an increasing occurrence of flash floods, mud flows, and debris flows in the last 100 years.The main problem with this surface erosion area is the impossibility of reconstruction or mitigation of erosion processes or human activity. Therefore, it is essential to establish EWSs to notify residents about the possibility of occurrence of a hazard in a timely manner. The study catchment is small, with a large coefficient of runoff, distinct steep slopes, and a short response time of the rain event, which means that the time period from the beginning of the rain event until the maximum hydrograph peak can be measured in minutes. Therefore, it is essential to develop a model with the capability for fast response, such as a data-driven ANN model.


## 3.2. Data Collection

Continuous data monitoring points of the hydrological and meteorological parameters have been established since 2012. Water levels in the Slani Potok creek waterbed are measured by a Mini Diver pressure probe (manufactured by Schlumberger Water Services) at the mouth of the Slani Potok creek as it enters the Dubračina River. Meteorological parameters were measured using a Vantage Pro 2 meteorological station (manufactured by Davis Instruments Corporation) near Belgrade with a measurement frequency interval of two minutes. The position of the installed equipment is presented in Figure8. After three years of data collection, rain events from 2013 were selected as the representative data set.Figure 8
Location of the monitoring points in the Slani Potok catchment.


## 3.3. Data Processing and Model Implementation

Because this area is known as a hazard area, the impact of the rainfall on the erosion base was recognized many years ago. An immediate hazard is possible when the rainfall starts to erode the surface, causing local landslides, which bring mud and debris mixed with water downstream.Selection of the input layer data and output layer data was conducted to develop the ANN model. In this case study, the following meteorological parameters were selected as input data: (i) rain, (ii) rain rate, (iii) air temperature, (iv) humidity, (v) air pressure, and (vi) solar radiation. River water levels were used as output data (target data), as shown in Figure9. Those meteorological parameters were selected because they directly or indirectly influence the prediction of the rain event or because they define the hydrometeorological conditions of the catchment.Figure 9
Schematized ANN model structure.Using the software MATLAB 2012a (MathWorks, Natick, Massachusetts, US), selected data were processed to recognize errors and then locally smoothed by using locally weighted polynomial regression (LOESS method) [34] to eliminate data noise, and then the time between input and output layers was synchronized. After data processing, data were divided into training, validation, and evaluation data in a proportion of 70% for training, 15% for validation, and 15% for evaluation. Statistics of the data used for the model are shown in Table 2.Table 2
Statistics of data used for training and evaluation of the ANN model.
Statistics∗
Input layer
Output layer
Rain
Rain rate
Air temperature
Humidity
Air pressure
Solar radiation
Water level
[mm]
[mm/h]
[°C]
[%]
[hPa]
[W/m²]
[cm]
Model training data
n
92948
92948
92948
92948
92948
92948
92948
Max.
7.68
230.4
33.3
96
773
1092
156.7
Min.
0
0
5.8
32
750.2
0
8.1
µ
0.0066
0.20
16.78
68.68
762.42
113.25
64.90
σ
0.0955
2.86
4.58
14.88
3.86
214.69
7.79
Model validation data
n
19912
19912
19912
19912
19912
19912
19912
Max.
2.85
85.4
27.5
96
772.1
860.0
104.0
Min.
0
0
6.20
47.0
753.7
0
63.3
µ
0.0053
0.158
12.78
76.03
762.43
60.20
70.88
σ
0.0558
1.6744
4.89
10.81
4.41
131.85
4.47
Model evaluation data
n
19912
19912
19912
19912
19912
19912
19912
Max.
10.11
303.2
29.8
95
764.80
938
210.54
Min.
0
0
14.1
38
752.6
0
61.47
µ
0.0166
0.499
21.14
69.80
760.66
151.77
66.99
Σ
0.218
6.525
3.37
12.76
2.25
229.973
5.86
∗
n: number of observation; Max.: maximum; Min.: minimum; µ: sample mean; σ: standard deviation.The training data included 92,948 samples, with over ten large rain events that caused a maximum water level of 156.7 cm. The validation data included 19,912 samples with six rain events, with a maximum water level of 104.0 cm. The evaluation data set included 19,912 samples with five rain events, one of which resulted in a water level of 210.54 cm and induced debris flow and infrastructure damage; thus, this data set is excellent for evaluating the predictive ability of the model.As described in the methodology of this paper, an MLP mesostructure is used to develop the data-driven ANN model for small catchments, with sigmoid and linear activation functions trained by the LM algorithm. The model is conducted with the help of the software MATLAB 2012a Neural Network Toolbox (MathWorks, Natick, Massachusetts, US).To test the predictive capability of the model, ten steps of delay were used in the input layer. In other words, meteorological parameters from the last ten measured parameters (twenty minutes) were used in every step of the calculation. In the output layer, future steps for prediction at the fifteenth step (S15;t+30 minutes), thirtieth step (S30; t+60 minutes), and sixtieth step (S60; t+120 minutes) were selected. The schematized structure of the prediction model is shown in Figure 10.Figure 10
Schematized ANN prediction model.Furthermore, 10 neurons are chosen to provide calculation in the hidden layer.Validation of the developed model, after the training process and according to the proposed measures, is presented visually by comparing the water level targets with the ANN model response in Figure11, as well as by calculations according to (6) and (9) for all prediction steps, as shown in Table 3.Table 3
Performance statistics of the ANN model during validation.
Validation prediction step
MSE
r
2
S15 (t + 30 minutes)
0.603
0.960
S30 (t + 60 minutes)
1.150
0.940
S60 (t + 120 minutes)
1.391
0.932Figure 11
Graphical presentation of the target water level data and response water level data for the ANN model during validation: (a) S15, (b) S30, and (c) S60.
(a)
(b)
(c)The validation results have shown that each of three prediction steps can be used, but their prediction quality must be evaluated. The validation measures presented in Table3 for the MSE are small in all prediction steps, which means that the models do not have many global errors. r2 indicates that the models can be categorized as “very good” according to the model quality criteria in Table 1. Additionally, by visual comparison of the target data with the response of the ANN model in Figure 11, a good match with the data is visible, with some deviations in the maximum water levels.


## 3.4. Water Level Prediction Capability: Model Results and Discussion

The predictive ability of the model is tested by visual and quantitative evaluation measures for prediction steps S15, S30, and S60. A graphical presentation of the water level target data and the ANN model response is shown in Figure12. The performance of the model is quantitatively evaluated according to (6), (7), (8), and (9), as shown in Table 4.Table 4
Performance statistics of the ANN model during evaluation.
Evaluation prediction step
MSE
MSRE
CE
r
2
S15
5.737
0.0003
0.833
0.902
S30
9.359
0.0005
0.728
0.849
S60
11.656
0.0007
0.661
0.809Figure 12
Graphical presentation of the target data and response of the ANN prediction model during evaluation: (a) S15, (b) S30, and (c) S60.
(a)
(b)
(c)Because the model was evaluated using a data set that consists of data that were not used in the training process and the data set included one large rain event that caused a hazard, it is visibly apparent in Figure12 that errors in the prediction of the maximum water levels increase at every prediction step. Additionally, prediction of the other water levels did not result in large errors at all of the prediction steps. Although the visual evaluation indicates errors in the high water level prediction, the models still have a good time response to increases in water level.The results of the quantitative evaluation measures (see Table4) indicate an increase in the errors at every step of the prediction, as expected. The MSE and MSRE measures indicate small global errors in the models. The evaluation measure CE, according to the model quality criteria boundaries presented in Table 1, categorized prediction model S15 as “very good” and models S30 and S60 as “good,” whereas evaluation measure r2 categorized all prediction steps in the models as “very good.” The calculation evaluation measures show that all prediction models are usable for the prediction and do not reproduce large global model errors. Visual evaluation errors are recognized in predictions of high water levels. Therefore, to evaluate the predictive models using target water level data, data must be categorized into classes to recognize error clustering.Error clustering of the prediction models was evaluated by categorizing the target data into classes and solving (10) on every class. The data classes are presented in Figure 13, and the results of the class evaluation are presented in Table 5.Table 5
Performance statistics of the ANN model during evaluation: mean absolute error (MAE) for the data classes.
Target data versus output data[cm]
S15
S30
S60
Maximum absolute error[cm]
98.24
111.34
117.37
Class
Percentage[%]
Water level class limits[cm]
MAE
1
100, 75
210.5, 174.65
82.54
100.47
101.32
2
75, 50
174.65, 138.76
38.7
53.30
59.80
3
50, 25
138.76, 102.88
13.55
24.94
34.39
4
25, 0
102.88, 66.99
0.56
0.74
0.84
5
0, −25
66.99, 65.61
0.29
0.44
0.71
6
-25, −50
65.61, 64.23
0.23
0.67
1.35
7
-50, −75
64.23, 62.85
0.23
0.41
0.44
8
[
-75, −100]
[62.85, 61.47]
0.21
0.28
0.45Figure 13
Evaluation classes of the target water level data.The error clustering evaluation performed by the MAE shows that, for all prediction steps, the majority of the errors are placed in classes 1, 2, and 3. In other words, the values of evaluation measure MAE are larger if the predicted water level is in the range above 25% of the mean water level. In the data range between −100% and 25% of the data set, the MAE value is small aside from model S60, which showed large errors for all visual and quantitative evaluation measures. Therefore, the S15 and S30 models can be used for prediction purposes.The conducted evaluation indicates that, for all prediction steps, errors in maximum water levels occurred and increased at each time prediction step. The majority of errors are clustered near maximum water level predictions, which can be explained by the use of a data set, for the training process, that did not have a sufficient variety in water levels to predict the maximum water level, which was not used in the training process.For the development of the EWS, the main objective is to obtain a model that is able to predict the time when the water level will start to increase according to meteorological parameters; this objective has been fulfilled. After the evaluation data set is implemented in the training process, the errors in maximum water level are expected to decrease, and, thus, the models will have better water level prediction performance.According to the visual and calculated evaluation measures, it is difficult to determine which prediction step is optimal for use because all of the measures (apart from MAE) categorized model S15 as “very good” and models S30 and S60 as “good.” As noted above, there is a significant problem in evaluation of the models. In this case, it is the best to exclude model S60 because visual evaluation and the MAE indicated large clustering errors.


## 4. Conclusions

In this paper, the methodology for a data-driven ANN model for the prediction of river water levels conducted from meteorological parameters as a basis for EWS development in a small catchment is proposed. The model is implemented for the case study of the Slani Potok catchment in the Republic of Croatia, and its predictive ability is evaluated. An MLP mesostructure, with sigmoid and linear activation functions trained by the LM algorithm, is used in the ANN model development. The developed model was trained, validated, and evaluated on data set with 132,772 monitored meteorological and hydrological parameter samples that were divided in the proportions of 70% for training, 15% for validation, and 15% for verification. The predictive ability of the model was tested for time steps of thirty minutes (S15), sixty minutes (S30), and one hundred and twenty minutes (S60).The validation of the models resulted in their classification as “very good” (with small global error) for all prediction steps.Common quantitative evaluation measures (MSE, MSRE, CE, andr2) of the developed models showed that the predictive abilities of the models are classified as “very good” for model S15 and as “good” for models S30 and S60. The evaluation measure r2 categorized all model prediction steps as “very good.” Visual evaluation indicated errors in the prediction of high water levels. Thus, new measures for evaluating prediction error clustering in the small catchment were proposed. The error clustering evaluation was based on the MAE for the target data set and divided into percentage classes according to the mean data value. This showed substantial clustering of the errors in the prediction of the maximum water levels, which are 25% to 100% larger than the mean value of the water level in the Slani Potok river bed for the S30 and S60 models. Those models were developed based on the observed data sets, implying that a data set with larger variety in the training process will yield an improved prediction performance.Overall, the evaluation also showed that all models accurately predict the time when the water level starts increasing. Additionally, the evaluation showed that the model’s response is more important for the development of the EWS than precise water level prediction when considering the short time of the response of water level to rainfall in the small catchments.The conducted evaluation demonstrates that the models S15 and S30 can be used for the prediction. For EWS development, the prediction time for a small catchment does not have to be long, so a prediction time based on a maximum time step of sixty minutes (S30) can be considered sufficiently long to announce a hazard.The proposed methodology for the development, validation, and evaluation of predictive models for a small catchment can serve as the basis for the implementation of the EWS if continuous meteorological and hydrological monitoring, measured on a short time frequency, is established.



---

*Source: 9125219-2015-12-30.xml*