# Multivariate Streamflow Simulation Using Hybrid Deep Learning Models

Authors: Eyob Betru Wegayehu; Fiseha Behulu Muluneh
Journal: Computational Intelligence and Neuroscience
 (2021)

---

## Abstract

Reliable and accurate streamflow simulation has a vital role in water resource development, mainly in agriculture, environment, domestic water supply, hydropower generation, flood control, and early warning systems. In this context, these days, deep learning algorithms have got enormous attention due to their high-performance simulation capacity. In this study, we compared multilayer perceptron (MLP), long short-term memory (LSTM), and gated recurrent unit (GRU) with the proposed new hybrid models, including CNN-LSTM and CNN-GRU. Hence, we can simulate one-step daily streamflow in different agroclimatic conditions, rolling time windows, and a range of variable input combinations. The analysis used daily multivariate and multisite time series data collected from Awash River Basin (Borkena watershed: Ethiopia) and Tiber River Basin (Upper Tiber River Basin: Italy) stations. The datasets were subjected to rigorous quality control processes. Consequently, it rolled to a different time lag to remove noise in the time series and further split into training and testing datasets using a ratio of 80 : 20, respectively. Finally, the results showed that integrating the GRU layer with the convolutional layer and using monthly rolled average daily input time series could substantially improve the simulation of streamflow time series.


---

## Body

## 1. Introduction

One of the emerging research areas in hydrology is hydrological simulation [1], through which catchment responses are evaluated in terms of meteorological forcing variables. Hydrological simulation is also crucial for water resource planning and management, such as flood prevention, water supply distribution, hydraulic structure design, and reservoir operation [2, 3]. However, river flow simulation is not an easy task since river flow time series are commonly random, dynamic, and chaotic. The relationship between streamflow generation and other hydrologic processes is nonlinear, which is controlled not only by external climatic factors and global warming but also by physical catchment characteristics.Stream flows are mostly recorded at river gauging stations. However, different research studies show that the availability of gauging station records is generally decreasing in most parts of the world [4]. Tourian et al. [5] gathered a time series plot of the number of stations with available discharge data from the Global Runoff Data Centre (GRDC). This time series indicates a decline in the total monitored annual stream flows between 1970 and 2010. Besides, inadequate discharge observation and malfunctioned gauging stations worsen the situation in developing countries [6]. Sparsely distributed rain gauge stations in Ethiopia also limit the performance of physical hydrological models. Therefore, research studies on the robustness of innovative discharge data estimation models are undeniably important.Streamflow simulation models in the literature generally are divided into two: (1) process or physical-based models that are generated from catchment characteristics and (2) data-driven models that depend on historically collected data [2, 3, 7]. Process-based models commonly use the experimental formula that provides insight into physical characteristics and has extensive data requirements. On the other hand, data-driven models are suitable and can function easily without considering the internal physical mechanism of the watershed system [2, 3, 7].Artificial neural networks (ANNs) are the most used and studied “black-box” models. They are utilized in many scientific and technological areas than the list of available black-box algorithms, such as support vector machine (SVM), genetic programming (GP), fuzzy logic (FL), recurrent neural network (RNN), and long short-term memory (LSTM) [7, 8]. ANN is available in different functionalities and architectural forms, from simple to advanced levels. A recurrent neural network (RNN) is one of the advanced ANN architectures. It has been considered a specially designed deep learning network for time series analysis that quickly adapts to temporal dynamics using previous time step information [2]. However, RNN cannot capture long-time dependencies, and it is susceptible to vanishing and exploding gradients.Couta et al. suggested advanced RNN or long short-term memory (LSTM) as one of the most effective approaches [8]. The LSTM unit has a cell that comprises an input gate, an output gate, and a forget gate [9]. Due to these gates, the LSTM model has shown promising results in different applications, including speech recognition, time series modelling, natural language processing, handwriting recognition, and traffic flow simulation [3, 10]. Studies have also shown that LSTM has powerful performance for streamflow simulation over different powerful multilayered (ML) tools [3, 11]. Campos et al. [10] applied autoregressive integrated moving average (ARIMA) and LSTM network to forecast floods on four Para’ iba do Sul’s River stations in Brazil. Aljahdali et al. [7] also compared the LSTM network and layered RNN to forecast streamflow in the USA’s two rivers, the Black and Gila rivers. A recent article by Rahimzad et al. [12] used time-lagged Qt-1, Qt-2, and other climatic variables to forecast Qt in the future and concluded that the LSTM network outperforms linear regression (LR), multilayer perceptron (MLP), and support vector machine (SVM) in forecasting daily streamflow.A few years back, Cho et al. [13] introduced gated recurrent units (GRUs) similar to LSTM with a forget gate which have fewer parameters than LSTM, as it lacks an output gate. GRU’s capacities in speech signal modelling and natural language processing were similar to those of LSTM. However, there are debates on the relative performance of these two architectures for streamflow and reservoir inflow simulation, which is not well studied with different timescales and environments.Notwithstanding the difference in their performance, selecting appropriate time series models from various known deep learning network architectures is difficult. LSTMs and GRUs are not always the ideal sequence prediction option. However, simulation with better prediction accuracy, fast running time, and less complicated models requires more research. Hence, this comparative analysis on the network architectures helps decide the optimized alternative for time series analysis. Recently, different hybrid deep learning models are getting wide attention from researchers in various fields of study. Chen et al. [14] used convolutional neural network (CNN), LSTM, and hybrid CNN-LSTM models for nitrogen oxide emission prediction. They concluded that CNN-LSTM has an accurate and stable forecast of periodic nitrogen oxide emissions from the refining industry. Moreover, Li et al. [15] used univariate and multivariate time series data as input for LSTM and CNN-LSTM models. Hence, for the analysis of air quality using particulate matter (PM2.5) concentration prediction, the proposed multivariate CNN-LSTM model gives the best result due to low error and short training time.The integration of CNN and LSTM models benefits time series prediction models such that the LSTM model can efficiently capture long time sequences of pattern information. In contrast, CNN models can filter out the noise of the input data and extract more valuable features, which could increase the accuracy of the prediction model [16]. Moreover, integrating CNN with GRU can also lead us to robust preprocessing of data, providing a viable option to improve the model’s accuracy [17]. Even though combining CNN with LSTM showed remarkable results in different studies, its application in hydrological fields still demands more research [18]. Muhammad et al. [19] used LSTM, GRU, and hybrid CNN-GRU models for streamflow simulation based on 35 years of Model Parameter Estimation Experiment (MOPEX) dataset of 10 river basins in the USA. They revealed that the proposed hybrid model outperforms the conventional LSTM; nevertheless, the performance is almost the same with GRU. Recently, Barzegar et al. [20] studied short-term water quality variable prediction using a hybrid CNN-LSTM model and effectively captured low and high water quality variables, mainly dissolved oxygen concentrations.Screening input variables for different model architectures is also a challenging task for the researchers. Even though rainfall, evaporation, and temperature are causal variables for streamflow modelling, data availability and study objectives limit the choice variability [21]. Van et al. [21] discussed that applying temperature and evapotranspiration input nodes into the model increases the network complexity and causes overfitting. In contrast, Parisouj et al. [22] concluded that using readily available input variables such as temperature and precipitation for data-driven streamflow simulation will provide a reliable result. Hence, this research will contribute a step to this debate by testing different input combinations of various climatic regions in the performance of the proposed models.To the best of our knowledge, we identify minimal literature that shows the performance variation of different hybrid models for streamflow simulation in various input variability conditions at once. Thus, we compared various forms of hybrid CNN-LSTM and CNN-GRU architectures with the classical MLP, GRU, and LSTM networks to simulate single-step streamflow using two climatic regions, available precipitation, and minimum and maximum temperature data. Moreover, the study tests the hybrid models with different layer arrangements and applies Keras tuner to optimize model hyperparameters. In general, the primary objective of this study will be to test the performance variation of the proposed models with extreme input variability conditions, which includes climatic, input combination, input time window, and average rolling time window variability.This study used different open-source software and machine learning libraries, including Python 3.6 for programming, NumPy, pandas, Scikit-learn, Hydroeval, Statsmodels, and Matplotlib libraries. All were used for data preprocessing, evaluation, and graphical interpretation. Moreover, TensorFlow and Keras deep learning frameworks were employed for modelling deep learning architectures.


## 2. Study Area

In the present study, two river subcatchments were selected in two climatic regions: the Awash River Basin, Borkena subcatchment in Ethiopia (Figure1(a)), and the Upper Tiber River Basin in Italy (Figure 1(b)).Figure 1
Location of case study areas. (a) Borkena. (b) UTRB.
(a)(b)


### 2.1. Borkena Watershed (Ethiopia)

The first case study area is in the Borkena watershed at the Kombolcha station outlet, located in the upper part of the Awash River Basin in the northern part of Ethiopia. The mainstream of the watershed emanates from Tosa mountain, which is found near Dessie town. The area’s altitude ranges from 1,775 m at the lowest site near Kombolcha to 2,638 m at the highest site upstream of Dessie. The main rainy season of this watershed is from July to September.


### 2.2. Upper Tiber River Basin (Italy)

The second case study area is located in the Upper Tiber River Basin (UTRB) in Italy. The Tiber River Basin (TRB) is the second-largest catchment in Italy [23]. Geographically, the basin is located between 40.5°N to 43°N latitudes and 10.5° E to 13° E longitudes, covering about 17,500 km2 that occupies roughly 5% of the Italian territory. The Upper Tiber River Basin (UTRB) is part of the TRB, covering 4145 km2 (∼20% of the TRB) with its outlet at Ponte Nuovo. The elevation of the catchment ranges from 148 to 1561 m above sea level. The area’s climate is the Mediterranean, with precipitation mainly occurring from autumn (March to May) to spring (September to November). The intense rainfall highly influences the basin’s hydrology at the upstream part that causes frequent floods in the downstream areas [24].


## 2.1. Borkena Watershed (Ethiopia)

The first case study area is in the Borkena watershed at the Kombolcha station outlet, located in the upper part of the Awash River Basin in the northern part of Ethiopia. The mainstream of the watershed emanates from Tosa mountain, which is found near Dessie town. The area’s altitude ranges from 1,775 m at the lowest site near Kombolcha to 2,638 m at the highest site upstream of Dessie. The main rainy season of this watershed is from July to September.


## 2.2. Upper Tiber River Basin (Italy)

The second case study area is located in the Upper Tiber River Basin (UTRB) in Italy. The Tiber River Basin (TRB) is the second-largest catchment in Italy [23]. Geographically, the basin is located between 40.5°N to 43°N latitudes and 10.5° E to 13° E longitudes, covering about 17,500 km2 that occupies roughly 5% of the Italian territory. The Upper Tiber River Basin (UTRB) is part of the TRB, covering 4145 km2 (∼20% of the TRB) with its outlet at Ponte Nuovo. The elevation of the catchment ranges from 148 to 1561 m above sea level. The area’s climate is the Mediterranean, with precipitation mainly occurring from autumn (March to May) to spring (September to November). The intense rainfall highly influences the basin’s hydrology at the upstream part that causes frequent floods in the downstream areas [24].


## 3. Data Source and Preprocessing

Borkena’s required hydrological and metrological datasets were collected from the Ministry of Water Irrigation and Energy (MoWIE) of Ethiopia and the National Meteorological Agency of Ethiopia (NMA), respectively. UTRB’s datasets were collected from the National Research Council of Italy (CNR) and archived for public use with the Water Resource Management and Evaluation (WRME) platform in the following link:http://hydrogate.unipg.it/wrme/.We collected 5844 available data series from the time window of January 1, 1999, to December 31, 2014, from the Borkena watershed. Similarly, for UTRB, 7670 data series were collected from January 1, 1958, to December 31, 1978. Both case study datasets are multivariate and multisite. Even though we are highly concerned and chose the series of time windows with minimum data gap for both stations, the datasets contain many missing values due to different reasons. Thus, our first task was to fill the missing values with the Monte Carlo approach for this research.The study applied linear correlation statistics to measure the strength of dependency between different input variables [25]. Even though Mehr and Gandomi [26] stated that linear correlation might mislead or provide abundant inputs, our study does no’t have a huge feature size that requires intensive feature selection criteria. Hence, we adopted a linear correlation coefficient. Moreover, Kun et al. [27] concluded that Pearson correlation coefficient (PCC) is the most applicable for multiple linear regressions (MLRs), and Oyebode [28] also stated that inputs selected with PCC showed superior model accuracy. Hence, this study applied Pearson linear correlation coefficient [29, 30]. It has a value ranging between “+1” and “-1,” where “+1” indicates a positive linear correlation, “0” is no linear correlation, and “-1” shows a negative linear correlation [25]. Equation (1) calculates the Pearson correlation coefficient, and Tables 1 and 2 present the result. Correlation values between positive (0 and 0.3) and negative (0 and −0.3) show a weak linear relationship among variables [31]. However, since we have a small number of variables and data size, for this study, we decided to omit Borkena station (Tmax) values, which have r values ranging between (−0.129) and (+0.107), and the details are presented in Table 1.(1)r=N∑XY−∑X∑YN∑x2−∑X2N∑Y2−∑Y2.Table 1
Descriptive statistics of split time series data for the Borkena watershed.
StationsData typePearson correlation with streamflowTraining data (80%)Testing data (20%)MeanMaxMinSDMeanMaxMinSDKombolchaStream flow (m3/sec)1.00010.9216.90.0023.210.194.80.020.2P (mm/day)0.3213.173.20.07.52.960.40.07.2Tmin (oc)0.27112.520.91.53.312.520.62.63.4Tmax (°c)−0.09927.233.616.42.527.333.019.62.1ChefaP (mm/day)0.3443.581.60.08.63.464.30.08.1Tmin (oc)0.26613.321.50.13.714.122.23.93.5Tmax (oc)−0.06929.938.018.52.830.338.022.22.5DessieP (mm/day)0.3353.580.60.08.62.967.00.07.3Tmin (oc)0.3198.515.50.12.57.815.50.03.1Tmax (oc)0.10723.830.016.01.924.130.015.02.1KemiseP (mm/day)0.3723.181.90.08.32.972.10.07.5Tmin (oc)0.28213.822.03.03.413.520.14.53.6Tmax (oc)−0.12931.038.314.02.731.937.823.52.4MajeteP (mm/day)0.3473.380.70.08.63.381.30.08.6Tmin (oc)0.20214.723.01.42.914.621.56.72.9Tmax (oc)−0.05728.637.817.22.829.138.020.82.4Table 2
Descriptive statistics of split time series data for the UTRB.
StationsData typePearson correlation with streamflowTraining data (80%)Testing data (20%)MeanMaxMinSDMeanMaxMinSDPonte NuovoStreamflow (m3/sec)1.00050.6939.01.975.550.6737.03.768.6Castel RigoneP (mm/day)0.3842.672.80.06.62.767.70.06.9MontecoronaroP (mm/day)0.3393.9229.00.010.74.0110.00.010.5Perugia (ISA)P (mm/day)0.3792.4120.40.06.62.561.80.06.3Tmin (oc)−0.3539.730.4−9.06.39.325.2−5.05.6Tmax (oc)−0.37917.437.4−4.58.116.333.00.67.2PetrelleP (mm/day)0.3452.5190.00.06.92.7117.10.07.4PietralungaP (mm/day)0.4283.22150.00.08.13.173.10.07.3SpoletoP (mm/day)0.4122.9113.60.07.92.994.20.07.8Tmin (oc)−0.2657.523.0−12.66.48.821.7−5.45.8Tmax (oc)−0.38318.838.7−3.58.618.736.82.07.8TorgianoP (mm/day)0.3642.4141.20.07.12.562.00.06.9GubbioTmin (oc)−0.3158.726.0−12.05.96.119.3−11.35.4Tmax (oc)−0.37718.139.0−8.08.117.434.1−0.97.5AssisiTmin (oc)−0.3259.225.6−11.66.28.221.5−8.05.6Tmax (oc)−0.37818.237.8−5.08.318.135.80.07.8After passing rigorous quality control processes, the raw data were then split chronologically into training and testing datasets with a ratio of 80 : 20, respectively. The time series graph and the corresponding box plot of split data for both stations are presented in Figure2. Different options existed in the literature to remove noise from the time series. A sliding window is the first option to temporarily approximate the time series data’s actual value [32]. In comparison, rolling windows (moving average) is the second option that smooths the time series data by calculating the average, maximum, minimum, or sum over a specific time [33]. Hence, for this study, we applied average rolling windows to smooth and remove noise from the time series by keeping the length of the data still.Figure 2
Streamflow time series graph and the corresponding box plot of split data. (a) Borkena. (b) UTRB.Then, daily, weekly, and monthly average rolling sliding windows were used to rebuild the input and output time series into a supervised learning format. Accordingly, the rolled time series data were then prepared with the time lag window of 30 or 45 for single-step streamflow simulation at Borkena and UTRB stations, respectively. Moreover, split time series data variable scaling was performed using Standard Scaler for the modelling process’s computational easiness and numerical stability.


## 4. Methods

In this study, three types of network architectures MLP, GRU, and LSTM, were compared with the proposed hybrid deep neural network architectures CNN-LSTM and CNN-GRU for the simulation of single-step streamflow by taking different combinations of precipitation (P), minimum temperature (Tmin), and maximum temperature (Tmax) as inputs. The proposed simulation model architectures with their input and output variables are briefly presented as a flowchart in Figure 3.Figure 3
A simple architecture of the proposed models.


### 4.1. Deep Learning Models

Deep learning models are part of a broader family of machine learning, including recurrent neural networks (RNNs), convolutional neural networks (CNNs), deep belief networks (DBNs), and deep neural networks (DNNs). These models have been applied to different fields of study, including speech recognition, computer vision, natural language processing, and time series analysis [13, 16, 34–36]. The following sections will briefly discuss some of these architectures that were used in the present study.


### 4.2. Artificial Neural Network (ANN)

Artificial neural network (ANN) is the most common machine learning model that has found application in streamflow simulation over the last two decades [1, 37]. It is known for modelling complex input-output relationships inherent in hydrological time series features within a river catchment. The traditional feedforward neural network (FFNN) with three layers of input-output and hidden layers trained by backpropagation (BP) algorithm gained popularity for nonlinear hydrological time series modelling.Figure4 displays the typical architecture of ANN.(2)yj^=fj∑h=1mwjh∗fh∑i=1nwhixi+whb+wjb,where i, h, j, b, and w indicate neurons of the input, hidden, output layers, bias, and applied weight of the neuron, respectively; fh and fj show the activation functions of the hidden layer and output layer, respectively; xi, n, and m represent, respectively, the input value, input neuron, and hidden neuron numbers; and y and yj^ denote the observed and calculated target values, respectively. In the calibration phase of the model, the values of the hidden and output layers and corresponding weights could be varied and calibrated [38].Figure 4
Typical architecture of ANN.The ability of ANN to link input and output variables in complex hydrological systems without the need for prior knowledge about the nature of the process has led to a huge leap in the use of ANN models in hydrological simulations [38].


### 4.3. Long Short-Term Memory (LSTM)

The difference of LSTM from the classical MLP network is that layers of the neurons in LSTM have recurrent connections; thus, the state from the previous activation time step is used to formulate an output. The LSTM replaces the typical neuron in the hidden layer with a memory cell and three gates: an input gate, a forget gate, and an output gate [39]. It is an advanced form of recurrent neural network (RNN) that can capture long-term dependencies. On the other hand, RNN is a circular network in which an additional input is added to represent the state of the neuron in the hidden layer at the previous time steps [40]. LSTM has two critical benefits over RNN: overcoming vanishing and exploding gradients and holding memory to capture long-term temporal dependency in input sequences. The mathematical formulation for different parameters is listed in Table 3, and Figure 5 displays the LSTM memory cell with three gated layers.∗Wi, Wf, Wo, and Wc are the weights that map the hidden layer input to the three gates of input, forget, and output. Ui, Uf, Uo, and Uc weight matrices map the hidden layer output to gates; bi, bf, bo, and bc are vectors. Ct and ht are the outcome of the cell and the outcome of the layer, respectively.Table 3
Mathematical formulation for LSTM cell.
Network gatesPurposeEquations∗Forget gateChooses the information to reject from the cellft=σ (ufxt + wfht-1+ bf)Input gateDecides what information is relevant to update in the current cell stateit=σ (uixt + wiht-1+ bi)Output gateDecides what to output based on input and the long-term memory of the cellot=σ (uoxt + woht-1+ bo)Cell stateLong-term memoryCet= tanh (WcXt + Ucht−1+ bc)Ct=ft∗ Ct−1+ it∗ CetHidden stateShort-term memoryht= tanh (Ct) ∗ OtFigure 5
LSTM memory cell with three gated layers [11].


### 4.4. Gated Recurrent Unit (GRU)

GRU is a special type of LSTM architecture in which it merges the input and forget gates and converts them into an update gate, which makes the parameter numbers fewer, and the training will be easier. There are two input features each time: the input vectorxt and the previous output vector ht−1. The output of each specific gate can be calculated through logical operation and nonlinear transformation of the input [34]. The mathematical formulations among inputs, outputs, and different parameters are listed in equations (3), (4), (5), and (6). Moreover, Figure 6 displays the structure of the gated recurrent unit (GRU) network.(3)Zt=σWzXt+Uzht−1+bz,(4)rt=σWrXt+Urht−1+br,(5)h⌢t=tanhWhXt+rt∗ht−1+bh,(6)ht=1−Zt∗ht−1+Zt∗h⌢t,where Zt is the update gate vector, rt is the reset gate vector, W and U are parameter matrices, σ is a sigmoid function, and tanh is a hyperbolic tangent.Figure 6
The structure of gated recurrent unit (GRU) network [34].


### 4.5. Convolutional Neural Network (CNN)

Convolutional neural network (CNN) is one of the most successful deep learning models, especially for feature extraction, and its network structures include 1D CNN, 2D CNN, and 3D CNN [15]. CNN structure generally consists of a convolution layer, a pooling layer, and a full connection layer [18].1D CNN is mainly implemented for sequence data processing [41], 2D CNN is usually used for text and image identification [42], and usually, 3D CNN is recognized for modelling medical image and video data identification [43]. Hence, since the aim of the present study is time series analysis, we implemented 1D CNN. The detailed process of 1D CNN is described in Figure 7.Figure 7
The process of 1D CNN [15].As depicted in Figure7, the input series is convoluted to the convolution layer from top to bottom (shown by the arrows). The grey or the mesh colours represent different filters where the size of the convolution layer depends on the number of input data dimensions, the size of the filter, and the convolution step length.


### 4.6. CNN-LSTM and CNN-GRU Hybrid Models

In this study, hybrid models were designed by integrating CNN with LSTM or GRU layers. Hence, the feature sequence from the CNN layer was considered as the input for the LSTM or GRU layer, and then the short and long-time dependencies were further extracted.The proposed CNN-LSTM or CNN-GRU models contain two main components: the first component consists of one dimensional single or double convolutional and average pooling layers. Moreover, a flatten layer is connected to further process the data into the format required by the LSTM or GRU. In the second component, the generated features are processed using LSTM, GRU, and dense layers. Additionally, dropouts are introduced to prevent overfitting. Figure8 shows the designed model inputs and outputs with a basic description of the convolutional, pooling, and LSTM or GRU layers proposed for this project.Figure 8
The basic architecture of the proposed CNN-LSTM or CNN-GRU models.


## 4.1. Deep Learning Models

Deep learning models are part of a broader family of machine learning, including recurrent neural networks (RNNs), convolutional neural networks (CNNs), deep belief networks (DBNs), and deep neural networks (DNNs). These models have been applied to different fields of study, including speech recognition, computer vision, natural language processing, and time series analysis [13, 16, 34–36]. The following sections will briefly discuss some of these architectures that were used in the present study.


## 4.2. Artificial Neural Network (ANN)

Artificial neural network (ANN) is the most common machine learning model that has found application in streamflow simulation over the last two decades [1, 37]. It is known for modelling complex input-output relationships inherent in hydrological time series features within a river catchment. The traditional feedforward neural network (FFNN) with three layers of input-output and hidden layers trained by backpropagation (BP) algorithm gained popularity for nonlinear hydrological time series modelling.Figure4 displays the typical architecture of ANN.(2)yj^=fj∑h=1mwjh∗fh∑i=1nwhixi+whb+wjb,where i, h, j, b, and w indicate neurons of the input, hidden, output layers, bias, and applied weight of the neuron, respectively; fh and fj show the activation functions of the hidden layer and output layer, respectively; xi, n, and m represent, respectively, the input value, input neuron, and hidden neuron numbers; and y and yj^ denote the observed and calculated target values, respectively. In the calibration phase of the model, the values of the hidden and output layers and corresponding weights could be varied and calibrated [38].Figure 4
Typical architecture of ANN.The ability of ANN to link input and output variables in complex hydrological systems without the need for prior knowledge about the nature of the process has led to a huge leap in the use of ANN models in hydrological simulations [38].


## 4.3. Long Short-Term Memory (LSTM)

The difference of LSTM from the classical MLP network is that layers of the neurons in LSTM have recurrent connections; thus, the state from the previous activation time step is used to formulate an output. The LSTM replaces the typical neuron in the hidden layer with a memory cell and three gates: an input gate, a forget gate, and an output gate [39]. It is an advanced form of recurrent neural network (RNN) that can capture long-term dependencies. On the other hand, RNN is a circular network in which an additional input is added to represent the state of the neuron in the hidden layer at the previous time steps [40]. LSTM has two critical benefits over RNN: overcoming vanishing and exploding gradients and holding memory to capture long-term temporal dependency in input sequences. The mathematical formulation for different parameters is listed in Table 3, and Figure 5 displays the LSTM memory cell with three gated layers.∗Wi, Wf, Wo, and Wc are the weights that map the hidden layer input to the three gates of input, forget, and output. Ui, Uf, Uo, and Uc weight matrices map the hidden layer output to gates; bi, bf, bo, and bc are vectors. Ct and ht are the outcome of the cell and the outcome of the layer, respectively.Table 3
Mathematical formulation for LSTM cell.
Network gatesPurposeEquations∗Forget gateChooses the information to reject from the cellft=σ (ufxt + wfht-1+ bf)Input gateDecides what information is relevant to update in the current cell stateit=σ (uixt + wiht-1+ bi)Output gateDecides what to output based on input and the long-term memory of the cellot=σ (uoxt + woht-1+ bo)Cell stateLong-term memoryCet= tanh (WcXt + Ucht−1+ bc)Ct=ft∗ Ct−1+ it∗ CetHidden stateShort-term memoryht= tanh (Ct) ∗ OtFigure 5
LSTM memory cell with three gated layers [11].


## 4.4. Gated Recurrent Unit (GRU)

GRU is a special type of LSTM architecture in which it merges the input and forget gates and converts them into an update gate, which makes the parameter numbers fewer, and the training will be easier. There are two input features each time: the input vectorxt and the previous output vector ht−1. The output of each specific gate can be calculated through logical operation and nonlinear transformation of the input [34]. The mathematical formulations among inputs, outputs, and different parameters are listed in equations (3), (4), (5), and (6). Moreover, Figure 6 displays the structure of the gated recurrent unit (GRU) network.(3)Zt=σWzXt+Uzht−1+bz,(4)rt=σWrXt+Urht−1+br,(5)h⌢t=tanhWhXt+rt∗ht−1+bh,(6)ht=1−Zt∗ht−1+Zt∗h⌢t,where Zt is the update gate vector, rt is the reset gate vector, W and U are parameter matrices, σ is a sigmoid function, and tanh is a hyperbolic tangent.Figure 6
The structure of gated recurrent unit (GRU) network [34].


## 4.5. Convolutional Neural Network (CNN)

Convolutional neural network (CNN) is one of the most successful deep learning models, especially for feature extraction, and its network structures include 1D CNN, 2D CNN, and 3D CNN [15]. CNN structure generally consists of a convolution layer, a pooling layer, and a full connection layer [18].1D CNN is mainly implemented for sequence data processing [41], 2D CNN is usually used for text and image identification [42], and usually, 3D CNN is recognized for modelling medical image and video data identification [43]. Hence, since the aim of the present study is time series analysis, we implemented 1D CNN. The detailed process of 1D CNN is described in Figure 7.Figure 7
The process of 1D CNN [15].As depicted in Figure7, the input series is convoluted to the convolution layer from top to bottom (shown by the arrows). The grey or the mesh colours represent different filters where the size of the convolution layer depends on the number of input data dimensions, the size of the filter, and the convolution step length.


## 4.6. CNN-LSTM and CNN-GRU Hybrid Models

In this study, hybrid models were designed by integrating CNN with LSTM or GRU layers. Hence, the feature sequence from the CNN layer was considered as the input for the LSTM or GRU layer, and then the short and long-time dependencies were further extracted.The proposed CNN-LSTM or CNN-GRU models contain two main components: the first component consists of one dimensional single or double convolutional and average pooling layers. Moreover, a flatten layer is connected to further process the data into the format required by the LSTM or GRU. In the second component, the generated features are processed using LSTM, GRU, and dense layers. Additionally, dropouts are introduced to prevent overfitting. Figure8 shows the designed model inputs and outputs with a basic description of the convolutional, pooling, and LSTM or GRU layers proposed for this project.Figure 8
The basic architecture of the proposed CNN-LSTM or CNN-GRU models.


## 5. Data Analysis

Simulation with deep learning requires selecting a probable combination of hyperparameters: batch size, epochs, number of layers, and number of units for each layer [8]. Optimizing hyperparameters is not always consistent as there is no hard rule to follow. “The process is more of an art than a science” [44]. Hence, in this study, we chose the Keras tuner optimizer developed by the Google team and included it in the Keras open library [45, 46].


### 5.1. Hyperparameter Optimization

Tuning machine learning model hyperparameters is critical. Varying hyperparameter values often results in models with significantly different performances [47]. The models applied in this study mainly contain two types of hyperparameters: constant hyperparameters that are not altered through the optimization process and variable hyperparameters. Adam optimizer is applied under the category of the constant hyperparameters because of its efficiency and easiness to implementation that requires minimum memory and is commonly suited in different problems [48]. In this category, rectified linear unit (Relu) was used as an activation function, and mean squared error (MSE) was used as a loss function.In contrast, the second type of changing hyperparameters is optimized by Keras tuner, and hyperparameter choices or value ranges for optimization are set using different trials. We also considered our PC capacity (processor: Intel(R) Core (TM) i7-6500U CPU 2.50 GHz and RAM: 8 gigabytes) with Windows 10 operating system. Hyperparameters are optimized with 20 trials, and since deep learning networks have different training and validation plots for each run, we decided to repeat the iteration three times.All hyperparameter ranges or choices are listed in Table4. CNN-LSTM1 and CNN-GRU1 models used hyperparameter values from numbers 1 to 13 for optimization (Table 4), while we omitted 4, 5, and 6 for CNN-LSTM2 and CNN-GRU2. The remaining deep learning models MLP, LSTM, and GRU used a list of hyperparameters from numbers 7 to 13. Finally, each optimized hyperparameter is used for each training and testing experiment. Moreover, the train and test traces from each run can be plotted to give a more robust idea of the behaviour of the model to inspect overfitting and underfitting issues.Table 4
Model hyperparameter choices or value ranges for optimization by Keras tuner.
NoHyperparametersValue ranges∗∗ChoicesDefaultMinMaxStep1Conv_1_filter8328∗∗2Conv_1_kernal∗∗∗2 or 3∗3Conv_1_pool_size∗∗∗2 or 3∗4Conv_2_filter8328∗∗5Conv_2_kernal∗∗∗2 or 3∗6Conv_2_pool_size∗∗∗2 or 3∗7CNN-LSTM1, CNN-LSTM2, CNN-GRU1, CNN-GRU2, LSTM, GRU, or MLP layer 1 units5305∗∗8Dropout 10.00.30.1∗0.29CNN-LSTM1, CNN-LSTM2, CNN-GRU1, CNN-GRU2, LSTM, GRU, or MLP layer 2 units5305∗∗10Dropout 20.00.30.1∗0.211Learning rate∗∗∗1e-2, 1e-3 or 1e-4∗12Number of epochs1010010∗∗13Number of batch sizes1010010∗∗∗∗Value ranges or choices for optimization by Keras tuner: (objective = “validation loss,” max trials = 20, and executions per trial = 3). ∗Not applicable.


### 5.2. Performance Measures

A wide variety of evaluation metrics are listed in the literature [49]. However, the popular ones are mean error (ME), coefficient of determination (R2), root mean square error (RMSE), mean absolute error (MAE), mean percentage error (MPE), mean absolute percentage error (MAPE), mean absolute scaled error (MASE), and Nash–Sutcliffe efficiency (NSE). This study used different input and model variability conditions. Hence, to concisely measure the analysis output and present the result, we applied the following top three standard performance evaluation criteria that can also have the potential to capture the extreme streamflow time series values effectively [50].Coefficient of determination (R2):(7)R2=n∑​Qobs∗Qsim−∑​Qobs∗∑​Qsimn∑​Qobs2−∑​Qobs21/2∗n∑​Qsim2−∑​Qsim2.Root mean square error (RMSE):(8)RMSE=∑n=1NQ0bst−Qsimt2N.Mean absolute error (MAE):(9)MAE=1n∑i=1nQ0bst−Qsimt,where Qobs = discharge observed, Qsim = discharge simulated, and n = number of observations. The range of R2 lies between 0 and 1, representing, respectively, no correlation and a perfect correlation between observed and simulated values, whereas smallest RMSE and MAE scores or values close to zero direct to the best model performance.


## 5.1. Hyperparameter Optimization

Tuning machine learning model hyperparameters is critical. Varying hyperparameter values often results in models with significantly different performances [47]. The models applied in this study mainly contain two types of hyperparameters: constant hyperparameters that are not altered through the optimization process and variable hyperparameters. Adam optimizer is applied under the category of the constant hyperparameters because of its efficiency and easiness to implementation that requires minimum memory and is commonly suited in different problems [48]. In this category, rectified linear unit (Relu) was used as an activation function, and mean squared error (MSE) was used as a loss function.In contrast, the second type of changing hyperparameters is optimized by Keras tuner, and hyperparameter choices or value ranges for optimization are set using different trials. We also considered our PC capacity (processor: Intel(R) Core (TM) i7-6500U CPU 2.50 GHz and RAM: 8 gigabytes) with Windows 10 operating system. Hyperparameters are optimized with 20 trials, and since deep learning networks have different training and validation plots for each run, we decided to repeat the iteration three times.All hyperparameter ranges or choices are listed in Table4. CNN-LSTM1 and CNN-GRU1 models used hyperparameter values from numbers 1 to 13 for optimization (Table 4), while we omitted 4, 5, and 6 for CNN-LSTM2 and CNN-GRU2. The remaining deep learning models MLP, LSTM, and GRU used a list of hyperparameters from numbers 7 to 13. Finally, each optimized hyperparameter is used for each training and testing experiment. Moreover, the train and test traces from each run can be plotted to give a more robust idea of the behaviour of the model to inspect overfitting and underfitting issues.Table 4
Model hyperparameter choices or value ranges for optimization by Keras tuner.
NoHyperparametersValue ranges∗∗ChoicesDefaultMinMaxStep1Conv_1_filter8328∗∗2Conv_1_kernal∗∗∗2 or 3∗3Conv_1_pool_size∗∗∗2 or 3∗4Conv_2_filter8328∗∗5Conv_2_kernal∗∗∗2 or 3∗6Conv_2_pool_size∗∗∗2 or 3∗7CNN-LSTM1, CNN-LSTM2, CNN-GRU1, CNN-GRU2, LSTM, GRU, or MLP layer 1 units5305∗∗8Dropout 10.00.30.1∗0.29CNN-LSTM1, CNN-LSTM2, CNN-GRU1, CNN-GRU2, LSTM, GRU, or MLP layer 2 units5305∗∗10Dropout 20.00.30.1∗0.211Learning rate∗∗∗1e-2, 1e-3 or 1e-4∗12Number of epochs1010010∗∗13Number of batch sizes1010010∗∗∗∗Value ranges or choices for optimization by Keras tuner: (objective = “validation loss,” max trials = 20, and executions per trial = 3). ∗Not applicable.


## 5.2. Performance Measures

A wide variety of evaluation metrics are listed in the literature [49]. However, the popular ones are mean error (ME), coefficient of determination (R2), root mean square error (RMSE), mean absolute error (MAE), mean percentage error (MPE), mean absolute percentage error (MAPE), mean absolute scaled error (MASE), and Nash–Sutcliffe efficiency (NSE). This study used different input and model variability conditions. Hence, to concisely measure the analysis output and present the result, we applied the following top three standard performance evaluation criteria that can also have the potential to capture the extreme streamflow time series values effectively [50].Coefficient of determination (R2):(7)R2=n∑​Qobs∗Qsim−∑​Qobs∗∑​Qsimn∑​Qobs2−∑​Qobs21/2∗n∑​Qsim2−∑​Qsim2.Root mean square error (RMSE):(8)RMSE=∑n=1NQ0bst−Qsimt2N.Mean absolute error (MAE):(9)MAE=1n∑i=1nQ0bst−Qsimt,where Qobs = discharge observed, Qsim = discharge simulated, and n = number of observations. The range of R2 lies between 0 and 1, representing, respectively, no correlation and a perfect correlation between observed and simulated values, whereas smallest RMSE and MAE scores or values close to zero direct to the best model performance.


## 6. Results

Streamflow simulation result with the proposed seven deep learning architectures, different input time window series, two climatic regions, two input combinations, and three average rolling time windows is presented in Tables5, 6, and 7. Regardless of the combination of these conditions, the CNN-GRU model showed promising results in most of these scenarios (Tables 5 and 7). The highest scores are presented here.(1)
In daily streamflow simulation for Borkena station, CNN-GRU1 scored 7.94, 3.66, 0.85, and 0.63, and for UTRB station, CNN-GRU2 scored 45.61, 21.79, 0.57, and 0.64 for RMSE, MAE, R2, and training time per epoch, respectively.(2)
In weekly rolled streamflow simulation for Borkena station, CNN-LSTM2 scored 7.33, 3.86, 0.87, and 0.52, and for UTRB station, GRU scored 25.21, 14.83, 0.77, and 5.56 for RMSE, MAE, R2, and training time per epoch, respectively.(3)
In monthly rolled streamflow simulation, the CNN-GRU2 model showed high performance with 5.15, 3.18, 0.92, and 0.78 scores for Borkena station and 17.98, 12.99, 0.83, and 0.71 for UTRB station, which are RMSE, MAE, R2, and training time per epoch, respectively.Table 5
Daily streamflow simulation: performance comparison of the proposed models for different input variables and climatic conditions.
Model 1BorkenaUTRBP + TminPP + Tmin + TmaxPRMR2TRMR2TRMR2TRMR2TMATMATMATMATSEPSEPSEPSEPEE∗ (sec)EE∗ (sec)EE∗ (sec)EE∗ (sec)MLP9.915.010.770.899.384.630.790.6349.1122.740.490.7856.5728.140.330.41GRU8.784.370.823.617.943.640.853.3246.6320.890.552.6151.0926.740.453.39LSTM8.414.090.832.359.654.870.782.9248.6422.790.513.8648.5925.000.515.98CNN-LSTM18.094.070.840.468.574.670.820.4151.2022.950.451.1956.1626.550.340.57CNN-LSTM27.994.090.850.729.144.500.800.4545.3821.850.570.8251.5725.840.441.85CNN-GRU17.943.660.850.638.324.090.830.8655.0623.490.371.1652.4224.980.430.83CNN-GRU29.074.190.801.018.434.260.830.2845.6121.790.570.6449.9625.380.480.68∗TTPE (training time per epoch). The bold values indicate the highest performance score.Table 6
Weekly rolled streamflow simulation: performance comparison of the proposed models for different input variables and climatic conditions.
Model 2BorkenaUTRBP + TminPP + Tmin + TmaxPRMR2TRMR2TRMR2TRMR2TMATMATMATMATSEPSEPSEPSEPEE∗ (sec)EE∗ (sec)EE∗ (sec)EE∗ (sec)MLP8.114.290.840.237.334.190.870.2233.0120.010.600.7438.0325.170.470.79GRU7.593.710.862.047.154.130.872.4325.2114.830.775.5631.3919.260.6416.79LSTM8.414.010.822.987.933.910.841.2731.0718.870.653.5531.0719.490.652.69CNN-LSTM17.904.090.850.787.724.250.850.6328.0417.330.710.9334.5721.920.570.62CNN-LSTM27.333.860.870.527.634.250.860.5528.4516.660.711.1435.0421.770.551.56CNN-GRU17.833.940.850.447.914.310.850.5030.5718.010.662.3235.1422.580.550.63CNN-GRU28.734.610.810.438.434.350.820.9727.8116.990.724.3733.7623.010.591.01∗TTPE (training time per epoch). The bold values indicate the highest performance score.Table 7
Monthly rolled streamflow simulation: performance comparison of the proposed models for different input variables and climatic conditions.
Model 2BorkenaUTRBP + TminPP + Tmin + TmaxPRMR2TRMR2TRMR2TRMR2TMATMATMATMATSEPSEPSEPSEPEE∗ (sec)EE∗ (sec)EE∗ (sec)EE∗ (sec)MLP6.684.370.870.585.573.800.910.4120.2413.840.780.4428.7921.050.560.41GRU5.153.520.911.625.223.060.923.3120.7914.300.7716.6326.4720.080.634.70LSTM5.553.490.912.755.763.510.902.5121.4915.110.764.1532.2924.470.455.09CNN-LSTM16.054.420.890.985.583.400.910.5821.5314.870.761.2927.4821.190.600.42CNN-LSTM25.363.170.921.416.874.050.861.4419.0713.530.810.7027.7920.900.590.42CNN-GRU15.763.620.900.525.773.560.900.6919.3113.780.804.8728.6721.070.573.08CNN-GRU25.363.250.920.625.153.180.920.7817.9812.990.830.7127.7720.360.591.22∗TTPE (training time per epoch). The bold values indicate the highest performance score.Moreover, from the proposed four hybrid models, CNN-GRU2 or the model designed by a single 1D CNN layer showed the highest promising result on trial model 1(UTRB) and model 3, as shown in Tables 5 and 7. In contrast, GRU on model 2 (UTRB), CNN-LSTM2 on model 2 (Borkena), and CNN-GRU1 on model 1 (Borkena) shared the second-highest promising result. Streamflow simulation with the CNN-GRU2 model generally showed the highest performance than the other tested hybrid deep learning models and state-of-the-art LSTM, GRU, and MLP models. In line with our objectives, the result is discussed with different variability conditions in the following paragraphs.


### 6.1. Climatic Region Variability

Testing models in different climatic conditions with historical data will likely provide robust deep learning models for streamflow simulation in the future [51]. Hence, this research also tested different models in two climatic regions, and irrespective of climatic and time window variation, the CNN-GRU model displayed the highest scores on tested case study areas.


### 6.2. Input Combination Variability

Input combination, minimum temperature (Tmin) with precipitation (P), does not show significant performance increment in the Borkena station (Tables5 and 6). In some scenarios, adopting P only as input increases the performance of the model (Table 7). In contrast, for UTRB, streamflow simulation with all input variables or Tmin, Tmax, and P showed significant performance increments (Table 7).


### 6.3. Average Rolling Time Window Variability

Streamflow simulation without rolling daily time series data had deficient performance compared to monthly rolled average time series. This could be because the time series noise in UTRB is visible compared to that in Borkena station. As a result, performance increment from daily to monthly rolled window models is much higher in UTRB than in Borkena station.Generally, the monthly rolled time window with the CNN-GRU2 model showed the top performance results in both stations (Table7). The corresponding training and test loss functions of this optimized high score hybrid model for both stations are displayed in Figure 9. Consequently, Figure 10 compares the true values and predicted values of this model. The optimized hybrid model boosts the performance score and lowers the training time per epoch much better than GRU and LSTM models. This model, input feature, and Keras tuner optimized hyperparameter values for both stations with its MSE score are presented in Tables 8 and 9. Moreover, the internal network structures of these models are also shown in Figures 11 and 12, which display the model input and output parameter matrices for each layer.Figure 9
Training and test loss function of the optimized high score hybrid model. (a) CNN-GRU2 model for Borkena Station. (b) CNN-GRU2 model for UTRB Station.
(a)(b)Figure 10
Comparison of true values and predicted values of the optimized high score hybrid model. (a) CNN-GRU2 model for Borkena Station. (b) CNN-GRU2 model for UTRB Station.
(a)(b)Table 8
Best hybrid model type, input feature, and Keras tuner optimized hyperparameter values for Borkena station with its MSE score.
Hyperparameters:CNN-GRU2Monthly rolled PConv_1_filter24Conv_1_kernal2Conv_1_pool_size3GRU_l1_units15Dropout10.1GRU_l2_units20Dropout20.2Learning rate0.0001Number of epochs80Number of batch sizes20Score (MSE)0.083Table 9
Best hybrid model type, input features, and Keras tuner optimized hyperparameter values for UTRB station with its MSE score.
Hyperparameters:CNN-GRU2Monthly rolled P,Tmin and TmaxConv_1_filter8Conv_1_kernal2Conv_1_pool_size2GRU_l1_units20Dropout10.3GRU_l2_units30Dropout20.2Learning rate0.0001Number of epochs60Number of batch sizes40Score (MSE)0.193Figure 11
Internal network structure of the optimized high score hybrid CNN-GRU2 model for Borkena Station.Figure 12
Internal network structure of the optimized high score hybrid CNN-GRU2 model for UTRB Station.


## 6.1. Climatic Region Variability

Testing models in different climatic conditions with historical data will likely provide robust deep learning models for streamflow simulation in the future [51]. Hence, this research also tested different models in two climatic regions, and irrespective of climatic and time window variation, the CNN-GRU model displayed the highest scores on tested case study areas.


## 6.2. Input Combination Variability

Input combination, minimum temperature (Tmin) with precipitation (P), does not show significant performance increment in the Borkena station (Tables5 and 6). In some scenarios, adopting P only as input increases the performance of the model (Table 7). In contrast, for UTRB, streamflow simulation with all input variables or Tmin, Tmax, and P showed significant performance increments (Table 7).


## 6.3. Average Rolling Time Window Variability

Streamflow simulation without rolling daily time series data had deficient performance compared to monthly rolled average time series. This could be because the time series noise in UTRB is visible compared to that in Borkena station. As a result, performance increment from daily to monthly rolled window models is much higher in UTRB than in Borkena station.Generally, the monthly rolled time window with the CNN-GRU2 model showed the top performance results in both stations (Table7). The corresponding training and test loss functions of this optimized high score hybrid model for both stations are displayed in Figure 9. Consequently, Figure 10 compares the true values and predicted values of this model. The optimized hybrid model boosts the performance score and lowers the training time per epoch much better than GRU and LSTM models. This model, input feature, and Keras tuner optimized hyperparameter values for both stations with its MSE score are presented in Tables 8 and 9. Moreover, the internal network structures of these models are also shown in Figures 11 and 12, which display the model input and output parameter matrices for each layer.Figure 9
Training and test loss function of the optimized high score hybrid model. (a) CNN-GRU2 model for Borkena Station. (b) CNN-GRU2 model for UTRB Station.
(a)(b)Figure 10
Comparison of true values and predicted values of the optimized high score hybrid model. (a) CNN-GRU2 model for Borkena Station. (b) CNN-GRU2 model for UTRB Station.
(a)(b)Table 8
Best hybrid model type, input feature, and Keras tuner optimized hyperparameter values for Borkena station with its MSE score.
Hyperparameters:CNN-GRU2Monthly rolled PConv_1_filter24Conv_1_kernal2Conv_1_pool_size3GRU_l1_units15Dropout10.1GRU_l2_units20Dropout20.2Learning rate0.0001Number of epochs80Number of batch sizes20Score (MSE)0.083Table 9
Best hybrid model type, input features, and Keras tuner optimized hyperparameter values for UTRB station with its MSE score.
Hyperparameters:CNN-GRU2Monthly rolled P,Tmin and TmaxConv_1_filter8Conv_1_kernal2Conv_1_pool_size2GRU_l1_units20Dropout10.3GRU_l2_units30Dropout20.2Learning rate0.0001Number of epochs60Number of batch sizes40Score (MSE)0.193Figure 11
Internal network structure of the optimized high score hybrid CNN-GRU2 model for Borkena Station.Figure 12
Internal network structure of the optimized high score hybrid CNN-GRU2 model for UTRB Station.


## 7. Conclusions

This study showed a comparative analysis of different hybrid deep learning algorithms with state-of-the-art machine learning models for one-step daily streamflow simulation at two river basins or subcatchment stream flow outlets. The proposed algorithms for this study are CNN-LSTM and CNN-GRU hybrid deep learning models, each model having one or two 1D CNN layers with the classic MLP, LSTM, and GRU models. This study conducted a series of experiments to observe the performance variation of the proposed models by introducing different input combinations, rolling time windows, and climatic conditions for streamflow simulation. The following list of points will summarize the significant findings of this study.(i)
CNN-GRU2 with one 1D CNN layer showed the best simulation performance reporting the lowest RMSE, MAE, and R2 out of all models in both case study areas. Such results dictate that the performance of the selected architectures is irrespective of the climatic characteristics of the basins.(ii)
Combining temperature data with precipitation as input and inserting to the proposed models had minimum performance increment in Borkena station compared to UTRB case study area, which clearly showed that temperature data scarcity has more performance loss implication in UTRB station. On the other hand, the Borkena station has significant natural streamflow variability than UTRB, which is also reflected in the model results. This implies the consideration of catchment response before any deep learning model applications.(iii)
Rolling the time window of input and output time series for streamflow simulation using the proposed models considerably increases performance in the UTRB than in the Borkena station.(iv)
The analysis results also showed that training time per epoch for the hybrid deep learning models is much lower than that of GRU and LSTM models.Deep learning models usually require massive datasets, and their performance drops with small to medium datasets. However, from this case study, acceptable results and considering hybrid models’ hyperparameters sensitivity and complexity, future research may further design optimized configurations. Moreover, they can test these hybrid models for long-term streamflow simulation in ephemeral, seasonal, and perennial river systems and other fields of study. Our future research will try to synchronize the highly performed hybrid deep learning models in this study with remote sensing datasets for the problem we experience in the ungauged catchments.



---

*Source: 5172658-2021-10-27.xml*