# Design of Deep Belief Networks for Short-Term Prediction of Drought Index Using Data in the Huaihe River Basin

Authors: Junfei Chen; Qiongji Jin; Jing Chao
Journal: Mathematical Problems in Engineering
 (2012)

---

## Abstract

With the global climate change, drought disasters occur frequently.  Drought prediction is an important content for drought disaster management, planning and management of water resource systems of a river basin. In this study, a short-term drought prediction model based on deep belief networks (DBNs) is proposed to predict the time series of different time-scale standardized precipitation index (SPI). The DBN model is applied to predict the drought time series in the Huaihe River Basin, China. Compared with BP neural network, the DBN-based drought prediction model has shown better predictive skills than the BP neural network for the different time-scale SPI. This research can improve drought prediction technology and be helpful for water resources managers and decision makers in managing drought disasters.


---

## Body

## 1. Introduction

With the global environmental degradation and water resource shortages, droughts are becoming increasingly eye catching and have aroused the attention of many countries and regions. Drought is considered the most complex but least understood of all natural hazards, affecting more people than any other disasters [1]. In recent years, drought disasters continuously happened and caused serious impact on production and life in China. The losses caused by drought ranked the first in all natural hazards in China [2]. For example, in the extreme drought in Southwest China during 2009 to 2010, five provinces and cities suffered droughts which have seriously threatened people’s life and economic production activities. The Chinese northern region also suffered severe drought in 2011. Long and severe droughts have direct impacts on industrial production, people’s lives, and ecological environment and even lead to desertification and other natural disasters. Droughts have become serious constraints to the sustainable development of Chinese society and economy [3, 4]. Drought prediction is an important content in the planning and management of water resource systems of a river basin. How to effectively monitor and forecast droughts has become the research focus, which can help to take effective strategies and measures to mitigate the damages of droughts.There are some forecasting methods used in drought prediction fields. Lohani and Loganathan used a nonhomogeneous Markov chain model to characterize the stochastic behavior of drought, and an early-warning system in the form of a decision tree enumerating is proposed for drought management [5]. Jia et al. established a grey-time series combined method (GTCM) to predict annual precipitation of Huangcun Meteorological Observation, Daxing county, Beijing [6]. Yang et al. proposed a chaotic Bayesian method based on multiple criteria decision making to forecast nonlinear hydrological time series, which can be applied in drought forecast [7]. The predictability of drought severity from spatiotemporal varying indices of large-scale climate phenomena was studied by integrating linear and nonlinear statistical data models, and the model was used for the Murray-Darling Basin (MDB) in Australia [8]. Meteorological droughts were characterized using the standardized precipitation index (SPI) developed by McKee et al. [9]. Drought classes based on standardized precipitation index (SPI) values were derived by Markov chain model in Alentejo, Southern Portugal [10]. Peng et al. used weighted Markov chain to predict the future drought index, weighted by the standardized self-coefficients. The drought indexes of Nanjing city from 1959 to 2004 were a specific application with this method and satisfactory results were obtained [11]. SPI is calculated from monthly precipitation data collected from 36 weather stations in Guanzhong plain and Weibei tablelands, and the Markov chain model with weights was applied to predict SPI drought intensity by using standardized self-coefficients as weights [12]. The vegetation temperature condition index (VTCI) based on remote sensing data is used for drought monitoring. The ARIMA models were developed to simulate the VTCI series and be used in Guanzhong Plain in China [13]. The loglinear modeling for three-dimensional contingency tables was used for short-term prediction of drought severity classes. The results show that three-dimensional loglinear modeling of monthly drought class transitions is able to capture the trends for both drought initiation, establishment, and drought dissipation [14]. Mishra and Desai compared linear stochastic model (ARIMA/SARIMA), recursive multistep neural network (RMSNN), and direct multisteps neural network (DMSNN) for drought forecasting by using standardized precipitation index (SPI) series as drought index in the Kansabati River Basin in India [15]. Traditionally, forecasting research and practice has been dominated by conventional statistical methods. Recently, the study of long range or long memory has received many attentions in forecasting. Hurst developed a test for long-range dependence and found significant long-term correlations among fluctuations in the Nile’s outflows and described these correlations in terms of power laws [16]. Mathematical models with long-range dependence were first introduced to statistics by Mandelbrot and his workers [17–19]. Long-range dependence is often encountered in practice, not only in hydrology, geophysics, and finance, but also in all fields of statistical applications [20–24]. Pelletier and Turcotte present power spectra of time-series data for tree ring width chronologies, atmospheric temperatures, river discharges, and precipitation averaged over hundreds of stations worldwide. They thought that long-range persistence can have a dramatic effect on the likelihood of severe hydrologic drought and computed recurrence intervals for droughts of different magnitudes, durations, and coefficients of variation [25]. Radziejewski and Kundzewicz computed fractal dimensions of crossings of Warta flows by using a novel variant of the box-counting method, and spectral properties are compared between the time series of flows [26]. Li et al. computed long-range dependence (LRD) of sea level and thought that sea level is multiscaled and heavy tailed [27].Recently, deep belief networks (DBNs) are proposed by Hinton. The DBN is a probabilistic generative model, the bottom layer is observable, and the multiple hidden layers are created by stacking multiple restricted Boltzmann machines (RBMs) on top of each other [28]. Hinton et al. derived a way to perform fast, greedy learning of deep belief networks (DBN) one layer at a time, with the top two layers forming an undirected bipartite graph [29]. DBNs and restricted Boltzmann machines (RBMs) have already been applied successfully to solve many problems [30]. Lee et al. present a convolutional deep belief networks and are used to scale the realistic image sizes [31]. A novel text classification approach based on deep belief networks is proposed, and the proposed method outperforms traditional classifier based on support vector machine [32]. Zhou et al. present a discriminative deep belief networks (DDBNs) to address the image classification problem with limited labeled data. Experiments on the artificial dataset and real image datasets show that DDBN outperforms most semisupervised algorithms [33]. Chao et al. proposed a deep belief network (DBN) to forecast the foreign exchange rate. In their experiments, both British pound/US dollar and Indian rupee/US dollar exchange rates are forecasted, and the results show that deep belief networks (DBNs) achieve better performance than feed-forward neural networks [34]. Deep learning techniques have also been shown to perform significantly better than other techniques for problems such as image classification and handwriting analysis [31].In this paper, we propose a deep belief network (DBN) for short-term prediction of drought index. The aims of this study are to present and evaluate the performance of DBN model as a drought prediction method. This model was applied to forecast drought index using standardized precipitation index (SPI) series in the Huaihe River Basin, China. The results are compared and analyzed with BP neural network for demonstration of the validity of the DBN model. The remainder of the paper is organized as follows. In Section2, the standardized precipitation index (SPI) and BP neural network are introduced, and the deep belief networks (DBN) model for drought index prediction is proposed. In Section 3, a case is studied, and discussions are arranged. Finally in Section 4, the main conclusions and a discussion for future work are given.


## 2. Methodology

### 2.1. Standardized Precipitation Index (SPI)

The SPI was formulated by Mckee et al. of the Colorado Climate Center in 1993. The purpose is to assign a single numeric value to the precipitation which can be compared across regions with markedly different climates [11]. The SPI is an index based on the probability of precipitation for any time scale. Technically, the SPI is the number of standard deviations that the observed value would deviate from the long-term mean, for a normally distributed random variable. The SPI can be computed for different time scales and can provide early warning of drought and help assess drought severity. The SPI is a probability index that considers only precipitation, while Palmer’s indices are water balance indices that consider water supply (precipitation), demand (evapotranspiration), and loss (runoff). So, SPI is less complex than PDSI [35]. Now, the standardized precipitation index (SPI) is widely accepted and used throughout the world [36]. The computing procedure of the SPI value is as follows [37, 38].Assuming that a precipitation series of some time scale isx, then its probability density function of Γ distribution is expressed as(2.1)f(x)=1βγΓ(γ)xγ-1e-x/β,x>0,
where Γ(γ) is a gamma function and Γ(γ)=∫0∞xγ-1e-xdx. β and γ are the shape parameter and the scale parameter, respectively, and β>0, γ>0. The precipitation value x>0.The shape and scale parameters can be estimated by the maximum likelihood method as follows:(2.2)γ̂=1+1+4A/34A,β̂=x̅γ̂,
where A=ln(x̅)-(1/n)∑i=1nlnxi,n stands for the number of precipitation observations, xi are the samples of the precipitation data, and x̅ is the mean of these samples.The gamma distribution is not defined forx=0; however, the actual precipitation can be 0. Therefore, cumulative probability of precipitation for a certain time scale can be calculated using the following formula [38, 39]:(2.3)H(x)=u+(1-u)F(x),
where F(x)=(1/Γ(γ̂))∫0xtγ̂-1e-tdt and t=x/β̂. u is the probability of zero precipitation and can be calculated as m/n. m is the total number of precipitation series, and n is the number of zeros in the precipitation series.The cumulative probability,H(x), is then transformed to the standard normal random variable with mean as zero and variance as one. Following Edwards and Mckee [40] and Hughes and Saunders [41], SPI can be obtained as follows:
(2.4)SPI={-(t-c0+c1t+c2t21+d1t+d2t2+d3t3)for0<H(x)≤0.5,t-c0+c1t+c2t21+d1t+d2t2+d3t3for0.5<H(x)<1,
where
(2.5)t={ln⁡⁡(1(H2(x)))for0<H(x)≤0.5,ln⁡⁡(1(1-H(x))2)for0.5<H(x)<1.In (2.4), the ci and di are parameters during the computing process and c0=2.515517, c1=0.802853, c2=0.010328, d1=1.432788, d2=0.189269, and d3=0.001308.According to SPI, drought can be classified. When the value of SPI is continuously negative, a drought event occurs. The event ends when the SPI becomes positive.


### 2.2. Backpropagation Neural Network (BPNN)

The BP neural network is a kind of multilayer feed-forward networks with training by error backpropagation algorithm [42]. It is a kind of supervised learning neural network, the principle behind which involves using the steepest gradient descent method to reach any small approximation. A general model of the BP neural network has a structure as described in Figure 1.Figure 1
The structure of BP neural network.In Figure1, there are three layers contained in BP: input layer, hidden layer, and output layer. Two nodes of each adjacent layer are directly connected, which is called a link. Each link has a weighted value presenting the relational degree between two nodes. The algorithm of BP neural network is to input the training samples from the input layer and then obtain the calculation output through the operation of corresponding thresholds, functions, and connection weights between nodes [42, 43]. The node function has usually selected S-type function as follows:(2.6)f(x)=11+e-x/Q.
The Q in the equation is a Sigmoid parameter which is the form of adjusted activation function, and the specific algorithm is introduced in [44]. The output error is obtained by the comparison between the calculation output and the sample output. If the error does not meet the requirements, the network weights and thresholds usually are adjusted along the negative gradient direction of network error and finally reach the minimum network error [45]. The number of hidden layer nodes is firstly determined by employing an empirical formula in the design stage and finally adjusted by comparing the efficiencies of different numbers of hidden layer nodes in neural network training stage [46].


### 2.3. Deep Belief Networks

A deep belief network (DBN) is a generative model with an input layer and an output layer, separated by many layers of hidden stochastic units. The multilayer neural network can efficiently be trained by composing RBMs using the feature activations of one layer as the training data for the next. Figure2 shows an example of a DBN structure [28].Figure 2
A DBN structure with L hidden layers.Usually a DBN consists of two kinds of different layers. They are visible layer and hidden layer. Visible layers contain input nodes and output nodes, and hidden layers contain hidden nodes. Hinton et al. proposed a greedy layerwise unsupervised learning algorithm for DBNs which is based on sequence training with restricted Boltzmann machines (RBMs) [28, 34]. A restricted Boltzmann machine (RBM) is composed of two different layers of units, with weighted connection between them. It consists of one layer of visible nodes (neurons) and one layer of hidden units. Figure 3 shows an RBM structure. Nodes in each layer have no connections between them and are connected to all other units in another layer. Connections between nodes are bidirectional and symmetric. Restricted Boltzmann machines (RBMs) have been used as generative models of many different types of data including labeled or unlabeled images windows of mel-cepstral coefficients that represent speech, and so on. Their most important use is as learning modules that are composed to form deep belief nets [28].Figure 3
A RBM structure.Letvi and hj represent the states of visible node i and hidden node j, respectively. For binary state nodes, that is, vi and hj∈{0,1}, the state of hj is set to 1 with probabilities [47]:(2.7)phj=p(hj=1∣ν)=σ(bj+∑iwijνi),
where σ(x) is the logistic sigmoid function 1/(1+exp(-x)), bj is the bias of j, and νi is the binary state. wij is the weight between νi and hj. After binary states have been chosen for the hidden units, then set the state of νi to be 1 with probability(2.8)pνi=p(νi=1∣h)=σ(bi+∑jwijhj).The training process of the RBM is described as follows. Firstly, a training sample is presented to the visible nodes, and the{νi} is obtained. Then the hidden nodes state that {hj} are sampled according to probabilities in (2.7). This process is repeated once more to update the visible and then the hidden nodes, and the one-step “reconstructed” states νi′ and hj′ are obtained. The update in a weight is given as follows:(2.9)Δwij=η(〈νihj〉-〈νi′hj′〉),
where η is the learning rate, and 〈·〉 refers to the expectation of the training data.A continuous restricted Boltzmann machine (CRBM) is considered by Chao et al. [34] and Chen and Murray [48]. Suppose the inputs nodes with state {si}, then the output nodes sj can be computed as follows:(2.10)sj=φj(∑iwijsi+σ⋅Nj(0,1)),
where φj(x) is a sigmoid function with lower and upper asymptotes at θL and θH, φj(xj)=θL+(θH-θL)·(1/(1+e-ajxj)). Nj(0,1) represents a unit Gaussian. σ is a constant, and parameter aj is a “noise-control” parameter which controls the slope of the sigmoid function [49]. The update equations for wij and aj are(2.11)Δwij=ηw(〈sisj〉-〈si′sj′〉),Δaj=ηaaj2(〈sj2〉-〈sj′2〉),
where ηw and ηa represent the learning rates, sj′ denotes the one-step sampled state of node j, and 〈·〉 refers to the expectation of the training data. We train sequentially as many RBMs as the number of hidden layers in the DBN to construct a DBN model. We adopt the learning algorithm according to [28, 34, 50]. The method of stacking CRBMs makes it possible to train many layers of hidden units efficiently and is one of the most common deep learning strategies. As each new layer is added, the overall generative model gets better. This process of learning is continued until a prescribed number of hidden layers in the DBN have been trained. In order to apply DBN model to drought prediction using SPI series, the DBN model with two hidden layers is selected in this paper. The main steps using DBN model for drought index prediction are as follows.Step 1.
Compute the different time-scale SPI series by precipitation data.
The different time-scale SPI series are computed by precipitation data by the description method in Section2.1, and different time-scale SPI series are obtained.Step 2.
Normalize the SPI series by formula (2.12) as follows:
(2.12)SPI′=SPI-SPImin⁡SPImax⁡-SPImin⁡,
where SPI′ and SPI represent the normalized and original SPI data, respectively. The SPImin represents the minimum value of the corresponding SPI series, and SPImax represents the maximum value of the corresponding SPI series.Step 3.
Determine the optimal network structure by experiments.
Determine the number of input nodes, the numbers of the first hidden and second hidden nodes, and weight coefficients by learning algorithm. The data of SPI series are split into two parts. The first part is used as a training sample, and the rest is used as a testing sample. During the training process, the network structures for different time-scale SPI series are determined according to the criterion of smallest RMSE and MAE.Step 4.
Forecast drought index based on DBN model and results analysis.


## 2.1. Standardized Precipitation Index (SPI)

The SPI was formulated by Mckee et al. of the Colorado Climate Center in 1993. The purpose is to assign a single numeric value to the precipitation which can be compared across regions with markedly different climates [11]. The SPI is an index based on the probability of precipitation for any time scale. Technically, the SPI is the number of standard deviations that the observed value would deviate from the long-term mean, for a normally distributed random variable. The SPI can be computed for different time scales and can provide early warning of drought and help assess drought severity. The SPI is a probability index that considers only precipitation, while Palmer’s indices are water balance indices that consider water supply (precipitation), demand (evapotranspiration), and loss (runoff). So, SPI is less complex than PDSI [35]. Now, the standardized precipitation index (SPI) is widely accepted and used throughout the world [36]. The computing procedure of the SPI value is as follows [37, 38].Assuming that a precipitation series of some time scale isx, then its probability density function of Γ distribution is expressed as(2.1)f(x)=1βγΓ(γ)xγ-1e-x/β,x>0,
where Γ(γ) is a gamma function and Γ(γ)=∫0∞xγ-1e-xdx. β and γ are the shape parameter and the scale parameter, respectively, and β>0, γ>0. The precipitation value x>0.The shape and scale parameters can be estimated by the maximum likelihood method as follows:(2.2)γ̂=1+1+4A/34A,β̂=x̅γ̂,
where A=ln(x̅)-(1/n)∑i=1nlnxi,n stands for the number of precipitation observations, xi are the samples of the precipitation data, and x̅ is the mean of these samples.The gamma distribution is not defined forx=0; however, the actual precipitation can be 0. Therefore, cumulative probability of precipitation for a certain time scale can be calculated using the following formula [38, 39]:(2.3)H(x)=u+(1-u)F(x),
where F(x)=(1/Γ(γ̂))∫0xtγ̂-1e-tdt and t=x/β̂. u is the probability of zero precipitation and can be calculated as m/n. m is the total number of precipitation series, and n is the number of zeros in the precipitation series.The cumulative probability,H(x), is then transformed to the standard normal random variable with mean as zero and variance as one. Following Edwards and Mckee [40] and Hughes and Saunders [41], SPI can be obtained as follows:
(2.4)SPI={-(t-c0+c1t+c2t21+d1t+d2t2+d3t3)for0<H(x)≤0.5,t-c0+c1t+c2t21+d1t+d2t2+d3t3for0.5<H(x)<1,
where
(2.5)t={ln⁡⁡(1(H2(x)))for0<H(x)≤0.5,ln⁡⁡(1(1-H(x))2)for0.5<H(x)<1.In (2.4), the ci and di are parameters during the computing process and c0=2.515517, c1=0.802853, c2=0.010328, d1=1.432788, d2=0.189269, and d3=0.001308.According to SPI, drought can be classified. When the value of SPI is continuously negative, a drought event occurs. The event ends when the SPI becomes positive.


## 2.2. Backpropagation Neural Network (BPNN)

The BP neural network is a kind of multilayer feed-forward networks with training by error backpropagation algorithm [42]. It is a kind of supervised learning neural network, the principle behind which involves using the steepest gradient descent method to reach any small approximation. A general model of the BP neural network has a structure as described in Figure 1.Figure 1
The structure of BP neural network.In Figure1, there are three layers contained in BP: input layer, hidden layer, and output layer. Two nodes of each adjacent layer are directly connected, which is called a link. Each link has a weighted value presenting the relational degree between two nodes. The algorithm of BP neural network is to input the training samples from the input layer and then obtain the calculation output through the operation of corresponding thresholds, functions, and connection weights between nodes [42, 43]. The node function has usually selected S-type function as follows:(2.6)f(x)=11+e-x/Q.
The Q in the equation is a Sigmoid parameter which is the form of adjusted activation function, and the specific algorithm is introduced in [44]. The output error is obtained by the comparison between the calculation output and the sample output. If the error does not meet the requirements, the network weights and thresholds usually are adjusted along the negative gradient direction of network error and finally reach the minimum network error [45]. The number of hidden layer nodes is firstly determined by employing an empirical formula in the design stage and finally adjusted by comparing the efficiencies of different numbers of hidden layer nodes in neural network training stage [46].


## 2.3. Deep Belief Networks

A deep belief network (DBN) is a generative model with an input layer and an output layer, separated by many layers of hidden stochastic units. The multilayer neural network can efficiently be trained by composing RBMs using the feature activations of one layer as the training data for the next. Figure2 shows an example of a DBN structure [28].Figure 2
A DBN structure with L hidden layers.Usually a DBN consists of two kinds of different layers. They are visible layer and hidden layer. Visible layers contain input nodes and output nodes, and hidden layers contain hidden nodes. Hinton et al. proposed a greedy layerwise unsupervised learning algorithm for DBNs which is based on sequence training with restricted Boltzmann machines (RBMs) [28, 34]. A restricted Boltzmann machine (RBM) is composed of two different layers of units, with weighted connection between them. It consists of one layer of visible nodes (neurons) and one layer of hidden units. Figure 3 shows an RBM structure. Nodes in each layer have no connections between them and are connected to all other units in another layer. Connections between nodes are bidirectional and symmetric. Restricted Boltzmann machines (RBMs) have been used as generative models of many different types of data including labeled or unlabeled images windows of mel-cepstral coefficients that represent speech, and so on. Their most important use is as learning modules that are composed to form deep belief nets [28].Figure 3
A RBM structure.Letvi and hj represent the states of visible node i and hidden node j, respectively. For binary state nodes, that is, vi and hj∈{0,1}, the state of hj is set to 1 with probabilities [47]:(2.7)phj=p(hj=1∣ν)=σ(bj+∑iwijνi),
where σ(x) is the logistic sigmoid function 1/(1+exp(-x)), bj is the bias of j, and νi is the binary state. wij is the weight between νi and hj. After binary states have been chosen for the hidden units, then set the state of νi to be 1 with probability(2.8)pνi=p(νi=1∣h)=σ(bi+∑jwijhj).The training process of the RBM is described as follows. Firstly, a training sample is presented to the visible nodes, and the{νi} is obtained. Then the hidden nodes state that {hj} are sampled according to probabilities in (2.7). This process is repeated once more to update the visible and then the hidden nodes, and the one-step “reconstructed” states νi′ and hj′ are obtained. The update in a weight is given as follows:(2.9)Δwij=η(〈νihj〉-〈νi′hj′〉),
where η is the learning rate, and 〈·〉 refers to the expectation of the training data.A continuous restricted Boltzmann machine (CRBM) is considered by Chao et al. [34] and Chen and Murray [48]. Suppose the inputs nodes with state {si}, then the output nodes sj can be computed as follows:(2.10)sj=φj(∑iwijsi+σ⋅Nj(0,1)),
where φj(x) is a sigmoid function with lower and upper asymptotes at θL and θH, φj(xj)=θL+(θH-θL)·(1/(1+e-ajxj)). Nj(0,1) represents a unit Gaussian. σ is a constant, and parameter aj is a “noise-control” parameter which controls the slope of the sigmoid function [49]. The update equations for wij and aj are(2.11)Δwij=ηw(〈sisj〉-〈si′sj′〉),Δaj=ηaaj2(〈sj2〉-〈sj′2〉),
where ηw and ηa represent the learning rates, sj′ denotes the one-step sampled state of node j, and 〈·〉 refers to the expectation of the training data. We train sequentially as many RBMs as the number of hidden layers in the DBN to construct a DBN model. We adopt the learning algorithm according to [28, 34, 50]. The method of stacking CRBMs makes it possible to train many layers of hidden units efficiently and is one of the most common deep learning strategies. As each new layer is added, the overall generative model gets better. This process of learning is continued until a prescribed number of hidden layers in the DBN have been trained. In order to apply DBN model to drought prediction using SPI series, the DBN model with two hidden layers is selected in this paper. The main steps using DBN model for drought index prediction are as follows.Step 1.
Compute the different time-scale SPI series by precipitation data.
The different time-scale SPI series are computed by precipitation data by the description method in Section2.1, and different time-scale SPI series are obtained.Step 2.
Normalize the SPI series by formula (2.12) as follows:
(2.12)SPI′=SPI-SPImin⁡SPImax⁡-SPImin⁡,
where SPI′ and SPI represent the normalized and original SPI data, respectively. The SPImin represents the minimum value of the corresponding SPI series, and SPImax represents the maximum value of the corresponding SPI series.Step 3.
Determine the optimal network structure by experiments.
Determine the number of input nodes, the numbers of the first hidden and second hidden nodes, and weight coefficients by learning algorithm. The data of SPI series are split into two parts. The first part is used as a training sample, and the rest is used as a testing sample. During the training process, the network structures for different time-scale SPI series are determined according to the criterion of smallest RMSE and MAE.Step 4.
Forecast drought index based on DBN model and results analysis.


## 3. Case Study

### 3.1. Experimental Design

We use four data sets of precipitation in the experiments. Four hydrologic stations were considered in this study. They are Bengbu, Fuyang, Xuchang, and Zhumadian in Huaihe River Basin which is located in the eastern part of China. Data sets contain monthly precipitation during 1958–2006. These data are used to calculate four different time scales of standardized precipitation index (SPI), that is, SPI3, SPI6, SPI9, and SPI12. Taking the SPI3 as an example, all of the SPI sets are divided into two parts. The observations during 1958–1999 are as training set, and the remaining observations during 2000–2006 are as testing set.Our purpose of this research is to explore if the DBN model can be used well in drought prediction by using the monthly rainfall data of four hydrologic stations from January 1958 to 2006 to calculate different time scales of SPI in Huaihe River Basin.In this paper, we use two criteria to evaluate the performance of a DBN in drought forecasting. They are root mean square error (RMSE) and mean absolute error (MAE). The formulas of this two predictive accuracy measures are listed as follows:(3.1)RMSE=∑i=1T(yi-yi′)2T,MAE=∑i=1T|yi-yi′|T,
where yi is the observations of SPI, yi′ is the predicted SPI values, and T is the total number of predictions.We use the learning sample to find an optimal network structure for these four different time-scales SPI. Taking the SPI3 of Bengbu data as an example, we explain how to determine an optimal network structure. In our experiment, the DBN has two hidden layers. The key for our experiment is to determine the numbers of input and hidden nodes. We determine the optimal number of input nodes and two hidden layer nodes by experiments. On one hand, neural networks with too few hidden nodes may not have enough power to model the data. On the other hand, neural networks with too many hidden nodes may lead to overfitting problems and finally result in poor forecasting performance [30]. In our experiment, the number of input nodes and hidden nodes of the DBN network structures is selected by experimentation. The number of input nodes ranges from 2 to 10. Because the forecasting performance of neural networks is not as sensitive to the number of hidden nodes as to the number of input nodes, so the number of hidden nodes is selected by five levels, that is, 5, 10, 15, 20, and 25. We did the experiment for 45 times to find the optimal structure of DBN. We compared the RMSE and MAE, and we determined the number of every layer node. The results are shown in Table 1.Table 1
The CRBM results of Bengbu SPI3.
Number of input nodesNumber of hidden nodesRMSEMAENumber of input nodesNumber of hidden nodesRMSEMAE250.12700.0980750.11070.0869100.12760.0983100.11140.0877150.13060.1002150.11130.0878200.12760.0983200.11130.0877250.12740.0982250.11230.0885350.12700.0987850.11130.0865100.12930.1000100.11200.0871150.12720.0988150.11150.0866200.12740.0989200.11170.0869250.12730.0988250.10970.0864450.12920.0993950.11050.0863100.12910.0990100.11170.0876150.12830.0988150.11190.0876200.13010.0995200.11200.0876250.12860.0992250.11810.0945550.12250.09291050.11120.0874100.12300.0927100.11160.0875150.12350.0931150.11230.0884200.12320.0928200.11150.0885250.12480.0939250.11210.0886650.12350.0931100.12390.0930150.12360.0934200.12350.0930250.12440.0942In Table1, we find when the CRBM structure is 8-25, the RMSE is the smallest, and when the CRBM structure is 9-5, the MAE is the smallest. We can find that the most optimal structure is most likely to appear when the number of input nodes is 8 or 9. Then we do the next step. The results of the next step have just been shown in Table 2. We can find that the best DBN structure is 9-5-10-1. That is, the DBN has 9 input nodes, 5 nodes in the first hidden layer, 10 nodes in the second hidden layer, and 1 output node, and the RMSE and MAE are the smallest of all.Table 2
The DBN results of Bengbu SPI3.
Number of input nodesNumber of first hidden nodesNumber of second hidden nodesRMSEMAENumber of input nodesNumber of first hidden nodesNumber of second hidden nodesRMSEMAE50.69240.549750.69050.5510100.69110.5567100.68420.54255150.71030.57875150.68580.5473200.70060.5712200.71380.5860250.80510.6858250.70680.576950.70480.570250.68810.5472100.69110.5506100.68990.547910150.69620.561610150.69540.5548200.71470.5874200.70900.5775250.77490.6606250.70320.580550.68520.545350.69300.5545100.69260.5596100.71650.5887815150.69690.5636915150.73000.6106200.72650.6113200.72180.6070250.73160.6236250.74880.637550.72240.600150.69150.5516100.69230.5525100.69620.559220150.71520.594020150.74070.6245200.69550.5560200.71400.5885250.77940.6679250.73090.613650.69350.557050.69190.5563100.68770.5496100.70130.563725150.70370.576125150.69490.5617200.74700.6322200.76690.6566250.71740.5868250.72140.6104According to above processes, we can determine the optimal structures of DBN for the four stations and different time-scale SPI series. We try nine levels of input nodes from 2 to 10 in combination with five hidden nodes (5, 10, 15, 20, and 25) for CRBM training. We can find the optimal network structure in a similar way for all of the SPI series. The optimal network structures of DBN for the different four stations and different time-scale SPI series are shown in Table3.Table 3
The optimal network structures of DBN.
SPI seriesStationNumber of input nodesNumber of first hidden nodesNumber of second hidden nodesSPI3Bengbu9510Fuyang82015Xuchang82015Zhumadian82015SPI6Bengbu10510Fuyang755Xuchang755Zhumadian855SPI9Bengbu1055Fuyang755Xuchang655Zhumadian10515SPI12Bengbu855Fuyang1055Xuchang955Zhumadian1055


### 3.2. Results and Discussion

In this paper, the DBN and BP neural network model are used for forecasting the different time-scale SPI series, and the results of their prediction are compared. The quantitative performance evaluations of DBN and BP neural network are carried out by using RMSE and MAE. The results are shown in Table4.Table 4
The comparison of RMSE and MAE between BP and DBN.
StationModelErrorsSPI3SPI6SPI9SPI12BengbuDBNRMSE0.68420.65920.53550.4797MAE0.54250.52740.39590.3553BP neural networkRMSE0.98970.69870.58990.5809MAE0.75640.55230.41570.4532FuyangDBNRMSE0.81120.66340.55900.5620MAE0.65270.48120.39230.4282BP neural networkRMSE1.08760.80220.80320.5773MAE0.82020.58670.55090.4080XuchangDBNRMSE0.72580.57640.52620.4236MAE0.57140.43420.38800.2976BP neural networkRMSE0.82230.69380.67830.4454MAE0.67860.54110.47250.3268ZhumadianDBNRMSE0.77940.62390.56860.4990MAE0.62760.47920.38110.3504BP neural networkRMSE1.07800.79560.79960.4474MAE0.81470.63360.53650.3144We can find that the prediction errors of the DBN are smaller than the prediction errors of BP neural network in Table4. The errors results demonstrate that DBN model is suitable for the drought prediction in the Huaihe River Basin. DBN model can obtain smaller RMSE and MAE compared with BP neural network. With the change of the time scale of SPI from little to large, the RMSE and MAE become smaller. That is, the fitting results of SPI12 are better than SPI9, SPI9 is better than SPI6, and SPI6 is better than SPI3. In a word, DBN has a higher precision in drought prediction based on SPI than BP neural network.Figure4 shows the test results of SPI3, SPI6, SPI9, and SPI12 of Bengbu station. It is obvious that the prediction values of different time-scale SPI series are very close to the actual ones. The comparison results between observations and predicted data of Fuyang station, Xuchang station, and Zhumadian station are shown in Figures 5, 6, and 7 using DBN and BP neural network for SPI6 series.Results comparison between observations and predicted data using DBN for different time-scale SPI series of Bengbu station.
(a)(b)(c)(d)Figure 5
A comparison of DBN and BP neural network for SPI6 series of Fuyang station.Figure 6
A comparison of DBN and BP neural network for SPI6 series of Xuchang station.Figure 7
A Comparison of DBN and BP neural network for SPI6 series of Zhumadian station.From Figures5, 6, and 7, the predicted data of SPI based on DBN model agreed with observations very well. The majority of DBN outputs are nearer to the real SPI values than those of BP neural network. The results show that the DBN model is appropriate for short term of drought index and can obtain higher precision.


## 3.1. Experimental Design

We use four data sets of precipitation in the experiments. Four hydrologic stations were considered in this study. They are Bengbu, Fuyang, Xuchang, and Zhumadian in Huaihe River Basin which is located in the eastern part of China. Data sets contain monthly precipitation during 1958–2006. These data are used to calculate four different time scales of standardized precipitation index (SPI), that is, SPI3, SPI6, SPI9, and SPI12. Taking the SPI3 as an example, all of the SPI sets are divided into two parts. The observations during 1958–1999 are as training set, and the remaining observations during 2000–2006 are as testing set.Our purpose of this research is to explore if the DBN model can be used well in drought prediction by using the monthly rainfall data of four hydrologic stations from January 1958 to 2006 to calculate different time scales of SPI in Huaihe River Basin.In this paper, we use two criteria to evaluate the performance of a DBN in drought forecasting. They are root mean square error (RMSE) and mean absolute error (MAE). The formulas of this two predictive accuracy measures are listed as follows:(3.1)RMSE=∑i=1T(yi-yi′)2T,MAE=∑i=1T|yi-yi′|T,
where yi is the observations of SPI, yi′ is the predicted SPI values, and T is the total number of predictions.We use the learning sample to find an optimal network structure for these four different time-scales SPI. Taking the SPI3 of Bengbu data as an example, we explain how to determine an optimal network structure. In our experiment, the DBN has two hidden layers. The key for our experiment is to determine the numbers of input and hidden nodes. We determine the optimal number of input nodes and two hidden layer nodes by experiments. On one hand, neural networks with too few hidden nodes may not have enough power to model the data. On the other hand, neural networks with too many hidden nodes may lead to overfitting problems and finally result in poor forecasting performance [30]. In our experiment, the number of input nodes and hidden nodes of the DBN network structures is selected by experimentation. The number of input nodes ranges from 2 to 10. Because the forecasting performance of neural networks is not as sensitive to the number of hidden nodes as to the number of input nodes, so the number of hidden nodes is selected by five levels, that is, 5, 10, 15, 20, and 25. We did the experiment for 45 times to find the optimal structure of DBN. We compared the RMSE and MAE, and we determined the number of every layer node. The results are shown in Table 1.Table 1
The CRBM results of Bengbu SPI3.
Number of input nodesNumber of hidden nodesRMSEMAENumber of input nodesNumber of hidden nodesRMSEMAE250.12700.0980750.11070.0869100.12760.0983100.11140.0877150.13060.1002150.11130.0878200.12760.0983200.11130.0877250.12740.0982250.11230.0885350.12700.0987850.11130.0865100.12930.1000100.11200.0871150.12720.0988150.11150.0866200.12740.0989200.11170.0869250.12730.0988250.10970.0864450.12920.0993950.11050.0863100.12910.0990100.11170.0876150.12830.0988150.11190.0876200.13010.0995200.11200.0876250.12860.0992250.11810.0945550.12250.09291050.11120.0874100.12300.0927100.11160.0875150.12350.0931150.11230.0884200.12320.0928200.11150.0885250.12480.0939250.11210.0886650.12350.0931100.12390.0930150.12360.0934200.12350.0930250.12440.0942In Table1, we find when the CRBM structure is 8-25, the RMSE is the smallest, and when the CRBM structure is 9-5, the MAE is the smallest. We can find that the most optimal structure is most likely to appear when the number of input nodes is 8 or 9. Then we do the next step. The results of the next step have just been shown in Table 2. We can find that the best DBN structure is 9-5-10-1. That is, the DBN has 9 input nodes, 5 nodes in the first hidden layer, 10 nodes in the second hidden layer, and 1 output node, and the RMSE and MAE are the smallest of all.Table 2
The DBN results of Bengbu SPI3.
Number of input nodesNumber of first hidden nodesNumber of second hidden nodesRMSEMAENumber of input nodesNumber of first hidden nodesNumber of second hidden nodesRMSEMAE50.69240.549750.69050.5510100.69110.5567100.68420.54255150.71030.57875150.68580.5473200.70060.5712200.71380.5860250.80510.6858250.70680.576950.70480.570250.68810.5472100.69110.5506100.68990.547910150.69620.561610150.69540.5548200.71470.5874200.70900.5775250.77490.6606250.70320.580550.68520.545350.69300.5545100.69260.5596100.71650.5887815150.69690.5636915150.73000.6106200.72650.6113200.72180.6070250.73160.6236250.74880.637550.72240.600150.69150.5516100.69230.5525100.69620.559220150.71520.594020150.74070.6245200.69550.5560200.71400.5885250.77940.6679250.73090.613650.69350.557050.69190.5563100.68770.5496100.70130.563725150.70370.576125150.69490.5617200.74700.6322200.76690.6566250.71740.5868250.72140.6104According to above processes, we can determine the optimal structures of DBN for the four stations and different time-scale SPI series. We try nine levels of input nodes from 2 to 10 in combination with five hidden nodes (5, 10, 15, 20, and 25) for CRBM training. We can find the optimal network structure in a similar way for all of the SPI series. The optimal network structures of DBN for the different four stations and different time-scale SPI series are shown in Table3.Table 3
The optimal network structures of DBN.
SPI seriesStationNumber of input nodesNumber of first hidden nodesNumber of second hidden nodesSPI3Bengbu9510Fuyang82015Xuchang82015Zhumadian82015SPI6Bengbu10510Fuyang755Xuchang755Zhumadian855SPI9Bengbu1055Fuyang755Xuchang655Zhumadian10515SPI12Bengbu855Fuyang1055Xuchang955Zhumadian1055


## 3.2. Results and Discussion

In this paper, the DBN and BP neural network model are used for forecasting the different time-scale SPI series, and the results of their prediction are compared. The quantitative performance evaluations of DBN and BP neural network are carried out by using RMSE and MAE. The results are shown in Table4.Table 4
The comparison of RMSE and MAE between BP and DBN.
StationModelErrorsSPI3SPI6SPI9SPI12BengbuDBNRMSE0.68420.65920.53550.4797MAE0.54250.52740.39590.3553BP neural networkRMSE0.98970.69870.58990.5809MAE0.75640.55230.41570.4532FuyangDBNRMSE0.81120.66340.55900.5620MAE0.65270.48120.39230.4282BP neural networkRMSE1.08760.80220.80320.5773MAE0.82020.58670.55090.4080XuchangDBNRMSE0.72580.57640.52620.4236MAE0.57140.43420.38800.2976BP neural networkRMSE0.82230.69380.67830.4454MAE0.67860.54110.47250.3268ZhumadianDBNRMSE0.77940.62390.56860.4990MAE0.62760.47920.38110.3504BP neural networkRMSE1.07800.79560.79960.4474MAE0.81470.63360.53650.3144We can find that the prediction errors of the DBN are smaller than the prediction errors of BP neural network in Table4. The errors results demonstrate that DBN model is suitable for the drought prediction in the Huaihe River Basin. DBN model can obtain smaller RMSE and MAE compared with BP neural network. With the change of the time scale of SPI from little to large, the RMSE and MAE become smaller. That is, the fitting results of SPI12 are better than SPI9, SPI9 is better than SPI6, and SPI6 is better than SPI3. In a word, DBN has a higher precision in drought prediction based on SPI than BP neural network.Figure4 shows the test results of SPI3, SPI6, SPI9, and SPI12 of Bengbu station. It is obvious that the prediction values of different time-scale SPI series are very close to the actual ones. The comparison results between observations and predicted data of Fuyang station, Xuchang station, and Zhumadian station are shown in Figures 5, 6, and 7 using DBN and BP neural network for SPI6 series.Results comparison between observations and predicted data using DBN for different time-scale SPI series of Bengbu station.
(a)(b)(c)(d)Figure 5
A comparison of DBN and BP neural network for SPI6 series of Fuyang station.Figure 6
A comparison of DBN and BP neural network for SPI6 series of Xuchang station.Figure 7
A Comparison of DBN and BP neural network for SPI6 series of Zhumadian station.From Figures5, 6, and 7, the predicted data of SPI based on DBN model agreed with observations very well. The majority of DBN outputs are nearer to the real SPI values than those of BP neural network. The results show that the DBN model is appropriate for short term of drought index and can obtain higher precision.


## 4. Conclusion

In this paper, we proposed a deep belief network (DBN) for short-time drought index prediction. The forecasting model based on DBN is used to forecast different time-scale SPI series of four stations in Huaihe River Basin, China. Compared with the BP neural network, the DBN-based model is more reliable and efficient for short-term prediction of drought index. The errors results show that the DBN model outperforms the BP neural network. This study shows that the DBN model is a useful tool for drought prediction. Due to the complexity of the formation mechanism of the drought disasters and the long memory of hydrological data, some new method which can deal with long-range dependence will be thought about, and further studies are needed to deal with more complex situations for drought prediction.



---

*Source: 235929-2012-05-08.xml*