京 都 大 学 防 災 研 究 所 年 報 第48号 B 平成17年4月
Annuals of Disas. Prev. Res. Inst., Kyoto Univ., No. 48 B, 2005

Stochastic Operation for Multi-purpose Reservoir Using Neuro-Fuzzy Systems

Paulo CHAVES*, Toshiharu KOJIRI and Tomoharu HORI**Graduate school of Engineering, Kyoto University
Synopsis
We introduce a new approach for system optimization, named stochastic fuzzy neuralnetwork, which can be defined as a neurofuzzy system that is stochastically trained and can yield a‘quasi’ optimal solution. The method intends to overcome some of the problems related tostochastic dynamic programming models. For validation, a real case of storage reservoir
optimization with stochastic inflow discharge is considered. The SFNN model proved to show
improvements in the optimization results. Moreover, to deal with the uncertainties related todiscretization of inflows in finding the stochastic representative inflows, conditional probability of a
fuzzy event is used.
Keywords: Stochastic optimization, Fuzzy Neural Network, Conditional probability of a fuzzy
event, Reservoir operation

1.

Introduction

One of the most important and applied techniques forthe optimization of reservoirs is surely dynamic
programming (DP), which was developed by Richard
Bellman (1959). Applications of dynamic programming
to water resources systems can be found in many works,
such as Yakowitz (1982) and Esogbue (1989). DP
presents various advantages over other methods tohandle water resources management and optimization
problems and it can be associated with otherprogramming methods, such as stochastic techniques
resulting in the so called stochastic DP (SDP). Someapplications of SDP for water resources management isfound in Torabi et al. (1973) and Kelman et al. (1990),
where river discharge is treated as the stochastic variable.
Moreover, DP may also be used for the optimization ofmultistage fuzzy control systems, called fuzzy DP(Fontane et al. 1997, Kojiri et al. 1993 and Hori et al.
1997), in which decision and state variables as well as

constraints, can be set as fuzzy membership functions.
The basic characteristics of water resources and reservoir
operation that lead to the use of DP are: stage-wise
structure and non-linearity of the system.
DP can be divided into two different approaches as
continuous or discrete, with the latter the mostcommonly used for computational convenience.
However, for reservoir operation, the roughdiscretization of state variables, such as storage levels in
the case of reservoir optimization, may reduce theefficiency of the optimization model. Another importantcharacteristic of DP is the direction in which thecalculation is carried out, defined as forward-moving andbackward-moving. For deterministic DP (DDP),
calculation in either direction can be easily implemented.
However, the application of forward-moving calculationfor SDP models is not straightforward and may be
extremely complex to model (Chaves et al, 2003). The
direction in which the calculation is executed becomes
extremely important when DP based optimization

models have to be combined or embedded with othermodels, such as rainfall-runoff and water quality
simulation models (Chaves, 2002 and Chaves et a.l,2003), which require a time-wise calculation as actual
values depend on previous ones. Moreover, the mostimportant drawback of using DP is certainly the curse ofdimensionality, caused by the large number of state anddecision variables.
To overcome these problems related to SDP, a new
optimization method is proposed here to handle thediscretization of state variables through fuzzy theory andthe limitations of the direction of calculation through a
neuro-fuzzy system. The new proposed method is alsoable to handle the stochastic characteristics of variables,such as river discharge, in a forward-moving calculation
direction. Moreover, the problem resulting from theincrease in the number of state variables is easily
overcome, as new state variables can simply be
introduced as new neuron units. The proposed method isnamed stochastic fuzzy neural network (SFNN). For
validation of the proposed method, a real storage
reservoir optimization problem considering the stochastic
characteristics of inflow discharge information ispresented. Then, results using the SFNN are compared tothe ones obtained by other dynamic programming
formulations.
2.

Methodology

There are few applications of ANN models for
optimization problems, whereas the ANNs are usuallyapplied as simulation or prediction models. It was only
recently that some attempts to solve the storage reservoir
optimization using ANN have appeared. Raman andChandramouli (1996) and Chandramouli and Raman(2001) proposed the use of ANN to generate operating
rules based on the optimal results from a deterministic
DP model, for the case of a single and multiple reservoir
system, respectively. In a similar way, Cancellier et al.
(2002) developed a model to derive the operating rules
for an irrigation supply reservoir. In an attempt toconsider the stochastic characteristics of inflow,
Ponnambalam et al. (2003) trained an adaptive
neuro-fuzzy inference system (ANFIS; Jang, 1993)
based on the optimal results obtained by a stochastic
optimization model. Chang et al. (2001) combines
genetic algorithm (GA) and ANFIS, in which GA isapplied to search the optimal reservoir operating

histogram, which is used as the training pattern of the
ANFIS model, intended to estimate the optimal water
release with current storage level and inflow conditions
as input information.
There are some important characteristics of the
proposed method, which differ from other applications ofneuro-fuzzy systems for reservoir operation. First, mostof the ANN based models has the ANN trained based on
already-optimized results from other models to derived
operating rules. However, for the proposed SFNN model,
the optimization is carried out directly considering theobjective functions. The second and most important
point is that the SFNN is trained directly under stochastic
input conditions, presenting a probability of occurrence
depending on the known previous value, defined asconditional probability. The most common approach todefine the conditional probability is the Markov chain
technique (Howard, 1960) which considers one previous
stage of the conditional probability. The Markov chain isapplicable for continuous processes or, for computational
convenience, discrete processes. Some applications of
optimization problems using the Markov chain technique
can be found in Butcher (1971) and Yakowitz (1982).
The inflow into the reservoir can be considered as aMarkov chain process, where the system state, within acertain stage, is considered to be dependent on the stateof the previous stage and the known probabilities.
Depending on the available data and the number ofassumed intervals to pursue the calculation of theMarkov chain or one previous stage conditional
probability (CP), the resulting probabilistic curves may
present unrealistic characteristics due to incomplete data
set. The use of such results in stochastic models may
greatly affect its efficiency. To reduce the uncertainties
of the discretization process when calculating theconditional probabilities, due to incomplete data set andrough assumption of crisp discrete intervals, theintroduction of conditional probability of a fuzzy event is
proposed for the calculation of the conditional
probabilities of the representative inflow discharges
between two consecutive months.
3.

Conditional Probability of an event

For the (nonfuzzy) conditional probability of a fuzzy
event, the difference is that instead of crisp discreteintervals, the domain is divided into fuzzy linguistic
variables as can be seen in the lower part of Fig 1, whereL, M and H stand for low, medium and high, V and X forvery and extreme, respectively, with the center values ofeach linguistic variable described in the axis of thebottom part of Fig 1.
1

2

3

4

5

6

7

Discrete Intervals
Inflow

0 5

20

XL VL

40

L

60

M

80

H

Inflow Cumulative
95 100 Probability (%)

VH XH Linguistic VariablesN

F ( Ak / B j ) =Grademin 10305070Inflow Cumulative

90 max Probability (%)

Fig 1. Discretization of discharge: discrete (upper) andfuzzy (lower) intervals
The percentage scale in Fig 1 represents the chosen
limits for the discrete intervals and center values of
linguistic variables, corresponding to the monthly inflow(unconditioned) cumulative probabilities. This increases
the efficiency of the stochastic optimization model as
already carried out by Chaves (2002), reducing to acertain extent null-probability values, as a result of many
discretized interval or small time-series data.
Conditional probability is traditionally defined as theprobability of an event A under the condition that an
event B occurs. This probability is denoted by P(A/B)
and defined as shown in equation ( 1 ).
P ( AB) P ( A ∩ B )
=
P( B)
P( B)

(1)

for P(B) > 0 and where P(AB) is the intersection of
events A and B, and P(B) is the probability of event B.
As stated by Kaufman (1975) and Kacprzyk (1997), theabove concept is valid for both cases of nonfuzzy andfuzzy events.
For the case of the conditional probability of inflow
discharge, we are interested to find the probability of
inflow t when inflow t-1 occurs, which would
correspond to events A and B, respectively. The P(AB)can be calculated based on the observed time series data.
For the case of discrete intervals, the frequency ofoccurrence F can be found through equation ( 2 ) as
shown below:

∑c

(2)

n −1 + 1

n =1; c 0 = 0

Where cn is a counter operator for which the term +1
accounts for each occurrence between consecutive events(intervals) Ak and Bj, whereas k and j are indicesrepresenting the intervals [1,2,…7] in the upper part ofFig 1. Note that the summation of all F(Ak/Bj) is equal to
the total number of pairs of observations N.
P ( Ak ∩ B j ) =Inflow

P( A / B) =if and only if ( I t ∈ Ak ) when ( I t −1 ∈ B j ) then :F ( Ak / B j )(3)

NFinally, as the probability P(Bj) of the inflow event Bj
corresponding to interval j can easily be found by
dividing the frequency of inflows contained withininterval j by the total number of observations N. Then, it
is possible to obtain the final one-previous-stage orMarkov chain conditional probability using equation( 4 ):P ( Ak / B j ) =P ( Ak ∩ B j )(4)

P( B j )For the case using the fuzzy method, the discharge
values are fuzzified using the membership functions asshown in the lower part of Fig 1. Then, also based on theobserved data, frequency of conditioned occurrence of a
fuzzy event, hereafter denoted by Fµ(Ak/Bj) can beanalogously calculated for each linguistic variable using
equation ( 5 ):NFµ ( Ak / B j ) =∑ µ (I ) ⋅ µ (I
A, k

t,n

B, j

t −1, n )

(5)

n =1

Where A and B are now fuzzy consecutive events, forexample inflows of January and February; µA,k and µB,j
are the fuzzy membership function of these events for the
linguistic value k and j, respectively, k and j:[XL,VL…XH], for each pair n of observed data (forexample, for every year). The membership function here
represents the degree to which an observed value
corresponds to a certain class or linguistic variable. For
example, if both consecutive values are equal 1, thiswould indicate that the observed values are equal to therepresentative inflows and equation ( 5 ) would yield thesame results of frequency for the nonfuzzy method,equation ( 2 ). Values different than 1, could be roughly
said to represent a certain degree of “frequency” for two
consecutive observed values. Note that the summation of
all Fµ(Ak/Bj) for k and j, is equal to total number of pairof observations N, analogous to the nonfuzzy counterpartmethod. Hence, the Pµ (AB) of the fuzzy events can bethen calculated using equation ( 6 ):Pµ ( Ak ∩ B j ) =Fµ ( Ak / B j )(6)N

The (nonfuzzy) probability of a fuzzy event A isdenoted by Pµ (A) and is defined as in equation ( 7 ), as
described by Kaufman (1975) and Kacprzyk (1997),
based on the classic definition of Zadeh (1968), which isby far the most popular and widely used.
NPµ ( B ) =∑ µ ( x ) ⋅ p( x )
B

n

n

(7)

evidence and fuzzy measures.
4.

Stochastic Fuzzy Neural Network(SFNN)

Fig 2 shows the flowchart of the overall calculation
procedures for the SFNN optimization model where
inflow values are considered as stochastic variables
based on conditional probabilities. According to Fig 2,
first the SFNN model is initiated with a set of parameters,
which can be randomly created.

n =1

ANN Parameters

Where x is the actual value of the variable in question
for occurrence n. Hence, for our case of inflow
discharge:
NPµ ( B j ) =∑ µ (IB, jt −1, n ) ⋅ p ( I t −1, n )

(8)

n =1

Fuzzification of It-1 based on the same limits of the Cond. Prob.
Calculation of Expected Cond. Prob. Pexp[It,k/It-1] based on Fuzzified It-1
k=k+1

ANN Model with Input: It-1 & St & Output: St+1
Ek = E(It,k, St, St+1, Rt,k) . Pexp[It,k/It-1]
NO

Where p(It-1,n) is the probability of a certain inflow
It-1 and µB,j(It-1,n) is the value of the fuzzy membershipfunction for the inflow corresponding to observation nfor event (linguistic variable) j. That is to say, Pµ (Bj) isthe probability of a fuzzy event B to occur. Note that thesummation of p(It-1,n) is equal to 1 and (in our case) for
each pair of observation n, the probability of a certain
observed inflow presents the same value 1/N. Finally theconditional probability of a fuzzy event Ak, Pµ(Ak/Bj), is
calculated according to equation ( 9 ).
Pµ ( Ak / B j ) =t=t+1

Pµ ( Ak ∩ B j )Pµ ( B j )(9)The summation of Pµ(Ak/Bj) in respect to j is also
equal to 1 (one). Note that the summation is equal to 1
only if the linguistic variables are arranged in a way suchas illustrated in Fig 1, where the minimum values ofmembership functions for each class coincide with thecenter values of adjacent classes. In any other case, forexample using a different membership function, theprobability of each month has to be compensated by the
total summation for all classes.
It is important to mention that the proposed method
to define the Markov chain conditional probabilities is
not dealing with the vagueness and uncertainties related
to the probabilities themselves, but only the uncertainties
originating from the discretization process and thefuzziness of discharge observations. For moresophisticated analyses to handle such probabilistic
problems, the readers are referred to other methods suchas possibility theory, Dempster-Shafer theory of

k = K?
YES

Eexp= Σ(Ek) : Ft = Ft-1 + Eexp
NO

Final Fuzzy-NN ParametersYES

GA Operators

t = T?
YES

FT = max?

NO

Validation & Application

Fig 2. Flowchart of the calculation for SFNNThe SFNN can be applied to both types ofconditional probabilities presented in the previous
section. The values found through the traditional
conditional probability, which uses the crisp intervals,
can be directly used for the SFNN optimization model orthey may be interpolated according to the actual previous
inflow and the representative inflows. As for the CP of
fuzzy events, the fuzzy representative inflows may be
used. However, in doing so, the optimization model may
result in optimum fuzzy values of release andend-of-period storage that are too difficult for users tounderstand and too vague for their practical consideration.
The propagation of the fuzziness associated with thesevalues, when applied to the optimization model, would
tend to generate greater vagueness, resulting in lessefficient operation rules. Therefore, a method that could
be considered analogous to interpolation is proposed, inwhich only the center values are considered andaccording to the actual previous inflow, the conditional
probability values is changed.
As the previous inflow value is known, based on thealready calculated conditional probabilities of a fuzzy
event, the new expected value can be obtained for eachof the representative stochastic inflow It,k, which is also

defined as the center value of the fuzzy event k. The
expected conditional probability for the SFNN model iscalculated based on equation ( 10 ). The center values ofthe fuzzy membership function for the fuzzification of
the previous inflow are the same as the ones used for the
calculation of the conditional probabilities themselves,
having the center and limit values defined as the(unconditioned) probability for each month as shown in
the lower part of Fig 1.
J

Pexp ( I t ,k / I t −1 ) =∑ [Pµ ( A / B ) ⋅ µ (I )]k( 10 )

t −1

j

j =1

Where Pµ is the conditional probability of a fuzzy
event, µ is the membership function or fuzzy grade valueof previous inflow It-1 and A and B two consecutive fuzzy
events. An example of the calculation of the expected
probabilities is shown below in Fig 3, where values in
the center table represent the conditional probabilities,
calculated as in the previous section, between two
consecutive fuzzy events, i.e. previous and actual inflows.
The stochastic fuzzy neural network (SFNN) is thentrained considering the representative inflow values It,k
and the respective expected conditional probabilities
Pexp(It,k/It-1).
Actual Inflow (t)Grade for It-1
0.2
0.8

L
0.26
0.25
0.32
0.02
0.25
0.26
0.00

M
0.00
0.06
0.38
0.38
0.17
0.16
0.00

H
0.00
0.00
0.07
0.42
0.21
0.21
0.17

VH
0.00
0.00
0.01
0.12
0.12
0.16
0.31

XH
0.00
0.00
0.00
0.01
0.06
0.10
0.53

Sum
1.00
1.00
1.00
1.00
1.00
1.00
1.00

Expected Probability

0.06

0.25

0.30

0.32

0.06

0.01

0.00

1.00

It-1

VL
L
M
H
VH
XH

Previous Inflow (t-1)

VL
0.39
0.37
0.22
0.03
0.15
0.11
0.00

XL

XL
VL
L
M
H
VH
XH

XL
0.35
0.32
0.00
0.01
0.03
0.00
0.00

Fig 3. Example for the calculation of the expected
probability Pexp
Usually for the training of neuro-fuzzy models, thetarget vector is previously defined and usually accounts
for observed values. However, in the case of
optimization using the ANN based models; the training
is proposed to be based directly on the maximization of
the recursive function F itself. The following formulation
are proposed for the recursive equations ( 11 ) and ( 12 ),considering two different types of fuzzy decision:
Weighted average type[F ( S t ) = max E exp ( Et , k ) ⋅ α + Ft −1 ( S t −1 )S t +1]( 11 )

Minimization type[ [F ( S t ) = max min E exp ( E t , k ), Ft −1 ( S t −1 )S t +1]]( 12 )

Where α is the weight, here we consider all theweights to be the same and equal to 1/T. Nevertheless,different weights may be incorporated to increase theimportance of a particular period in time or as a discountfactor, where T is the total number of stages (time-steps).
The summation of α is equal to 1, which guarantees the
final value of the recursive function ( 11 ) to be between[0, 1]. This does not affect the final optimum storage orrelease sequence, but instead facilitates the comparisonof the final results with other decision types, such as theminimization type ( 12 ). Eexp is the expected value of theevaluation function, as shown in equation ( 13 ), which
depends on the expected probability.
K

E exp =

∑ E ⋅ P (I / I )
k

exp

t,k

t −1

( 13 )

k =1

where the Pexp is the expected conditional probability
presented in equation ( 10 ), k represents the index of
conditional probabilities, referring to the representative
stochastic inflows It,k. Ek is defined according to the kindof aggregation operator to be used, such as shown inequations ( 14 ), ( 15 ) and ( 16 ), for three aggregation
operators: weighted-average, product and minimization,
respectively.
N

Ek =∑ E ⋅ β where ∑ β = 1
N

n

n =1

n =1

1N

⎛ N
⎞
⎜
⎟
Ek = ⎜
En ⎟⎜⎟⎝ n =1⎠∏E k = min (E n )n( 14 )( 15 )( 16 )

Where β is the weight, here assume the same for all
objectives but which could vary depending on theimportance of each objective. N is the total number offuzzy objectives and index n represents each fuzzy
objective. All of them are represented by a fuzzy
objective membership functions En. The fuzzy objective
membership functions used here are shown later in Fig 7.
The calculation of the expected value Eexp is used in thecalculation of the recursive function shown in equation( 11 ) and ( 12 ). Optimization results using the above
fuzzy decisions and aggregation operators are shownlater.
Having a set of parameters for the neuro-fuzzy
system, it is possible to obtain the end-of-period storagelevel St+1, as we know St and It-1 values. For eachstochastic inflow value It,k, we can obtain a release value
Rt,k based on the reservoir mass balance equation ( 17 ),which neglects other losses such as evaporation andinfiltration that also could be considered.

Rt , k = St − St −1 + I t , k( 17 )

This operation is carried out for the whole training
period T. If the parameters of the neuro-fuzzy system areconsidered to be optimal, giving the maximum or quasi
maximum value for the recursive function then the
training process can be concluded. After validation themodel can be finally applied as the network is now
representing the reservoir stochastic optimal operation
rules.
It is important to note that in the case of SDP models
all alternatives are tested for a period of one year, or thetime representing one cycle of the stochastic variable,
here being the monthly inflow discharges. The results ofa SDP model are presented as guidecurves that can beused to obtain the end-of-period storage when the actual
month, previous inflow and initial storage values areprovided. On the other hand, the SFNN model is training
using directly observed data, considering the conditional
probability between stages and also the long-term
characteristics of river discharge. In the case thatobservations are scarce or considered to be incomplete,
the SFNN model can be trained with artificiallygenerated random or synthetic time series data. On theother hand, only pure random values can be used,however the long-term characteristics of inflow are nolonger considered, which could decrease part of the
SFNN model efficiency. It is important to keep in mind
that the computational effort increases with an increasein the data set size. Nevertheless, for longer training data,
more efficient operation rules may be expected. The
optimal results found by SFNN are no longer presented
as guidecurves, but instead as a fuzzy neural networkmodel, stochastically trained, which has theend-of-period storage represented by its output neuron,and initial storage and previous inflow represented
through its input neurons.

5.

Fuzzy Neural Network Model

Artificial neural network (ANN) models have been
widely used in various fields, such as aerospace, finance,
robotics, environmental assessment and hydrology, fordifferent purposes such as classification, pattern
identification, simulation and prediction (Hagan et al.,1995). The combination of ANNs with other techniques,
such as fuzzy theory, has been proposed by some authors.
The use of fuzzy theory with the ANN aims to combine
the ability of fuzzy sets to represent knowledge that isunderstandable to human beings with the learning
capability of ANNs (Jin, 2003). Therefore, a neuro-fuzzy
system is a fuzzy system that can learn from data as well
as from experts’ experience. Application of fuzzy theory
with ANNs may change the model from being a“black-box” into a “gray-box” type. As described by Jin(2003), there have been some attempts to develop
neuro-fuzzy models and the most well known example isthe adaptive-network-based fuzzy inference system(ANFIS) proposed by Jang and Sun (1993, 1995). The
ANFIS model has been successfully applied to many
problems, including storage reservoir operation and it
can serve as a basis for the construction of fuzzy if-thenrules. However, this method can face greatcomputational effort, as the increase in the number ofparameters is proportional to its number of linguistic
variables and input nodes.
5.1 Network Architecture
Based on the above explanation, a simpler
neuro-fuzzy network is proposed, whereas the basicarchitecture of the model is briefly presented as follows.
The training process is carried out by a genetic algorithm(GA) model, where the objective function is the as theone used for the reservoir operation, such as the ones
shown later in Fig 7. The developed neuro-fuzzy systemarchitecture is shown in Fig 4.
Crisp Input

Fuzzification

Average

Rule Firing

Σ

g

Σ

g

Σ

g

Defuzzificationµ1,1
Inflow
t-1

µ1,2
µ1,3
µ2,1

InitialStorageµ2,2Storage
Variationµ2,3

Fig 4. Architecture of the proposed neuro-fuzzy
inference system
Previous inflow (t-1) and initial storage arerepresented by two input nodes, whereas the output node
gives the variation of the end-of-period storage value,
which is the consequence of the operating rules. Eachmonth has its own network, which proved tosignificantly increase the optimization efficiency. Hence,
the operation rules represented by the neuro-fuzzy
system are analogous to the operation guide-curves
derived by the SDP models, where by three types of
information is required (actual month, previous inflow
and initial storage).

found to be the most appropriate as robustness of thewhole system increases. Probably, this is due to the useof triangular functions that present only two membershipvalues other than zero, which in a product typeaggregation could null all the membership values. The
resulted fuzzy number µy,k after the combination throughthe transfer function of previous units i is presented inequation ( 18 ).

µ j, k
5.2 Fuzzification of Inflows and Storage
Considering a crisp input value xi, such as previous
inflow and initial storage, the fuzzification function iscarried out as shown in Fig 5. The parameters Xk aredefined as the modal value of the fuzzy membershipfunction of each linguistic variable k. For the case ofprevious inflow (t-1), the parameters Xk, wherek[1,2,…7], are defined based on the cumulative
probability function for each month, accounting for thesame values as the ones used for the calculation of theconditional probability of a fuzzy event, as shown in Fig
1. For the case of initial storage, the parameters Xk are
optimized during the training process. The extreme
values X1 and X7 are set equal to the minimum andmaximum expected values, i.e. constrain values use forthe reservoir operation here assumed equal to 600 hm3
and 3250 hm3, respectively. Seven linguistic values areused as they proved to be enough for modeling thesystem. Moreover, this number has also been usedwidely in other applications of fuzzy inference. The
resulted membership values are represented by thesymbol µi,k, which is represented graphically in Fig 5.
1

XL

VL L

X1

X2

X3

xi

M

H

VH

XH

X4

X5

X6

X7

x

Fig 5. Membership function for the fuzzification of
input variables xi
The resulted membership values µx,k for eachlinguistic variable k, found after the fuzzification of thecrisp input value x, are then combined. There are various
methods used within fuzzy inference theory, such as
product, minimization and maximization. However, after
trying these different methods, the weighted sum is(W ⋅ µ )
∑
=
∑ W
N

ij

i =1

i, k

N

( 18 )ij

i =1

Where N is the total number of previous units and Wij
is the weight between neurons xi and yj, W is also
optimized during training. The resulting fuzzy inference
is represented by µj,k.
Knowing the fuzzy value µi.k, it is possible to obtain acrisp value yj, which in our case represent the variation of
final (end-of-period) storage, through many available
defuzzification methods. The most common
defuzzification methods are the center of gravity, centerof sums and center of areas (Jin, 2003). A method basedon the center of gravity and described in Jin (2003) is
adopted here, in equation ( 19 ).

(µ ) ⋅ Y∑y =∑ (µ )
K

j

k =1
K

k =1

j,k

g

j,k

k

g

( 19 )

Where Yk is the kernel or the modal value and is
identified during training; g [0,∞) is a power parameter
responsible for “firing the rule”, which can be properly
adjusted to obtain a more efficient performance for thedefuzzification method. This parameter is also optimized
during the training period. After calculating the variationof storage (dS = yi) and knowing the initial storage (Sinitial),
the final storage (Sfinal) can then be calculated by equation( 19 ):

S final = Sinitial + dS
6.

( 20 )

Case Study

Barra Bonita reservoir is located in the middle Tietê
River basin, in the São Paulo State of Brazil, (22o29’ S
and 48o34’ W) with maximum surface water at analtitude of 453 m. The location of the reservoir is shown

Fig 6. Location of Barra Bonita reservoir
6.1 Operation Objectives
The proposed methodology is applied in thedevelopment of a simple storage reservoir optimization
model considering fuzzy objectives related to water
quantity characteristics of the system. These functions
are defined here as flow stabilization µstab(Rt) or STAB, asa function of operated release, and hydropower
generation (in KW) µpow(Rt, St) or POW, as a function(indirectly) of operated release and storage level. Bothobjectives are represented by fuzzy membershipfunctions, defined between 0 and 1, representing the low
and high satisfaction of operation, respectively. The
fuzzy membership functions represent the degree ofsatisfaction or conformity in attaining the operation
objectives. Due to a lack of realistic information about
the reservoir operation, the two objectives consideredhere for optimization have been hypotheticallyformulated and their fuzzy membership functions areshown in Fig 7.
HYDROPOWER GENERATIONFLOW STABILIZATION

1.0

1

0.8

0.8
Fuzzy Value

Fuzzy Valuein Fig 6.The reservoir has a water surface area ofapproximately 340 km2, total volume of 3.6 km3 andlength of 50 km. Maximum and average depth arearound 25 and 10 meters, respectively. Average waterfluctuation of the reservoir is approximately 5 meters.
Hydropower energy is the primary water use of thereservoir. Other uses include navigation, recreation,water supply and fishery production. The Reservoir is thefirst of a series of six reservoirs, around 300 km
downstream from Brazil’s biggest city, Sao Paulo. It can
be classified as a subtropical/tropical reservoir with anintermediate retention time of around one to two months.
Air temperature normally varies only by 15oC between
winter and summer. The wet season occurs between
September and March. Annual cumulative precipitationis around 1400 mm, with maximum wind velocity of 5 to
7 m/s during winter. The average annual flushing rate is
414 m3/s.

0.6
0.4

0.6
0.4
0.2

0.2

0

0.0
0

20000

40000
60000
Generated Energy

80000

100000

0

500

1000

1500

2000

2500

3

Release (m /s)

Fig 7. Fuzzy objective membership functions for the
reservoir operation
The release values of 350 and 400 m3/s are assumed
to represent maximum satisfaction in the flow
stabilization membership function, which is intended tostabilize the release around the historical average inflow
for the reservoir. It is very important, particularly in thecase of Brazil, to consider power generation in the
optimization. The membership function is based on the
produced and demanded energy. Due to a lack of data
regarding the energy production, an average for all
months is used to represent energy demand. It may be
improved with realistic data and generation targets forthe specific reservoir. Another way to improve the power
generation analysis is by using statistical analysis topredict demand. However, for the purpose of this study, avalue of 100MW is assumed as the monthly average
demand for all months. The generated energy (KW) is
calculated based on the release R (m3/s) and water head
H (m) by equation ( 21 ), where H is a function of theaverage of initial- and end-of-period storage values ofstage t.
Generated Energy = 8.319 ⋅ Rt ⋅ H t

7.

( 21 )

Validation of the SFNN Model

To test the proposed methodology, the Barra Bonita
reservoir is optimized through five different optimization
schemes, including the Stochastic and deterministic FNN
method and three SDP formulations, all for monthly
time-steps. The total time series is divided into two data
sets, where 22 years of observations are used to identify
the conditional probabilities for all models and to train
the SFNN model, whereas 11 years is used to validate
the proposed methodology in a practical reservoir
optimization. All the optimization schemes use the same
fuzzy objective membership functions presented in Fig 7,
which are combined using the product aggregation

method and the weighted-average type of decision, forthe same length of optimization horizon. The first
scheme is a simple deterministic DP model where theinflow in stage t is considered to be known. Therefore,
the DDP is expected to give the maximum satisfaction
for the operation objectives. The recursive equation for
the DDP is presented in equation ( 22 ).

Ft (S t ) = max{Et (.) + Ft +1 (S t +1 )}St +1( 22 )

Where F is the recursive equation, E is the evaluation
function and S, R and I represent storage, release andinflow, respectively, for time step t.
To exemplify the importance of storage discretization
within the DP optimization, some levels of storage
discretization were tested for the optimization of theBarra Bonita reservoir applying the deterministic
formulation, considering the whole inflow data set (33years). The final results of the recursive equation for thewhole optimization horizon (referred to as average in thelegend of Fig 8), the minimum values among all Et
values and its standard deviation are presented in Fig 8.
0.92

St. Deviation
Min
Average

0.7
0.6

0.91

0.90

0.5

0.89

0.4
0.3

0.88

0.2
0.87

0.1
0.0

Objective Function (Average)

Object. Function (Min & St.Dev.)

0.8

0.86
0

5

10

15

20

25

to the cumulative probability values and Et(.) is theevaluation function.
⎧ K⎫⎪[Et (⋅) + Ft +1 ( S t +1 , I t )]⋅ Pµ I t , k I t −1 ⎪⎬Ft ( S t , I t −1 ) = max ⎨S t +1 ⎪⎪⎩ k =1⎭∑()( 24 )

Where Pµ is the fuzzy conditional probability which
has been already defined in equation ( 9 ). The statetransition function has been presented in equation ( 17 ).
Note that this is the inverted form of the mass balance
equation (Labadie, 1993). This form is chosen for its
coding simplicity. The optimization is subjected to theconstraint of minimum and maximum acceptable valuesof storage and release. Moreover, FT = 0, where T is thelast chronological time, and therefore, for the backward
calculation, the first stage used for calculation.
Since the transition probabilities repeat every 12
months, the undiscounted stochastic DP calculations arerepeated via successive approximations to confirm andguarantee that the optimum guidecurves of end-of-period
storage for each month are converging to stationary
values. As a result, optimal guidecurves may be applied
to each year over the entire operational horizon for any
sequence of inflow. As referred by Labadie (1993), if
this procedure converges, then the solution must be
optimum. Finally, the reservoir is optimized through the
SFNN proposed method, also considering the conditional
probability of a fuzzy event. The SFNN recursive
equation has already been shown in equation ( 11 ). Notethat both recursive equations for the SDP models arepresented as backward-moving while the SFNN can beformulated for the forward-moving.

30

Discretization of Storage

8.
Fig 8. Relation between storage discretization and DP
optimization results
Then, stochastic characteristics of inflow areconsidered by applying two SDP models. The first one isbased on discrete conditional probability and the second
uses the proposed fuzzy formulation. This is to confirm
the efficiency of the proposed conditional probability ofa fuzzy event. The recursive equations for the twostochastic optimization schemes using SDP are presented
in equations ( 23 ) and ( 24 ), respectively.
⎧ K⎫⎪
Ft ( S t , I t −1 ) = max ⎨[Et (⋅) + Ft +1 ( S t +1 , I t )]⋅ P I t , k I t −1 ⎪⎬S t +1 ⎪⎪⎩ k =1⎭∑()( 23 )

Where P is the conditional probability for the discrete
inflow intervals k, It,k is the stochastic inflow which refer

Results

Fig 9 shows the results of cumulative conditional
probability curves for the month of February carried out
for the monthly inflow discharge of the Barra Bonita
reservoir using the CP for discrete intervals of inflow,whereas Fig 10 shows the results after using theconditional probability of fuzzy inflow.
It can be seen that the results of the fuzzy based
method (Fig 10) are much smoother than the ones fromthe formulation based on discrete intervals. Therefore,
they are much more in keeping with what is expected ofrealistic conditional probability curves. Moreover, results
after optimization using a SDP model showed to be more
efficient than the ones using the traditional method, as
can be seen in Fig 11.

1.0

Previous
Inflow (%)

Cumulative CP

0.8

Min - 55 - 2020 - 40
40 - 60
60 - 80
80 - 95
95 - Max

0.6
0.4
0.2
0.0
Min - 55 - 2020 - 40

40 - 60

60 - 80

80 - 95 95 - Max

Actual Inflow (Discrete Interval)

Fig 9. Cumulative CP assuming discrete intervals
1.0

Previous
Inflow

Cumulative CP

0.8
0.6
0.4

XL

VL

L

M

H

VH

0.2
XH
0.0
XL

VL

L

M

H

VH

XH

Actual Inflow (Linguistic variables)

Fig 10. Cumulative CP assuming fuzzy linguistic
variables
Fig 11 shows a comparison of evaluation function
results after the five different optimization schemes using
the weighted-average aggregation operator for theobjective functions, equation ( 14 ): a deterministic DP(DDP) model (maximum expected value, for whichinflows are considered to be known), a deterministic
FNN model (maximum expected value, for whichinflows are also considered to be known), SDP modelusing both method (discrete and fuzzy interval) for the
calculation of conditional probabilities and the proposed
SFNN model (considering the CP of a fuzzy event).
0.9
0.8

Recurssive Function

0.7
0.6

SDP (Cond. Prob. of Discrete Inflow)
SDP (Cond. Prob. of Fuzzy Inflow)
SFNN (Cond. Prob. of Fuzzy Inflow)
DDP (Maximum expected value)
FNN (Maximum expected value)

0.5
0.4
0.3
0.2
0.1
0.0

St. Deviation

Min

AverageFig 11. Results after five optimization schemes(weighted-average aggregation)

It can be seen that the SFNN model presents a meanvalue slightly superior to the ones found with the SDP
models, showing the increase in the efficiency of
optimization when the SFNN model is compared to theother two SDP formulations (as values near to 1 mean
higher satisfaction and consequently more efficient
operation rules). Moreover, the minimum value of theevaluation function of the SFNN is greater, and its
standard deviation is lower than the SDP models, also
confirming the efficiency of the proposed method.
Within the two SDP models, it is possible to conclude
that the use of CP of fuzzy inflow improved theoperation performance, as the “average” value washigher (closer to 1) when using the SDP based on the CP
pf fuzzy event. The DDP and FNN formulations are only
intended to show what would be the maximum expected
results that could be obtained if future inflows were
known exactly. The determinist optimization using thesame FNN architecture also showed excellent resultscompared to its counterpart DDP.
The final storage and release sequences after the
SFNN optimization are shown in. It can be seen that the
proposed SFNN operation model is flexible enough due
to discretization of storage levels based on fuzzy
inference, which can calculate any storage level
independent of limitations originating from thediscretization process, usually observed in SDP models.
Also in Fig 12, the training and validation periods used
for all schemes are shown, which correspond to 264 and132 months, respectively.
Fig 13 shows the behavior of the objective function
throughout the generations of the GA optimization
model. As expected, there is a great increase within thefirst generations and slower improvements in the optimalvalues towards the last ones.
Besides the previous formulation, another one wascarried out for the same optimization schemes as for theprevious case, but now the product aggregation operator,
equation ( 15 ), was considered. This second test wasnecessary to guarantee that the proposed model is robust
enough, independently of the operator being used. In thiscase as well, the SFNN model showed to be superior to
the SDP schemes as can be seen in Fig 14.

Conditional Probabilities and Training (264 months)

Validation (132 months)

Storage (hm ) & Flow (m /s)

3500
3

3000
2500
2000

3

1500
1000
500
0
0

12

24

36

48

60

72

84

96

108

120

132

144

156

168

180

192

204

216

228

240

252

264

276

288

300

312

324

Release calc

Time (months)

336

348

360

372

Sf calc

384

Inflow Obs

Fig 12. Results of operation using the SFNN model
0.78

three
aggregation
operators(minimization,weighted-average and product). The operation results forthe training and validation periods are condensed as
cumulative probability function, as shown in Fig 15 andFig 16, to create a better visualization of the final results
after finding the stochastically trained optimized
operation rules.

Objective Function

0.77
0.76
0.75

Objective
Function

0.74
0.73
0.72
0

100

200

300

400

500

600

TRAINING

700

1.0

Generations

0.9
0.8

Recurssive Function

0.7
0.6

SDP (Cond. Prob. of Discrete Inflow)
SDP (Cond. Prob. of Fuzzy Inflow)
SFNN (Cond. Prob. of Fuzzy Inflow)
DDP (Maximum expected value)
FNN (Maximum expected value)

0.9

Fuzzy Objective Function

Fig 13. Results of objective function versus GA
generations

0.8
0.7
0.6

mean(ProdQT)

0.5

mean(WavQT)

0.4

mean(MinQT)
0.3

min(ProdQT)

0.2min(WavQT)

0.1

0.5

min(MinQT)

0.0
0

20

40

60

80

100

Cumulative Probability (%)

0.4
0.3

Fig 15. Objective function results after six SFNN
formulations – training

0.2
0.1
0.0

VALIDATION

St. Deviation

Min

Average

1.0

Fig 14. Results from five different optimization schemes(product aggregation)
Within the two deterministic schemes, where inflows
are assumed to be known even for the future (validation
period), the FNN formulation also showed more efficientresults for the final recursive function (“average” in Fig
14). The standard deviation (“st. deviation”) andminimum value (“min”) are calculated based on thewhole validation horizon (132 months) as done before in
Fig 11.
Finally, we investigate how the proposed model
responds to different recursive functions. The
optimization is carried out using six different SFNN
formulations depending on the combination of two
decision types (minimization and weighted-average) andFuzzy Objective Function

0.9
0.8
0.7
0.6

mean(ProdQT)

0.5

mean(WavQT)

0.4

mean(MinQT)
0.3

min(ProdQT)

0.2min(WavQT)

0.1

min(MinQT)

0.0
0

20

40

60

80

100

Cumulative Probability (%)

Fig 16. Objective function results after six SFNN
formulations - validation
It can be seen that when using the minimization
operator, the overall results were moderately decreased.
For our case study, the stochastic training of theneuro-fuzzy system using such operators seems to be lessrobust, because, even though the minimum value could
be maximized for the training period, in the validation

396

period the operation policy represented by the SFNN was
not able to return the maximum of the minimum among
the other results. That is to say, for the validation period,
the resulted minimum value was not the minimum whenusing the minimization operator, which was not reallyexpected, as this operator was supposed to maximize theminimum values. This problem may also occur withother stochastic optimization techniques as well, such as
SDP. Fig 17 shows a comparison of the minimum results
found after training and validation for the six SFNN
formulations. When looking at Fig 17, the results should
be compared within the same aggregation operator forthe two periods, training and validation.
Minimum Value of Object. Funct.

0.6

Average Decision Type (Training)
0.5

Minimization Decision Type (Training)

0.4

Average Decision Type (Validation)
Minimization Decision Type (Validation)

0.3
0.2
0.1
0.0

Average

Product

Minimization

Aggregation Operator

Fig 17. Minimum value of objective function for six
SFNN formulations
Basically, it may be said that the decision on which
aggregation method and fuzzy type of decision to use
depends significantly on the judgment of operators. Even
though in our problem the minimization decision type
did not show great robustness, in other problems withother operation purposes and different shapes for the
fuzzy objective membership functions, the minimization
can also finds its use. This decision type demonstrated an
inability to reflect any correlation among intermediate
evaluation values within the optimization (training)
period, as it always chose the smallest among different
evaluation values, which in some operation problems
may be insufficient for properly achieving efficient
operation rules. This may be aggravated when a greaternumber of objectives and longer training periods are used.
The reason is simple, a water resources system such as
reservoir operation like many other systems, is designed
to present the least risk of failure as possible. However,systems presenting risk “zero” of failure may bephysically and economically impossible to achieve.
Therefore, if one objective in a certain time of theoperation horizon, cannot be satisfied at all (membershipfunction value equals zero) the whole system is assumed
to fail. That is why the minimization type is frequently
called the pessimistic decision. It is recommended thatwhen using the SFNN for deriving operation rules forpractical use, various types of objective functions,aggregation operators and decision types should be tested.
Nevertheless, it could be said that as harder objectives
are difficultly satisfied during the training period, thenthe calculated operation rules would be less efficient.
9.

Conclusions

First, the conditional probability of a fuzzy event
proved to increase the efficiency of the stochastic DP.
This was possible as a result of the use of more realistic
conditional probability curves, found through the fuzzy
methodology presented here. This was verified after acomparison of the results from the two SDP optimization
schemes. The CP considering stochastic representative
inflows as fuzzy events is believed to reduce some of theuncertainties related to the incomplete data set and thecrisp discretization process when calculating theconditional probability of discharge values.
The results from the proposed SFNN model showed
improvements in optimization efficiency, which could beobtained as a result of the introduction of fuzzy inference
to deal with storage discretization. Moreover, the SFNN
model has the ability to consider, not only the conditional
probability but also, the long-term characteristic ofinflow, as the model is trained with actual inflow
observation data. This is different from the SDP model
which tries all combinations related to the discretizedstorage levels for a twelve-month period only, ignoring
the long-term characteristics of inflow. Moreover, the
SFNN also addresses the uncertainties of the conditional
probability as the model is prepared to calculate a new
expected conditional probability for each time stage t,
through a kind of interpolation (based on themembership functions) of the CP curves.
Different from the SDP, the forward-moving
calculation scheme for the SFNN model was
successfully carried out, making the combination of thelatter with other models straightforward, such as runoff
and water quality models. The possible combination withother models may increase the efficiency of optimization
models and may expand its applications for optimizationof more complex systems, which for instance may
involve more state variables, such as the one presented inthe next chapter.
Even though further research is necessary, having thewhole simulation being possible at once after theparameters of the neuro-fuzzy system are defined, theapplication of SFNN seems to present great potential to
overcome the problem related to the curse ofdimensionality, commonly faced by SDP models, in thecase of many state and decision variables. For example,
in the case of multi-reservoir systems, new state variables
could simply be added as new neuron units.
The results, using different fuzzy decision andaggregation operators, indicated that the SFNN is
flexible enough to optimize different types of recursivefunctions, demonstrating great potential for its
application to a variety of other kinds of stochastic
optimization problems. As could happen to othertechniques as well, the minimization decision type may
present less robust results when optimization is carriedout for systems with a greater number of operation goals
and longer optimization horizon. Moreover, the shape ofthe fuzzy objective function may also influence theperformance of the model when using the minimizationand product decision types.
After proving the efficiency of the proposed modelfor the stochastic optimization of the proposed problem,
considering only quantity objectives for the reservoir
operation, in the next chapter, a more complex problem,
considering water quality objectives is considered.
References
Bellman, Richard (1959): Dynamic Programming.
Princeton University Press. New Jersey.
Butcher, W. (1971): Stochastic dynamic programmingfor optimum reservoir operation. Water Resources
Bulletin, Vol. 7, No. 1, pp. 115-123.
Cancelliere A., Giuliano, G., Ancarani, A. and Rossi, G.
(2002): A neural networks approach for deriving
irrigation reservoir operating policies. Water Resources
Research, Vol. 16, pp. 71-88.
Chandramouli, V. and Raman, H. (2001): Multireservoir
modeling with dynamic programming and neuralnetwork. Journal of Water Resources Planning andManagement, Vol. 127, No. 2.
Chang, L.C. and Chang, F.J. (2001): Intelligent control
for modeling of real-time reservoir operation.
Hydrological Processes, Vol. 15, pp. 1621-1634.
Chaves, P., Kojiri, T and Yamashiki, Y. (2003):

Optimization of storage reservoir considering water
quantity and quality, Hydrological Processes, Vol. 17,
pp. 2769-2793.
Chaves, P. (2002): Planning Operation of Storage
Reservoir for Water Quantity and Quality, Master
Thesis, Graduate School of Engineering, Kyoto
University, Japan.
Chaves, P., Tsukatani, T. and Kojiri, T. (2004):
Operation of storage reservoir for water quality byusing optimization and artificial intelligence techniques.
Mathematics and Computers in Simulation, Vol. 67,
No. 4-5, pp. 419–432.
Esogbue, Augustine O. (1989): Dynamic Programming
for Optimal Water Resources Systems Analysis,
Prentice Hall Inc. USA.
Fontane, D.G., Gates, T.K., Moncada, E. (1997):
Planning reservoir operation with imprecise objectives.
Journal of Water Resource Planning and Management,
Vol. 123, No. 3, pp. 154-162. May/June.
Hagan, M. T., Demuth, H.B., Beale, M. H., Demuth, B.
H. (1995): Neural Network Design, PWS Publishing
Company, Boston.
Hori, T. and Takasao, T. (1997): Drought control under
the interaction between uncertainty of inflow prediction
and the fuzziness of decision criteria. Proc. Int. Conf.
On Large Scale Water Resources Development in
Developing Countries: New Dimensions of Prospects
and Problems, pp. 53-60.
Howard, R. A. (1960): Dynamic Programming andMarkov Process. MIT Press, Cambridge, Mass.
Jang, S.R. and Sun, T. (1993): Functional equivalence
between radial basis functions and fuzzy inference
system. IEEE Transaction on Neural Networks 4, pp
156 – 158.
Jang, S.R. and Sun, T. (2003): Neuro-fuzzy modelingand control. Proceedings of IEEE, 83, pp 378 – 405,
1995.
Jin, Yaochu (2003): Advanced fuzzy systems design andapplications, Physica-Verlag, New York.
Kacprzyk, Janusz. (1997): Multistage Fuzzy Control, A
model based approach to fuzzy control and decision
making. John Wiley & Sons, England.
Kaufmann, A. (1975): Introduction to the theory of fuzzy
subsets, Vol. I. Fundamental theoretical elements.
Academic Press, Orlando.
Kelman, J. and Stendinger, J.R. (1990): Sampling
stochastic dynamic programming applied to reservoir
operation. Water Resources Research, Vol. 26, No. 3,

pp 447-454.
Kojiri, T., Sugiyama, Y., Paudyal, G.N. (1993): Fuzzy
reservoir operation with multi-objective and few
monitoring data. Proceedings of the International
Conference on Environment Sound Water Resources
Utilization, Bangkok, Thailand.
Labadie, J.W. (1993): Combining simulation andoptimization in river basin management. Stochastic
Hydrology and its Use in Water Resources Systems
Simulation and Optimization, pp. 345-371. NATO ASI
Series. Kluwer Academic Publishers. The Netherlands.
Ponnambalam, K., Karray, F. and Mousavi, S.J (2003):
Minimizing variance of reservoir system operations
benefits using soft computing tools. Fuzzy Sets andSystems, Vol. 139, pp. 451-461.
Torabi, M. and Mobasheri, F. (1973): A stochastic
dynamic programming model for the optimum
operation of a multi-purpose reservoir. Water
Resources Bulletin, American Water Resources
Association, Vol. 9, No. 6, December.
Yakowitz, Sidney (1982): Dynamic programming
applications in water resource. Water Resources
Research, Vol. 18, No. 4, pp. 673-696. August.
Zadeh, L.A. (1968): Probability measures of fuzzy sets,
Journal of Mathematical Analysis and Applications 23,
pp. 421 – 427.

ニューロ・ファジーシステムを用いた多目的貯水池の確率論的操作

Paulo CHAVES*・小尻利治・堀 智晴**

京都大学大学院工学研究科

要旨
本研究では，確率的ニューラルネットワーク（ＳＦＮＮ）と名付けたシステム最適化方法を新たに提案する．それは，推計学的に習練されたニューロ・ファジーシステムであり，準最適解を生み出せる方法として定義されている。この方法は，必要性のある後退スキームをはじめとしてＳＤＰモデルに関する諸問題を克服するために考案している。本提案方法の検証を目的として，確率変数を流入量として，実ケースの貯水池
最適化と操作を用いて考察した。同一の最適化問題について本提案手法と他のＤＰアプローチとで結果を比較したところ，ＳＦＮＮは最適化の結果に改善が見られた。さらに，確率的流入量を見出す際には，流入量の離散化における不確実性を扱うためにファジー事象の条件付確率の利用を提案した。この貯水池操作の多目的最適化においては，他の目的と容易に比較できるようにファジー論理を扱った。
キーワード: 確率的最適化，ファジーニューラルネットワーク，ファジー事象の条件付確率，貯水池操作