## 3 Results and Discussion
### Application of Ensemble Prediction
#### 3.3.2 Application of Ensemble Prediction  
In order to assess the utility of this approach, we compared each forecast with the naive forecast (NF) that is average of the streamflow recorded for that month over the previous years. The Nash-Sutcliffe efficiency (NSE, [62]) and the percent bias (PBIAS) were chosen as the metrics for performance evaluation.  
NSE is a normalized value that describes the relative magnitude of the residual variance (\"noise\") compared to the observed data variance [36] and is computed as follows in Equation (12):  
\[NSE=1-\frac{\sum_{t=1}^{T}\left(Q_{o}^{t}-Q_{m}^{t}\right)^{2}}{\sum_{t=1}^{T }\left(Q_{o}^{t}-\overline{Q}_{o}\right)^{2}} \tag{12}\]  
where \(Q_{o}^{t}\) and \(Q_{m}^{t}\) is the observed and forecasted value of the dam inflow, respectively, at time \(t\), and \(\overline{Q}_{o}\) is the mean observed value. NSE can range from \(-\infty\) to 1 and the coefficient of 1 means a perfect match of predicted value to the observed data. The coefficient of 0 indicate that the model predictions are same as the mean of the observed data, whereas the coefficients less than zero indicate that the observed mean is more accurate than the model's predicted value. Consequently, the closer the model's efficiency coefficient is to 1, the more accurate the model's prediction is.  
The PBIAS was calculated according to the Equation (13):  
\[PBIAS\left(\%\right)=\frac{\sum_{t=1}^{T}\left(Q_{o}^{t}-Q_{m}^{t}\right)}{ \sum_{t=1}^{T}\left(Q_{o}^{t}\right)}\times 100 \tag{13}\]  
The PBIAS is the forecast error normalized by the observation and quantifies the tendency of predicted value being either greater (over-forecast: negative values) or smaller (under-forecast: positive values) than the observed data.  
NSE values greater than 0.5 and PBIAS within 25% are considered \"adequate\" [63]. Table 6 shows the forecast results of SMA, BMA and NF with the value of NSE, PBIAS. BMA results showed that NSE was above 0.7 and PBIAS values were within 25% for all years. BMA and SMA methods showed more reliable prediction results than NF. Overall, the BMA method resulted in more accurate forecasts than the other two methods. In addition, the 95% prediction uncertainty ranges of the BMA method for the target year 2016, which is given as an example, envelop most observations in Figure 7. The dam watershed suffered severe meteorological droughts in 2015 with precipitation of 668.6 mm/year, approximately 60% of 40-year (1976-2015) average precipitation (1188 mm/year) [22]. The forecasting results show that the BMA method presented reliable predictability in drought year. These results showed the applicability that the proposed models could be used as a decision-making tool to manage dam operation for water supply in the context of BMA.  
\begin{table}
\begin{tabular}{c c c c c c c} \hline \hline \multirow{2}{*}{**Year**} & \multicolumn{2}{c}{**BMA**} & \multicolumn{2}{c}{**SMA**} & \multicolumn{2}{c}{**Naive Forecast**} \\  & **NSE** & **PBIAS (\%)** & **NSE** & **PBIAS (\%)** & **NSE** & **PBIAS (\%)** \\ \hline
2013 & 0.78 & 6.45 & 0.77 & 14.41 & \(-\)0.02 & 37.37 \\
2014 & 0.77 & 3.08 & 0.52 & 10.37 & \(-\)1.68 & 14.92 \\
2015 & 0.89 & \(-\)1.33 & 0.71 & \(-\)7.11 & \(-\)1.34 & 36.41 \\
2016 & 0.71 & \(-\)21.87 & 0.43 & \(-\)36.34 & 0.31 & 44.19 \\ \hline \hline \end{tabular}
\end{table}
Table 6: Performance statistics for Bayesian model averaging (BMA), simple model averaging (SMA) and naive forecast (NF).  
Figure 7: _Cont._Figure 7: Comparison of forecasts between SMA and BMA for January (**a**), February (**b**), March (**c**), April (**d**), May (**e**), November (**f**) and December (**g**) of target year 2016. The red line displays the mean point forecast of the BMA method, whereas the block dots are observations. Grey region indicates 95% prediction interval.