# Forecast Customization System (FOCUS): A Multimodel Ensemble-Based Seasonal Climate Forecasting Tool for the Homogeneous Climate Zones of Myanmar

Authors: Itesh Dash; Masahiko Nagai; Indrajit Pal
Journal: Advances in Meteorology
 (2019)

---

## Abstract

A Multi-Model Ensemble (MME) based seasonal rainfall forecast customization tool called FOCUS was developed for Myanmar in order to provide improved seasonal rainfall forecast to the country. The tool was developed using hindcast data from 7 Global Climate Models (GCMs) and observed rainfall data from 49 meteorological surface observatories for the period of 1982 to 2011 from the Department of Meteorology and Hydrology. Based on the homogeneity in terms of the rainfall received annually, the country was divided into six climatological zones. Three different operational MME techniques, namely, (a) arithmetic mean (AM-MME), (b) weighted average (WA-MME), and (c) supervised principal component regression (PCR-MME), were used and built-in to the tool developed. For this study, all 7 GCMs were initialized with forecast data of May month to predict the rainfall during June to September (JJAS) period, which is the predominant rainfall season for Myanmar. The predictability of raw GCMs, bias-corrected GCMs, and the MMEs was evaluated using RMSE, correlation coefficients, and standard deviations. The probabilistic forecasts for the terciles were also evaluated using the relative operating characteristics (ROC) scores, to quantify the uncertainty in the GCMs. The results suggested that MME forecasts have shown improved performance (RMSE = 1.29), compared to the raw individual models (ECMWF, which is comparatively better among the selected models) with RMSE = 4.4 and bias-corrected RMSE = 4.3, over Myanmar. Specifically, WA-MME (CC = 0.64) and PCR-MME (CC = 0.68) methods have shown significant improvement in the high rainfall (delta) zone compared with WA-MME (CC = 0.57) and PCR-MME (CC = 0.56) techniques for the southern zone. The PCR method suggests higher predictability skill for the upper tercile (ROC = 0.78) and lower tercile categories (ROC = 0.85) for the delta region and is less skillful over lower rainfall zones like dry zones with ROC = 0.6 and 0.63 for upper and lower terciles, respectively. The model is thus suggested to perform relatively well over the higher rainfall (Wet) zones compared to the lower (Dry) zone during the JJAS period.


---

## Body

## 1. Introduction

Rainfall in Myanmar is highly variable over space and time, largely because of a varied topography and multiple environmental influences. It is directly impacted by the Indian/South Asian monsoon systems as well as convective rainfall from the Bay of Bengal [1, 2]. The strength of seasonal rainfall in the country, to some extent, is influenced by the large-scale climate drivers such as El Niño Southern Oscillation (ENSO) and Indian Ocean Dipole (IOD) [1, 3–5]. According to the Department of Meteorology and Hydrology (DMH), it is observed that ENSO’s warm phase (El Niño) has resulted in deficient rainfall and higher temperatures, while La Niña, the cold phase, tends to have opposite impacts in the country [6]. Presence of such a teleconnection between the large-scale phenomena and the local climate of Myanmar is expected to enhance seasonal prediction.DMH’s operational seasonal forecasting is based on the analogue method [7]. According to this method, rainfall patterns associated with historical ENSO phases (El Niño and La Niña) is likely to re-occur during similar ENSO phases in future. The prediction of the present year would depend on the years with similar ENSO phases in the past. It was recommended that an improved seasonal forecasting system is required, despite the traditional method followed by DMH, during a user interaction forum conducted by DMH every year [8]. Furthermore, capacity self-assessment exercise conducted by RIMES [9] identified the need for development of a standard platform, in order to assist DMH in generating seasonal climate forecast and assist in analyzing different global models. Moreover, need for empirical studies focusing on rainfall variability and forecasting for operational applications in agriculture and water resource sector was emphasized in Asia [10–12] and Africa [13].Global climate models (GCMs) are useful tools for predicting seasonal climate; however, there is large uncertainty in its prediction, mainly because of the assumptions made in the initial atmospheric state [14]. To simulate and capture these uncertainties in the predictions, GCMs are processed with different initial conditions to generate multiple forecast members called the ensembles [15, 16]. Multimodel ensemble (MME) is a process, where ensemble members of one GCM are statistically assembled with another GCM or a set of GCMs [17–21]. The MME approach has increasingly demonstrated better prediction skills over the tropical Asian region in long-range forecasting, when compared to individual model performances [20, 22–24]. For instance, the MME system developed over India for monthly/seasonal prediction in real time during the South Asian monsoon season exhibited satisfactory predictions [25]. Similarly, North American MMEs (NMME) showed lower systematic error and higher forecast skills compared to the individual members over Southeast Asian region [26].MME schemes can be developed using various statistical methods: (1) by simply taking mean of all ensembles with assigning equal weight to individual ensemble members [20] or (2) by assigning higher weightage to the statistically significant members of GCMs according to their performance over the hindcast period [20, 24, 27–29] or using complex neural network algorithms. It is, however, a well-established concept that MMEs would be useful schemes for generating improved seasonal outlook. But so far, no attempts were made to develop a long-range prediction system for Myanmar using such type of advanced techniques. Tools, such as the Climate Prediction Tool (CPT) [30], Climate Information Toolkit (CLIK) [31] and Seasonal Climate Outlook in the Pacific Island Countries (SCOPIC) [32], have the functionalities to perform statistical analysis with climate data, but are limited in terms of their utility. For instance, CLIK is useful for providing predictions at regional scale, but not at the spatial scale suggested in this current study. CPT has the capability to do prediction specific to locations but cannot perform MME-based predictions. SCOPIC is designed to predict seasonal climate only for the Pacific Island countries and not yet applicable for other regions [33].The objective of the present work is to overcome the abovementioned limitations and to develop a web-based graphical user interface (web-GUI) forecast customization system tailored for national use. The tool allows generation of monthly and seasonal climate outlooks using the MME techniques and assist in evaluating the performance of outlooks. The tool is also capable of providing the outlooks for the defined climatological zones in Myanmar. The method will be described in the successive section.


## 2. Study Area

### 2.1. Zone Classification

Myanmar is geographically situated in Southeast Asia between latitudes 09° 32′ N and 28° 31′ N and longitudes 9° 10′ E and 101° 11′ E. Myanmar is climatologically divided into six major zones (Figures1(a) and 1(b), [10]): (1) central dry zone, which has the lowest average annual rainfall and intense agricultural practices; (2) eastern zone (shan); (3) northern zone which is mostly with high terrain and forest areas; (4) coastal (Rakhine); (5) delta zone (Ayeyarwady region); and (6) southern zone, which receives the maximum annual rainfall zones. This also synchronized well with classifications done in the past in references [1, 5, 20]. These classifications are based on the average annual rainfall in the country (Figure 2), agroecological zoning, and seasonal rainfall patterns, respectively.Figure 1
(a) Topography of Myanmar. (b) Rainfall during June to September overlaid with meteorological surface observatory and climatological zones. (c) Monthly climatology of Myanmar.Figure 2
Annual cycle of rainfall climatology for all six homogeneous zones (bar graph) and overall Myanmar (line graph).


### 2.2. Climatology of Myanmar

The annual rainfall cycle in Figure2 shows distribution of rainfall mostly concentrated over the JJAS period, which is mainly due to the influence of the southwest monsoon (Figure 1(c)). The monsoon onset is marked during May, peak during August, and withdrawal towards the end of September [34]. The spatial distribution of rainfall however varies significantly during this period over all the zones. The central dry zone receives the lowest amount of seasonal rainfall, while the southern region receives the highest amount [34]. As JJAS is major rainfall season for all the zones of the country, this study focused on investigating characteristics of rainfall over JJAS and also predicting seasonal rainfall for its operational application in agricultural and water resources management sector.


## 2.1. Zone Classification

Myanmar is geographically situated in Southeast Asia between latitudes 09° 32′ N and 28° 31′ N and longitudes 9° 10′ E and 101° 11′ E. Myanmar is climatologically divided into six major zones (Figures1(a) and 1(b), [10]): (1) central dry zone, which has the lowest average annual rainfall and intense agricultural practices; (2) eastern zone (shan); (3) northern zone which is mostly with high terrain and forest areas; (4) coastal (Rakhine); (5) delta zone (Ayeyarwady region); and (6) southern zone, which receives the maximum annual rainfall zones. This also synchronized well with classifications done in the past in references [1, 5, 20]. These classifications are based on the average annual rainfall in the country (Figure 2), agroecological zoning, and seasonal rainfall patterns, respectively.Figure 1
(a) Topography of Myanmar. (b) Rainfall during June to September overlaid with meteorological surface observatory and climatological zones. (c) Monthly climatology of Myanmar.Figure 2
Annual cycle of rainfall climatology for all six homogeneous zones (bar graph) and overall Myanmar (line graph).


## 2.2. Climatology of Myanmar

The annual rainfall cycle in Figure2 shows distribution of rainfall mostly concentrated over the JJAS period, which is mainly due to the influence of the southwest monsoon (Figure 1(c)). The monsoon onset is marked during May, peak during August, and withdrawal towards the end of September [34]. The spatial distribution of rainfall however varies significantly during this period over all the zones. The central dry zone receives the lowest amount of seasonal rainfall, while the southern region receives the highest amount [34]. As JJAS is major rainfall season for all the zones of the country, this study focused on investigating characteristics of rainfall over JJAS and also predicting seasonal rainfall for its operational application in agricultural and water resources management sector.


## 3. Data and Methods

### 3.1. GCM Data

Hindcast rainfall data from seven GCMs (listed in Table1) are obtained for the period 1982–2011. All the GCM hindcast datasets are at monthly temporal scale. Four fully coupled GCMs, namely, National Center for Environment Prediction’s Coupled Forecast System Model version 2 (NCEP CFSv2) [37], Geophysical Fluid Dynamics Laboratory (GFDL), COLA, and the System 4 (ECMWF Technical Memorandum, 2011) European Center for Medium-Range Weather Forecast (ECMWF) are used in the study. The CFSv2 model has a spectral triangular truncation of 126 waves (T126) horizontally (equivalent to nearly a 100 km grid resolution) and a finite differencing vertically with 64 sigma-pressure hybrid layers. The atmospheric component in this model is the Global Forecast System (GFS 2009) while Geophysical Fluid Dynamics Laboratory Modular Ocean Model 4 (GFDL MOM4) is considered as the oceanic component. The retrospective 9-month forecasts have initial conditions of the 0000, 0600, 1200, and 1800 UTC cycles for every 5th day, starting from 0000 UTC 1 January of every year. COLA and GFDL models are considered from the US National Multimodel Ensemble (NMME) project phase-II [36]. The COLA forecasts are made with the NCAR CCMv3.6 [35], a coupled climate model with components representing atmosphere, ocean, sea ice, and land surface connected by a flux coupler. Three 2-tier models, ECHAM4.5 CASST, ECHAM4.5 CFSSST, and CCMv3.6, are used. The ECHAM4.5 CASST model is forced with Constructed Analogue (CA) Sea Surface Temperature (SST) [38] as boundary conditions over tropical oceans (30S-30N), and CFSSST is forced with the Climate Forecasting System (CFS) SST data. The rainfall data for these global climate models are accessed from the International Research Institute data library available online at http://iridl.ldeo.columbia.edu/ [39]. Hindcast data for NCEP CFSv2 are downloaded from http://cfs.ncep.noaa.gov/cfsv2/downloads.html. The System 4 hindcast data from the ECMWF are retrieved from the Meteorological Archival and Retrieval System (MARS) available online at http://apps.ecmwf.int/archive-catalogue/.Table 1
Details of GCM datasets used.
Model
Resolution
Model type
Ensemble size
Source
Community climate system model (CCMv3.6)
2.813° × 2.789°
2-tier
24
National center for atmospheric research, Boulder, USA [35]
Center for ocean-land-atmosphere (COLA)
1.875° × 1.864°
Fully coupled
10
The center for ocean-land-atmosphere studies, Fairfax, USA [36]
Geophysical fluid dynamics laboratory (GFDL)
2.500° × 2.000°
Fully coupled
10
Geophysical fluid dynamics laboratory, Princeton, USA [36]
ECHAM 4.5 CA SST
2.813° × 2.789°
2-tier
24
Max Planck institute for meteorology, Denmark (Li and Goddard 2005)
ECHAM 4.5 CFS SST
2.813° × 2.789°
2-tier
24
Max Planck institute for meteorology, Denmark (CFS-predicted SST)
European center for medium-range weather forecasting (ECMWF)
1.500° × 1.500°
Fully coupled
41
European center for medium-range weather forecasting, reading, UK
Climate foresting system version 2 (CFSv2)
1.000° × 1.000°
Fully coupled
24
Climate prediction center


### 3.2. Observation Data

Observation rainfall data at daily time-step from 70 surface observatories for the period of 1982–2011 are obtained from the Department of Meteorology and Hydrology (DMH), Myanmar. However, data from only 49 stations are considered (shown in Figure1(b)) for this study based on the following quality checks: climatological and temporal checks, data homogeneity test, factoring human error, and percentage of missing data [40].


### 3.3. Methods

A complete schematic of the method is described in Figure3, which involves data acquisition from different global centers, data preparation and processing, bias correction and development of MME schemes, generation of probabilistic forecast, and finally evaluation of the model skill. These steps are described in the subsequent sections.Figure 3
Simplified methodology for the model development and forecast customization and generation of MME-based seasonal probabilistic forecast along with model skill evaluation.


### 3.4. Data Preparation

GCM hindcast datasets are maintained in different formats by different global producing centers (GPCs). For example, IRI data library stores data in sequential binary format, CFSv2 datasets are in gridded binary (grib2) format, and ECMWF MARS datasets are available in either Network Common Data Format (NetCDF) or grib2. At the same time, the synoptic observation datasets accessed from DMH are in the simple text (ASCII) format. Therefore, a data normalization algorithm was developed using Python programming language to bring all data to a standard format (.mat) to handle the data more efficiently.


### 3.5. Data Processing

The proposed methods would use the hindcast data to train the model; therefore, it is essential to combine the hindcast data with the forecast data, for the same forecast initialization month. For example, the study uses the May initial data for the prediction of JJAS. Therefore, it is required to combine hindcast data of May (Mayhc_1982–2011) with forecast data for May (Mayfc_2018). The model is chosen for the period from 1982 to 2011 to match with the observation data availability period. The data are then interpolated to a preferred resolution of 0.25° (∼30 km) using the bilinear interpolation method [41]. As the target spatial resolution of the seasonal prediction is at the climate zones, the rainfall data for both GCMs and observation are averaged over these zones. Furthermore, bias correction methods and different MME schemes are applied to datasets to generate bias-corrected deterministic forecast and probabilistic seasonal forecast for the defined climatological zones.


### 3.6. Model Bias Reduction

As global models exhibit large bias in simulating seasonal rainfall, the bias needs to be removed or minimized in order to provide skillful forecast. Several bias correction techniques are available, in which the quantile-to-quantile mapping method is widely used and proven to be effective for the Indian summer monsoon period [42]. The method removes systematic bias in the GCM simulations, using the inverse of cumulative distribution function (CDF) of observed values (Fob) at the probability corresponding to the ensemble mean output CDF (Fem) at the particular value. Then, for Ft¯, the bias-corrected forecast (Fbc) would be represented as(1)Fbc=Fob−1FemFt¯.This study utilized quantile mapping method to remove the systematic bias in the GCMs before they were used in the MME algorithms.


### 3.7. Development of MME Schemes

MME is a process of statistically assembling different global models. Therefore, in the MME process, n number of global models with t number of years of hindcast runs are statistically ensembled to construct a prediction for thet + 1 year. For example, the current study used 7 GCMs (n = 7), with 30 years of hindcast runs (t = 30), to provide prediction for the year 2018 (t + 1). A GCM will be considered only if it has more than one ensemble member. Table 1 lists the total number of ensemble members available for each global model. In this study, three different statistical ensemble MME schemes are used: (a) arithmetic mean multimodel ensemble (AM-MME), (b) weighted average multimodel ensemble (WA-MME), and (c) supervised principal component regression multimodel ensemble (PCR-MME). The MME schemes collectively make use of all the members to generate the final ensemble forecast.AM-MME is a simple averaging scheme of all individual model ensembles [20, 43]. All individual members of models are assigned with equal weight with the assumption that all models considered in this MME scheme predict the seasonal rainfall with uniform skills. All model forecast data are normalized by removing the mean (average calculated for the period 1982–2011) from the time series, and the observed interannual trend is added to derive forecast time series. The AM-MME forecast constructed with bias-corrected forecast data can be represented as(2)St=O¯+1N∑i=1NFi,t−Fi¯σFiσ0,where St = MME prediction at time t, Fi, t = ith model forecast at time t, Fi¯ = climatology of ith model forecast, O¯ = climatology of observations, σFi = interannual variation of ith model forecast, σ0 = interannual variation of observations, and N = no. of models.In the WA-MME scheme, a regression coefficient for each ensemble is obtained for the training phase (t) by using the singular value decomposition (SVD) technique. The regression coefficient assigns a weight to each ensemble based on the training data which is then used in computing a robust weighted average forecast [44] for the time t + 1. The WA-MME forecast is constructed with bias-corrected data using the following equation:(3)St=O¯+ai∑i=1NFi,t−Fi¯σFiσ0,where ai = regression coefficient obtained by a minimization procedure during the training period between model’s forecasts Fi’s and observation O. Other variables are the same as in the AM-MME scheme.The supervised principal component regression (SPCR) method is primarily used to eliminate presence of any significant correlation among individual models [45]. It is a dimension reduction/transformation technique to minimize the number of independent variables that describe the maximum variance of all variables. The prediction model considered in this scheme is based on the concept of principal component analysis (PCA), where the principal components (PCs) are calculated after the eigenvector decomposition of a correlation matrix. In this method, the principal components are considered for the regression process [25]. The PCs are selected based on their correlation with the observation (predictand) unlike the traditional PCR technique, where they are chosen according to their variances. PCs selection based on correlation would be very useful for choosing meaning predictors. The SPCR method ensures that predictors with higher correlation are selected for regression and forecast generation.


### 3.8. FOCUS: The GUI

The graphical user interface (GUI, see Figure4) is developed using a combination of Python programming language, for the backend operations such as processing data, performing statistical analysis, and developing statistical methods to generate forecast products. The front end was designed using the Microsoft .net framework as a web-based platform. The tool can be accessed from the following link: http://203.159.16.146/ForecastWeb/Login.aspx. Web data retrieval package, “wget,” is used at the backend to automatically download required global forecast dataset from the respective websites. FOCUS tool has built-in functionalities for data processing, combining and interpolation, bias correction, and generating ensemble probabilistic forecasts. The tool also utilized the superensemble technique to generate combined and reconstructed products with ensemble of MME forecasts [22]. Additionally, the tool can perform model forecast skill evaluation in terms of ROC score and forecast reliability.Figure 4
Screen capture of the Forecast Customization System (FOCUS) GUI developed using Python programming language (MME1 and MME2 refers to the AM-AMME and WA-MME schemes, respectively), showing the ROC score generation for the tercile categories.


### 3.9. Generation of Probabilistic Forecast

One of the best ways to express uncertainty in a consistent and verifiable way is through probability forecasts [14]. A probability forecast specifies how likely a defined event is to occur [46]. In the study, GCM ensemble members are used for estimation of the probability through the sampling method and identifying the possible range of forecasts. Deterministic forecasts produced from the MMEs are used to generate probabilistic forecast based on the observed climatology, meaning with equal (∼33%) chance of occurrence for each tercile category. Probability of an event can be defined with an event Ω as occurrence of X (rainfall) in an interval (x1, x2).IfF xβ is the distribution of the predictand X conditional on a given value of β, then the probability that X lies in an interval (x1, x2) conditional on β is represented as(4)PxΩβ=ProbXεx1,x2β.With Gaussian noiseε, the conditional probability can be expressed as(5)PxΩβ;σε=FNX2−βσε−FNX1−βσε,where FN is the distribution function of the standard normal distribution. The probability depends both on the value of β and the standard deviation of ε.As mentioned earlier, probabilistic predictions are generated for three tercile categories: (i) below normal, (ii) near normal, and (iii) above normal, in reference to the observed climatology and with the notion that each category has equal chance of manifestation. Finally, deterministic forecast is used as the mean of the forecast distribution, whereas the spread is calculated by the correlation method [29, 47] and the corresponding conditional probabilities of the events are given by(6)PxBβ;σε=FN−β−Xaσε,PxAβ;σε=FNβ−Xaσε,PxNβ;σε=1−PxBβ;σε−PxAβ;σε,and, FN, again, is the distribution function of the standard normal distribution and xa and xb are the boundaries.


### 3.10. Module for MME Performance Evaluation

Several standard techniques such as box and whisker plots, relative operating characteristics (ROC) plots, and Taylor diagrams are available to evaluate prediction skills of models. Box and whisker plot [48, 49] is used to interpret the distribution and variability. ROC is used for evaluating the skill of the probabilistic forecast performance [46].


### 3.11. ROC Curve

ROC curves are two-dimensional measure of classification performance and feature the underlying distribution of forecasts [50]. ROC curves are graphs constructed with hit rates (Hr) and false alarm rates (Fr) for the three different tercile categories. ROC area skill score (ROCASS) is a validation index about the probability forecasts with no value of information, i.e., Hr = Fr, and defined by(7)ROCASS≡2ROCA−0.5,0≤ROCASS≤1.ROCASS is the unit for quantifying the forecast, where a score zero to 0.5 represents no forecast skill, a score between >0.5 to 1 indicates a more skillful forecast, and any score ∼0.5 or less suggests no skill [50].


### 3.12. Taylor Diagram

Taylor diagram [51] provides a concise statistical summary of how well patterns match each other, in terms of their correlation coefficient, their root-mean-square difference (RMSE), and the ratio of their variances. These plots are used to devise skill scores that appropriately weight among the various measures of pattern correspondence.Mathematically, the three statistics displayed on a Taylor diagram are related by the following formula:(8)E′2=σr2+σt2−2σrσtρ,where E′ = centered RMS difference of observation and the prediction, ρ = correlation coefficient, and σr,σt = variances of the observation and the prediction.


## 3.1. GCM Data

Hindcast rainfall data from seven GCMs (listed in Table1) are obtained for the period 1982–2011. All the GCM hindcast datasets are at monthly temporal scale. Four fully coupled GCMs, namely, National Center for Environment Prediction’s Coupled Forecast System Model version 2 (NCEP CFSv2) [37], Geophysical Fluid Dynamics Laboratory (GFDL), COLA, and the System 4 (ECMWF Technical Memorandum, 2011) European Center for Medium-Range Weather Forecast (ECMWF) are used in the study. The CFSv2 model has a spectral triangular truncation of 126 waves (T126) horizontally (equivalent to nearly a 100 km grid resolution) and a finite differencing vertically with 64 sigma-pressure hybrid layers. The atmospheric component in this model is the Global Forecast System (GFS 2009) while Geophysical Fluid Dynamics Laboratory Modular Ocean Model 4 (GFDL MOM4) is considered as the oceanic component. The retrospective 9-month forecasts have initial conditions of the 0000, 0600, 1200, and 1800 UTC cycles for every 5th day, starting from 0000 UTC 1 January of every year. COLA and GFDL models are considered from the US National Multimodel Ensemble (NMME) project phase-II [36]. The COLA forecasts are made with the NCAR CCMv3.6 [35], a coupled climate model with components representing atmosphere, ocean, sea ice, and land surface connected by a flux coupler. Three 2-tier models, ECHAM4.5 CASST, ECHAM4.5 CFSSST, and CCMv3.6, are used. The ECHAM4.5 CASST model is forced with Constructed Analogue (CA) Sea Surface Temperature (SST) [38] as boundary conditions over tropical oceans (30S-30N), and CFSSST is forced with the Climate Forecasting System (CFS) SST data. The rainfall data for these global climate models are accessed from the International Research Institute data library available online at http://iridl.ldeo.columbia.edu/ [39]. Hindcast data for NCEP CFSv2 are downloaded from http://cfs.ncep.noaa.gov/cfsv2/downloads.html. The System 4 hindcast data from the ECMWF are retrieved from the Meteorological Archival and Retrieval System (MARS) available online at http://apps.ecmwf.int/archive-catalogue/.Table 1
Details of GCM datasets used.
Model
Resolution
Model type
Ensemble size
Source
Community climate system model (CCMv3.6)
2.813° × 2.789°
2-tier
24
National center for atmospheric research, Boulder, USA [35]
Center for ocean-land-atmosphere (COLA)
1.875° × 1.864°
Fully coupled
10
The center for ocean-land-atmosphere studies, Fairfax, USA [36]
Geophysical fluid dynamics laboratory (GFDL)
2.500° × 2.000°
Fully coupled
10
Geophysical fluid dynamics laboratory, Princeton, USA [36]
ECHAM 4.5 CA SST
2.813° × 2.789°
2-tier
24
Max Planck institute for meteorology, Denmark (Li and Goddard 2005)
ECHAM 4.5 CFS SST
2.813° × 2.789°
2-tier
24
Max Planck institute for meteorology, Denmark (CFS-predicted SST)
European center for medium-range weather forecasting (ECMWF)
1.500° × 1.500°
Fully coupled
41
European center for medium-range weather forecasting, reading, UK
Climate foresting system version 2 (CFSv2)
1.000° × 1.000°
Fully coupled
24
Climate prediction center


## 3.2. Observation Data

Observation rainfall data at daily time-step from 70 surface observatories for the period of 1982–2011 are obtained from the Department of Meteorology and Hydrology (DMH), Myanmar. However, data from only 49 stations are considered (shown in Figure1(b)) for this study based on the following quality checks: climatological and temporal checks, data homogeneity test, factoring human error, and percentage of missing data [40].


## 3.3. Methods

A complete schematic of the method is described in Figure3, which involves data acquisition from different global centers, data preparation and processing, bias correction and development of MME schemes, generation of probabilistic forecast, and finally evaluation of the model skill. These steps are described in the subsequent sections.Figure 3
Simplified methodology for the model development and forecast customization and generation of MME-based seasonal probabilistic forecast along with model skill evaluation.


## 3.4. Data Preparation

GCM hindcast datasets are maintained in different formats by different global producing centers (GPCs). For example, IRI data library stores data in sequential binary format, CFSv2 datasets are in gridded binary (grib2) format, and ECMWF MARS datasets are available in either Network Common Data Format (NetCDF) or grib2. At the same time, the synoptic observation datasets accessed from DMH are in the simple text (ASCII) format. Therefore, a data normalization algorithm was developed using Python programming language to bring all data to a standard format (.mat) to handle the data more efficiently.


## 3.5. Data Processing

The proposed methods would use the hindcast data to train the model; therefore, it is essential to combine the hindcast data with the forecast data, for the same forecast initialization month. For example, the study uses the May initial data for the prediction of JJAS. Therefore, it is required to combine hindcast data of May (Mayhc_1982–2011) with forecast data for May (Mayfc_2018). The model is chosen for the period from 1982 to 2011 to match with the observation data availability period. The data are then interpolated to a preferred resolution of 0.25° (∼30 km) using the bilinear interpolation method [41]. As the target spatial resolution of the seasonal prediction is at the climate zones, the rainfall data for both GCMs and observation are averaged over these zones. Furthermore, bias correction methods and different MME schemes are applied to datasets to generate bias-corrected deterministic forecast and probabilistic seasonal forecast for the defined climatological zones.


## 3.6. Model Bias Reduction

As global models exhibit large bias in simulating seasonal rainfall, the bias needs to be removed or minimized in order to provide skillful forecast. Several bias correction techniques are available, in which the quantile-to-quantile mapping method is widely used and proven to be effective for the Indian summer monsoon period [42]. The method removes systematic bias in the GCM simulations, using the inverse of cumulative distribution function (CDF) of observed values (Fob) at the probability corresponding to the ensemble mean output CDF (Fem) at the particular value. Then, for Ft¯, the bias-corrected forecast (Fbc) would be represented as(1)Fbc=Fob−1FemFt¯.This study utilized quantile mapping method to remove the systematic bias in the GCMs before they were used in the MME algorithms.


## 3.7. Development of MME Schemes

MME is a process of statistically assembling different global models. Therefore, in the MME process, n number of global models with t number of years of hindcast runs are statistically ensembled to construct a prediction for thet + 1 year. For example, the current study used 7 GCMs (n = 7), with 30 years of hindcast runs (t = 30), to provide prediction for the year 2018 (t + 1). A GCM will be considered only if it has more than one ensemble member. Table 1 lists the total number of ensemble members available for each global model. In this study, three different statistical ensemble MME schemes are used: (a) arithmetic mean multimodel ensemble (AM-MME), (b) weighted average multimodel ensemble (WA-MME), and (c) supervised principal component regression multimodel ensemble (PCR-MME). The MME schemes collectively make use of all the members to generate the final ensemble forecast.AM-MME is a simple averaging scheme of all individual model ensembles [20, 43]. All individual members of models are assigned with equal weight with the assumption that all models considered in this MME scheme predict the seasonal rainfall with uniform skills. All model forecast data are normalized by removing the mean (average calculated for the period 1982–2011) from the time series, and the observed interannual trend is added to derive forecast time series. The AM-MME forecast constructed with bias-corrected forecast data can be represented as(2)St=O¯+1N∑i=1NFi,t−Fi¯σFiσ0,where St = MME prediction at time t, Fi, t = ith model forecast at time t, Fi¯ = climatology of ith model forecast, O¯ = climatology of observations, σFi = interannual variation of ith model forecast, σ0 = interannual variation of observations, and N = no. of models.In the WA-MME scheme, a regression coefficient for each ensemble is obtained for the training phase (t) by using the singular value decomposition (SVD) technique. The regression coefficient assigns a weight to each ensemble based on the training data which is then used in computing a robust weighted average forecast [44] for the time t + 1. The WA-MME forecast is constructed with bias-corrected data using the following equation:(3)St=O¯+ai∑i=1NFi,t−Fi¯σFiσ0,where ai = regression coefficient obtained by a minimization procedure during the training period between model’s forecasts Fi’s and observation O. Other variables are the same as in the AM-MME scheme.The supervised principal component regression (SPCR) method is primarily used to eliminate presence of any significant correlation among individual models [45]. It is a dimension reduction/transformation technique to minimize the number of independent variables that describe the maximum variance of all variables. The prediction model considered in this scheme is based on the concept of principal component analysis (PCA), where the principal components (PCs) are calculated after the eigenvector decomposition of a correlation matrix. In this method, the principal components are considered for the regression process [25]. The PCs are selected based on their correlation with the observation (predictand) unlike the traditional PCR technique, where they are chosen according to their variances. PCs selection based on correlation would be very useful for choosing meaning predictors. The SPCR method ensures that predictors with higher correlation are selected for regression and forecast generation.


## 3.8. FOCUS: The GUI

The graphical user interface (GUI, see Figure4) is developed using a combination of Python programming language, for the backend operations such as processing data, performing statistical analysis, and developing statistical methods to generate forecast products. The front end was designed using the Microsoft .net framework as a web-based platform. The tool can be accessed from the following link: http://203.159.16.146/ForecastWeb/Login.aspx. Web data retrieval package, “wget,” is used at the backend to automatically download required global forecast dataset from the respective websites. FOCUS tool has built-in functionalities for data processing, combining and interpolation, bias correction, and generating ensemble probabilistic forecasts. The tool also utilized the superensemble technique to generate combined and reconstructed products with ensemble of MME forecasts [22]. Additionally, the tool can perform model forecast skill evaluation in terms of ROC score and forecast reliability.Figure 4
Screen capture of the Forecast Customization System (FOCUS) GUI developed using Python programming language (MME1 and MME2 refers to the AM-AMME and WA-MME schemes, respectively), showing the ROC score generation for the tercile categories.


## 3.9. Generation of Probabilistic Forecast

One of the best ways to express uncertainty in a consistent and verifiable way is through probability forecasts [14]. A probability forecast specifies how likely a defined event is to occur [46]. In the study, GCM ensemble members are used for estimation of the probability through the sampling method and identifying the possible range of forecasts. Deterministic forecasts produced from the MMEs are used to generate probabilistic forecast based on the observed climatology, meaning with equal (∼33%) chance of occurrence for each tercile category. Probability of an event can be defined with an event Ω as occurrence of X (rainfall) in an interval (x1, x2).IfF xβ is the distribution of the predictand X conditional on a given value of β, then the probability that X lies in an interval (x1, x2) conditional on β is represented as(4)PxΩβ=ProbXεx1,x2β.With Gaussian noiseε, the conditional probability can be expressed as(5)PxΩβ;σε=FNX2−βσε−FNX1−βσε,where FN is the distribution function of the standard normal distribution. The probability depends both on the value of β and the standard deviation of ε.As mentioned earlier, probabilistic predictions are generated for three tercile categories: (i) below normal, (ii) near normal, and (iii) above normal, in reference to the observed climatology and with the notion that each category has equal chance of manifestation. Finally, deterministic forecast is used as the mean of the forecast distribution, whereas the spread is calculated by the correlation method [29, 47] and the corresponding conditional probabilities of the events are given by(6)PxBβ;σε=FN−β−Xaσε,PxAβ;σε=FNβ−Xaσε,PxNβ;σε=1−PxBβ;σε−PxAβ;σε,and, FN, again, is the distribution function of the standard normal distribution and xa and xb are the boundaries.


## 3.10. Module for MME Performance Evaluation

Several standard techniques such as box and whisker plots, relative operating characteristics (ROC) plots, and Taylor diagrams are available to evaluate prediction skills of models. Box and whisker plot [48, 49] is used to interpret the distribution and variability. ROC is used for evaluating the skill of the probabilistic forecast performance [46].


## 3.11. ROC Curve

ROC curves are two-dimensional measure of classification performance and feature the underlying distribution of forecasts [50]. ROC curves are graphs constructed with hit rates (Hr) and false alarm rates (Fr) for the three different tercile categories. ROC area skill score (ROCASS) is a validation index about the probability forecasts with no value of information, i.e., Hr = Fr, and defined by(7)ROCASS≡2ROCA−0.5,0≤ROCASS≤1.ROCASS is the unit for quantifying the forecast, where a score zero to 0.5 represents no forecast skill, a score between >0.5 to 1 indicates a more skillful forecast, and any score ∼0.5 or less suggests no skill [50].


## 3.12. Taylor Diagram

Taylor diagram [51] provides a concise statistical summary of how well patterns match each other, in terms of their correlation coefficient, their root-mean-square difference (RMSE), and the ratio of their variances. These plots are used to devise skill scores that appropriately weight among the various measures of pattern correspondence.Mathematically, the three statistics displayed on a Taylor diagram are related by the following formula:(8)E′2=σr2+σt2−2σrσtρ,where E′ = centered RMS difference of observation and the prediction, ρ = correlation coefficient, and σr,σt = variances of the observation and the prediction.


## 4. Results and Discussion

### 4.1. Performance of the Raw GCMs

The ensemble averaged hindcast skill of seven models for the JJAS season over Myanmar for the period 1982 to 2011 is initially diagnosed based on their RMSE and correlation coefficient as shown in Figure5. It is seen that all the GCMs exhibit large error for simulation of rainfall with relatively less correlation with the observation. CFSv2 (0.39) and ECMWF (0.25) show better correlation with lesser errors 7.17 and 4.44, respectively. ECHAM4.5 models, both constructed analogue SST and CFS-forecasted SST, depicted larger RMS errors, similar to the findings of Singh et al. [52] for the Indian summer monsoon prediction. CCMv3.6 has better inverse correlation (−0.3) but with a very large RMS error (10.3). It is evident that none of the models can be utilized directly for the seasonal prediction and requires appropriate error correction and downscaling method to improve the performance of these models over Myanmar.Figure 5
JJAS performance comparison of the raw models with the bias-corrected (BC) models for the overall Myanmar. (a) Raw models. (b) Bias-corrected models. (c) MMEs.


### 4.2. Bias-Corrected Model and MME Performance over Myanmar

The bias-corrected results for the seven models over Myanmar shows reasonable improvement in RMS error and better agreement with the observation (Figure5(b)), especially ECHAM4.5 models which improved from −0.63 to 0.35 (CASST) and −0.67 to 0.35 (CFSSST) and with RMS error reduced from 14.01 to 6.8 for both CASST and CFSSST. ECMWF and CFSv2 have improved correlation from 0.25 to 0.46 and 0.39 to 0.50, respectively, with no significant improvement to the RMS error. At the same time, CCMv3.6, GFDL, and COLA exhibited negative impact of the bias corrections and degraded further with increase in RMS error. Though visible improvement in specific model performances over the country is noticed, this is still not adequate to operationally use them, as none of the models are consistent.Figure5(c) and Table 2 show the results of the three MME techniques for Myanmar, which indicates significant improvement with the correlation coefficient going as high as 0.64 for both WA-MME (MME2) and PCR method while the AM-MME (MME1) was slightly less with 0.5. At the same time, the RMS error reduced to 1.39 for MME1 and 1.29 for MME2 and PCR, respectively. The MMEs performing well over Myanmar provides the impetus to generate the climate information for the different climate zones and examine its performance.Table 2
Correlation coefficients, root mean square error, and standard deviation for the JJAS season for the six identified zones.
Methods/zones
AM-MME
WA-MME
PCR-MME
CC
SD
RMSE
CC
SD
RMSE
CC
SD
RMSE
East
0.32
0.53
0.69
0.36
0.66
0.75
−0.15
0.23
0.73
North
−0.03
0.92
1.79
0.11
0.87
1.66
0.11
0.66
1.58
Dry
0.02
0.44
0.75
0.46
0.5
0.59
0.44
0.35
0.57
Coastal
0.13
2
2.94
0.35
1.81
2.49
0.15
1.39
2.63
South
0.48
2.8
3.21
0.57
3.65
3.24
0.56
1.58
2.9
Delta
0.53
1.65
1.76
0.64
2.02
1.68
0.68
1.14
1.48


### 4.3. MME Performance over Climate Zones

#### 4.3.1. Quantifying the Observation and Model Variability

Figure6 shows the variability of the observed rainfall, individual model outputs that are bias corrected over the six climate zones. In general, the individual models are not able to capture the variability in the observation, whereas the MMEs captured the variability better than the individual models. Few models such as ECMWF and CFSv2 perform better in shan region and dry zones (Figures 6(a) and 6(c)), as the rainfall variability in the region itself is minimum when compared to the coastal, mountain, and southern regions (Figures 6(b), 6(d), and 6(e)). The way coupled models are designed and parameterized, the performance varies from region to region and from season to season. For instance, the predictability of CFSv2 and GFDL models over Indian region during JJAS months is much better when compared to other models such as ECMWF and CFSSST. Though the predictability skills of ECMWF are lower for the JJAS season, it performs well over the Indian region during the winter season [53]. In this study, CFSv2 performs well over the shan region and dry zones, but GFDL predictability skills are low. Further investigation on MME schemes over the study region indicated that the AM-MME scheme is not able to enhance the overall skill of the forecast mainly because an ensemble member with higher skill gets the same weight as a member with lower skill [16]. However, the WA-MME method performs better as weights were calculated and assigned to each ensemble member. The climatology for the same is shown in Figure 7.Figure 6
JJAS rainfall variability in observed (Obs, observed) and various model data (M1-AM-MME, M2-WA-MME, M3-PCRMME, M4-CCMv3.6, M5-ECHAM-CASST, M6-ECHAM-CFSSST, M7-CFSv2, M8-COLA, M9-GFDL, M10-ECMWF) for six zones of Myanmar: (a) shan, (b) north, (c) coastal, (d) dry, (e) south, and (f) delta.
(a)
(b)
(c)
(d)
(e)
(f)Figure 7
Observed and modeled rainfall during June to September period over the six climatological zones in Myanmar.


### 4.4. Correlation Coefficients and RMSE

Taylor diagrams were plotted for the different climate zones to quantify the regionwise skill of the MME methods as shown in Figure8. The results suggest that the WA-MME and PCR models show enhanced skill over the delta, coastal, and dry zones, while no significant improvement is observed over the eastern and northern zones. The AM-MME scheme performed better over the coastal and delta regions, most likely because the individual ensembles agree with each other when compared to regions, where the individual ensembles are not in agreement and the AM-MME performance is poor. Overall, all three MME schemes perform better over delta region meaning they depict the mean rainfall reasonably well. The observed temporal variability for the delta (2.1), coastal (2.4), and southern (3.6) regions is the highest, while for dry (0.6), north (1.5), and east (0.7) regions, variability is the lowest. Among all the models and methods, WA-MME scheme (Figure 8) captured the observed variation well except the northern zone.Figure 8
Correlation coefficient, root mean square error, and standard deviation for the JJAS period for all six climate zones: (a) delta zone; (b) southern zone; (c) coastal zone; (d) dry zone; (e) northern zone; (f) eastern shan zone in Myanmar. Reference point denotes the standard deviation for observation for each zone, respectively.
(a)
(b)
(c)
(d)
(e)
(f)


### 4.5. Measuring the Probabilistic Forecast Skill

The ROC scores shown in Table3 suggest that probabilistic forecast generated with the WA-MME scheme showed better skills among all three tercile categories: below normal (0.78), normal (0.83), and above normal (0.83) for overall Myanmar. In general, all three schemes were able to predict the above normal rainfall category very well, but the predictability skills for the “near normal” rainfall category is poor, especially for AM-MME and PCR-MME. Table 3 shows the ROC scores of the climate zones and suggests that the models are most skillful over the delta region followed by the southern and coastal regions, though it is satisfactory over the dry zone with PCR-MME performing better. However, the skills are very low for the eastern and northern regime, when compared to other zones. The reason for poor skill over the northern mountainous region or the eastern shan state could be mainly due to unavailability of good quality and sufficient number of observation points, which makes it difficult to define the predictand well for these regions as Kar et al. [47] described similar results over Indian monsoon prediction that the prediction skill is improved when a higher quality training dataset is deployed for the evaluation of the multimodel bias statistics [47]. On the other hand, it could also be due to failure of the global models to capture the rainfall variability over the high-elevation region over Myanmar which spreads over the northern to eastern zones. It is important to notice that the MME methods are skillful in predicting the lower (below normal) and upper (above normal) tercile categories better than the normal category which is a positive sign as often above and below normal rainfall categories are crucial to be known for carrying out seasonal preparedness measures, rather than the normal rainfall category.Table 3
ROC scores for three tercile categories over the six identified climate zones for the three MME schemes.
Tercile/regions
MMEs
Shan
North
Dry
Coastal
South
Delta
Myanmar
Below normal
AM
0.6
0.4
0.55
0.48
0.63
0.63
0.78
WA
0.55
0.55
0.6
0.63
0.7
0.63
0.78
PCR
0.7
0.63
0.6
0.55
0.78
0.78
0.75
Normal
AM
0.4
0.33
0.55
0.4
0.48
0.55
0.58
WA
0.48
0.48
0.55
0.63
0.6
0.4
0.83
PCR
0.63
0.4
0.6
0.5
0.63
0.63
0.55
Above normal
AM
0.52
0.33
0.45
0.55
0.63
0.7
0.8
WA
0.55
0.48
0.7
0.7
0.6
0.7
0.83
PCR
0.48
0.4
0.63
0.55
0.7
0.85
0.8


## 4.1. Performance of the Raw GCMs

The ensemble averaged hindcast skill of seven models for the JJAS season over Myanmar for the period 1982 to 2011 is initially diagnosed based on their RMSE and correlation coefficient as shown in Figure5. It is seen that all the GCMs exhibit large error for simulation of rainfall with relatively less correlation with the observation. CFSv2 (0.39) and ECMWF (0.25) show better correlation with lesser errors 7.17 and 4.44, respectively. ECHAM4.5 models, both constructed analogue SST and CFS-forecasted SST, depicted larger RMS errors, similar to the findings of Singh et al. [52] for the Indian summer monsoon prediction. CCMv3.6 has better inverse correlation (−0.3) but with a very large RMS error (10.3). It is evident that none of the models can be utilized directly for the seasonal prediction and requires appropriate error correction and downscaling method to improve the performance of these models over Myanmar.Figure 5
JJAS performance comparison of the raw models with the bias-corrected (BC) models for the overall Myanmar. (a) Raw models. (b) Bias-corrected models. (c) MMEs.


## 4.2. Bias-Corrected Model and MME Performance over Myanmar

The bias-corrected results for the seven models over Myanmar shows reasonable improvement in RMS error and better agreement with the observation (Figure5(b)), especially ECHAM4.5 models which improved from −0.63 to 0.35 (CASST) and −0.67 to 0.35 (CFSSST) and with RMS error reduced from 14.01 to 6.8 for both CASST and CFSSST. ECMWF and CFSv2 have improved correlation from 0.25 to 0.46 and 0.39 to 0.50, respectively, with no significant improvement to the RMS error. At the same time, CCMv3.6, GFDL, and COLA exhibited negative impact of the bias corrections and degraded further with increase in RMS error. Though visible improvement in specific model performances over the country is noticed, this is still not adequate to operationally use them, as none of the models are consistent.Figure5(c) and Table 2 show the results of the three MME techniques for Myanmar, which indicates significant improvement with the correlation coefficient going as high as 0.64 for both WA-MME (MME2) and PCR method while the AM-MME (MME1) was slightly less with 0.5. At the same time, the RMS error reduced to 1.39 for MME1 and 1.29 for MME2 and PCR, respectively. The MMEs performing well over Myanmar provides the impetus to generate the climate information for the different climate zones and examine its performance.Table 2
Correlation coefficients, root mean square error, and standard deviation for the JJAS season for the six identified zones.
Methods/zones
AM-MME
WA-MME
PCR-MME
CC
SD
RMSE
CC
SD
RMSE
CC
SD
RMSE
East
0.32
0.53
0.69
0.36
0.66
0.75
−0.15
0.23
0.73
North
−0.03
0.92
1.79
0.11
0.87
1.66
0.11
0.66
1.58
Dry
0.02
0.44
0.75
0.46
0.5
0.59
0.44
0.35
0.57
Coastal
0.13
2
2.94
0.35
1.81
2.49
0.15
1.39
2.63
South
0.48
2.8
3.21
0.57
3.65
3.24
0.56
1.58
2.9
Delta
0.53
1.65
1.76
0.64
2.02
1.68
0.68
1.14
1.48


## 4.3. MME Performance over Climate Zones

### 4.3.1. Quantifying the Observation and Model Variability

Figure6 shows the variability of the observed rainfall, individual model outputs that are bias corrected over the six climate zones. In general, the individual models are not able to capture the variability in the observation, whereas the MMEs captured the variability better than the individual models. Few models such as ECMWF and CFSv2 perform better in shan region and dry zones (Figures 6(a) and 6(c)), as the rainfall variability in the region itself is minimum when compared to the coastal, mountain, and southern regions (Figures 6(b), 6(d), and 6(e)). The way coupled models are designed and parameterized, the performance varies from region to region and from season to season. For instance, the predictability of CFSv2 and GFDL models over Indian region during JJAS months is much better when compared to other models such as ECMWF and CFSSST. Though the predictability skills of ECMWF are lower for the JJAS season, it performs well over the Indian region during the winter season [53]. In this study, CFSv2 performs well over the shan region and dry zones, but GFDL predictability skills are low. Further investigation on MME schemes over the study region indicated that the AM-MME scheme is not able to enhance the overall skill of the forecast mainly because an ensemble member with higher skill gets the same weight as a member with lower skill [16]. However, the WA-MME method performs better as weights were calculated and assigned to each ensemble member. The climatology for the same is shown in Figure 7.Figure 6
JJAS rainfall variability in observed (Obs, observed) and various model data (M1-AM-MME, M2-WA-MME, M3-PCRMME, M4-CCMv3.6, M5-ECHAM-CASST, M6-ECHAM-CFSSST, M7-CFSv2, M8-COLA, M9-GFDL, M10-ECMWF) for six zones of Myanmar: (a) shan, (b) north, (c) coastal, (d) dry, (e) south, and (f) delta.
(a)
(b)
(c)
(d)
(e)
(f)Figure 7
Observed and modeled rainfall during June to September period over the six climatological zones in Myanmar.


## 4.3.1. Quantifying the Observation and Model Variability

Figure6 shows the variability of the observed rainfall, individual model outputs that are bias corrected over the six climate zones. In general, the individual models are not able to capture the variability in the observation, whereas the MMEs captured the variability better than the individual models. Few models such as ECMWF and CFSv2 perform better in shan region and dry zones (Figures 6(a) and 6(c)), as the rainfall variability in the region itself is minimum when compared to the coastal, mountain, and southern regions (Figures 6(b), 6(d), and 6(e)). The way coupled models are designed and parameterized, the performance varies from region to region and from season to season. For instance, the predictability of CFSv2 and GFDL models over Indian region during JJAS months is much better when compared to other models such as ECMWF and CFSSST. Though the predictability skills of ECMWF are lower for the JJAS season, it performs well over the Indian region during the winter season [53]. In this study, CFSv2 performs well over the shan region and dry zones, but GFDL predictability skills are low. Further investigation on MME schemes over the study region indicated that the AM-MME scheme is not able to enhance the overall skill of the forecast mainly because an ensemble member with higher skill gets the same weight as a member with lower skill [16]. However, the WA-MME method performs better as weights were calculated and assigned to each ensemble member. The climatology for the same is shown in Figure 7.Figure 6
JJAS rainfall variability in observed (Obs, observed) and various model data (M1-AM-MME, M2-WA-MME, M3-PCRMME, M4-CCMv3.6, M5-ECHAM-CASST, M6-ECHAM-CFSSST, M7-CFSv2, M8-COLA, M9-GFDL, M10-ECMWF) for six zones of Myanmar: (a) shan, (b) north, (c) coastal, (d) dry, (e) south, and (f) delta.
(a)
(b)
(c)
(d)
(e)
(f)Figure 7
Observed and modeled rainfall during June to September period over the six climatological zones in Myanmar.


## 4.4. Correlation Coefficients and RMSE

Taylor diagrams were plotted for the different climate zones to quantify the regionwise skill of the MME methods as shown in Figure8. The results suggest that the WA-MME and PCR models show enhanced skill over the delta, coastal, and dry zones, while no significant improvement is observed over the eastern and northern zones. The AM-MME scheme performed better over the coastal and delta regions, most likely because the individual ensembles agree with each other when compared to regions, where the individual ensembles are not in agreement and the AM-MME performance is poor. Overall, all three MME schemes perform better over delta region meaning they depict the mean rainfall reasonably well. The observed temporal variability for the delta (2.1), coastal (2.4), and southern (3.6) regions is the highest, while for dry (0.6), north (1.5), and east (0.7) regions, variability is the lowest. Among all the models and methods, WA-MME scheme (Figure 8) captured the observed variation well except the northern zone.Figure 8
Correlation coefficient, root mean square error, and standard deviation for the JJAS period for all six climate zones: (a) delta zone; (b) southern zone; (c) coastal zone; (d) dry zone; (e) northern zone; (f) eastern shan zone in Myanmar. Reference point denotes the standard deviation for observation for each zone, respectively.
(a)
(b)
(c)
(d)
(e)
(f)


## 4.5. Measuring the Probabilistic Forecast Skill

The ROC scores shown in Table3 suggest that probabilistic forecast generated with the WA-MME scheme showed better skills among all three tercile categories: below normal (0.78), normal (0.83), and above normal (0.83) for overall Myanmar. In general, all three schemes were able to predict the above normal rainfall category very well, but the predictability skills for the “near normal” rainfall category is poor, especially for AM-MME and PCR-MME. Table 3 shows the ROC scores of the climate zones and suggests that the models are most skillful over the delta region followed by the southern and coastal regions, though it is satisfactory over the dry zone with PCR-MME performing better. However, the skills are very low for the eastern and northern regime, when compared to other zones. The reason for poor skill over the northern mountainous region or the eastern shan state could be mainly due to unavailability of good quality and sufficient number of observation points, which makes it difficult to define the predictand well for these regions as Kar et al. [47] described similar results over Indian monsoon prediction that the prediction skill is improved when a higher quality training dataset is deployed for the evaluation of the multimodel bias statistics [47]. On the other hand, it could also be due to failure of the global models to capture the rainfall variability over the high-elevation region over Myanmar which spreads over the northern to eastern zones. It is important to notice that the MME methods are skillful in predicting the lower (below normal) and upper (above normal) tercile categories better than the normal category which is a positive sign as often above and below normal rainfall categories are crucial to be known for carrying out seasonal preparedness measures, rather than the normal rainfall category.Table 3
ROC scores for three tercile categories over the six identified climate zones for the three MME schemes.
Tercile/regions
MMEs
Shan
North
Dry
Coastal
South
Delta
Myanmar
Below normal
AM
0.6
0.4
0.55
0.48
0.63
0.63
0.78
WA
0.55
0.55
0.6
0.63
0.7
0.63
0.78
PCR
0.7
0.63
0.6
0.55
0.78
0.78
0.75
Normal
AM
0.4
0.33
0.55
0.4
0.48
0.55
0.58
WA
0.48
0.48
0.55
0.63
0.6
0.4
0.83
PCR
0.63
0.4
0.6
0.5
0.63
0.63
0.55
Above normal
AM
0.52
0.33
0.45
0.55
0.63
0.7
0.8
WA
0.55
0.48
0.7
0.7
0.6
0.7
0.83
PCR
0.48
0.4
0.63
0.55
0.7
0.85
0.8


## 5. Conclusion

Agricultural system is predominantly dependent on skillful weather forecast with a longer lead time, preferably at seasonal scale. Critical decision making entails higher risks in the absence of such forecast systems. Thus, the forecast customization system (FOCUS) was developed to address this issue and it provides an enabling environment to the meteorological service in Myanmar with a standardized platform to access and evaluate various global models with a streamlined approach. The tool is developed using free and open-source scripting language, Python, and Microsoft’s .net framework. Three standard MME methods were developed and integrated into the FOCUS platform with components to interpolate and combine global model hindcast data with forecast. The MME-based forecast was then generated for the defined climate zones for the JJAS period.To quantify uncertainty, the MME outputs were evaluated for (i) accuracy with standard verification methods using RMSE and correlation coefficient and (ii) the predictability skill with ROC scores. The results suggested that, by utilizing the MME methods, the performance of forecast was significantly improved over the country and over the JJAS period, in terms of predictability skill. Among the MMEs, the weighted ensemble averaging method (ROC = 0.83) has slight advantage over the simple arithmetic averaging method (ROC = 0.58) in terms of predictability skills for the normal tercile category. The principal component regression method is performing well over the high-rainfall southern (ROC = 0.7) and delta regions (ROC = 0.85) for prediction of the upper terciles as well as for the lower terciles with ROC = 0.78 (southern region) and ROC = 0.78 (delta region). Overall, it is evident that MME performance is satisfactory, and especially both WA-MME and PCR-MME could be considered, with high reliability, for generating seasonal forecast for the high rainfall zones in the country. Again, it is worth noticing that the model is highly reliable for predictions of upper and lower terciles but failed to accurately predict the normal rainfall category.FOCUS tool uses well-defined methods and has the potential to be scaled up further, for other countries in the region, with use of more advanced statistical and computational techniques. However, it is necessary for the tool to have high-quality rainfall observation datasets with adequate spatial and temporal coverage. In conclusion, the MME-based approach incorporated in a user-friendly interface would be a very useful tool for generating skillful seasonal forecast for the tropical region. Again, an improved seasonal forecast enables effective decision making in all climate-sensitive sectors such as the agriculture and water resources.



---

*Source: 4957127-2019-12-18.xml*