# Evaluation of Coupled Model Forecasts of Ethiopian Highlands Summer Climate

Authors: Mark R. Jury
Journal: Advances in Meteorology
 (2014)

---

## Abstract

This study evaluates seasonal forecasts of rainfall and maximum temperature across the Ethiopian highlands from coupled ensemble models in the period 1981–2006, by comparison with gridded observational products (NMA + GPCC/CRU3). Early season forecasts from the coupled forecast system (CFS) are steadier than European community medium range forecast (ECMWF). CFS and ECMWF April forecasts of June–August (JJA) rainfall achieve significant fit (r
2
=
0.27, 0.25, resp.), but ECMWF forecasts tend to have a narrow range with drought underpredicted. Early season forecasts of JJA maximum temperature are weak in both models; hence ability to predict water resource gains may be better than losses. One aim of seasonal climate forecasting is to ensure that crop yields keep pace with Ethiopia’s growing population. Farmers using prediction technology are better informed to avoid risk in dry years and generate surplus in wet years.


---

## Body

## 1. Introduction

Agricultural production is typically planned around a range of climatic conditions that take into account the possibility of flood or drought every decade. Commercial farmers have access to technology and finance, while subsistence farmers get by on local resources [1–4]. According to current FAO statistics, 76% of Ethiopia’s 88 million people are engaged in farming on 15% of the land. There are orographic rains and a rich vegetation cover (Figure 1(a)), but a population density > 100 people/km2 and crop yields <2 T/ha put a strain food supplies. Drought, for example in 1992-1993 and 2002-2003, caused malnutrition that required state assistance to 7–10 million people [5].Figure 1
(a) Ethiopian crop cultivation areas (shaded green) and (b) station/ship and (c) buoy/drifter observations. Capital city (dot) and Lake Tana shown in (a); place names and box for model evaluation in (c).The climate observing network across northeast Africa, although adequate (Figure1(b)), is declining from civil instability and financial constraints. While satellites supplement the atmospheric observations, seasonal forecast models need measurements in the upper ocean. These have been enhanced through the Global Ocean Observing System [6, 7] by extension of buoys and drifters near Africa ([8], Figure 1(c)). New surface flux data for coupled modelling have become available via satellite estimation of transpiration, soil water, and marine winds.The predictability of climate is partly attributable to the Pacific El Niño Southern Oscillation (ENSO) and its overlying zonal circulation [9]. During ENSO warm phase, convection over most of Africa is suppressed [10, 11] and warm water spreads westward across the Indian Ocean [12–14]. The Atlantic Ocean has a meridional sea temperature dipole that affects West African convection [15]. The three tropical ocean basins each have distinct rhythms that may interfere or reinforce each other [16]. In addition to large scale ocean-atmosphere coupling, there are sources of local forcing that can alter the incoming climate signals. These derive from rapid feedback between land surface fluxes and the lower atmosphere [17]. Decadal rainfall cycles have been found in Ethiopia [18] driven by slow eastward moving air pressure waves [16] that afford opportunities for seasonal forecasts even during ENSO transitions. The Ethiopian National Meteorological Agency (NMA) and the regional Climate Outlook Forum (GHACOF) have made seasonal forecasts since the 1990s using statistical methods [19]. These can be aimed at integrating targets such as crop yield [20] or river flow but have the disadvantage of assuming historical replication. More recently coupled ensemble models have been tested and used in seasonal forecasts [21].In addition to the zonal circulation, the Hadley cell induces a large annual cycle over Ethiopia: soil moisture is depleted after January and replenished after June, when the equatorial trough reaches its northernmost limit. Given that highland crops are planted in June and harvested in October, users need seasonal forecasts during the warm spring season (March-April). This study evaluates coupled ensemble model forecasts (as in [22, 23]), considers factors driving Ethiopian climate fluctuations, and discusses potential applications.


## 2. Data and Methods

Retrospective forecasts from coupled ensemble models are compared with June to August (JJA) observations in the period 1981–2006 using data from the Climate Explorer website (http://climexp.knmi.nl). The models include CFSv2 (Coupled Forecast System, [24]) and ECMWFv3 (European Community Medium-range Weather Forecast, [25, 26]). Three observed products serve as reference: NMA interpolated rain gauge [27], GPCCv5 rainfall (Global Precipitation Climatology Center, [28, 29]), and CRUv3 (Climate Research Unit, [30]) maximum temperature, based on monthly 50 km gridded station data. Supplementary observations include low-resolution satellite-interpolated rainfall GPCPv2 (Global Precipitation Climatology Project, [31, 32]) and high-resolution reanalyses from CFS [33] and ECMWF [34]. Intercomparison of highlands area-average (7–14N, 36–40E) NMA interpolated rainfall yields GPCC r
=
0.92, GPCP 0.92, CRU3 0.85, CFS 0.71, and ECMWF 0.54, for continuous monthly data in the period 1981–2006. To form the observed highlands JJA rainfall dataset, NMA, and GPCC anomalies were averaged.To evaluate the coupled ensemble models, correlations between forecast and observed JJA seasonal rainfall(
R
) and maximum temperature (Tx) anomalies were analyzed at various lead times from January to June and for the first and second half of sample (breakpoint 1993). Correlation maps were calculated to determine the spatial patterns of model performance, while temporal trends were evaluated using averages over the highlands 7–14N and 36–40E. This target is big enough to capture climate signals at long lead time yet falls within a singular climate regime [35]. In the seasonal model evaluation, statistical significance (at 90% confidence) is reached with r
>
0.32 for 26 degrees of freedom. Scatterplots of forecast and observed JJA anomalies were evaluated for slope, range, and outliers. Using anomalies helps to offset the mean bias and is consistent with operational forecasts of departures. Global signals driving local climate fluctuations were studied by correlation of NCEP [36] zonal winds, temperature, humidity, and vertical motion in a north-south vertical slice over the highlands. Satellite vegetation (NDVI) data [37] were analyzed for standard deviation, as a measure of agricultural vulnerability to annual and year-to-year fluctuations.


## 3. Results

### 3.1. JJA Pattern and Annual Cycle

The 1981–2006 climatology of rainfall and maximum temperature as simulated and observed are illustrated in Figures2(a) and 2(b). Both the CFS and the ECMWF models exhibit a cool wet bias compared with CRU3 and GPCC/GPCP observations in JJA season (e.g., Tx is ~2°C below observed, R
~
2 mm/day above). The model outputs could be “real” given that most observations are taken at urban airports located in warm dry valleys. The ECMWF JJA rain pattern is shifted too far east, suggesting that orographic uplift on the escarpment at 40E is overplayed. The CFS JJA maximum temperature pattern is well-located but cool. The annual cycle of model outputs follows the R and Tx observations (Figures 2(a) and 2(b)). ECMWF has too much rain in JJA (consistent with [23]), while the CFS reflects an earlier onset of rains that conforms to the observed seasonal shape. Both models follow the annual cycle of maximum temperature with a cool offset. ECMWF and CFS Tx climatologies match in the rainy season, but ECMWF is warmer in the dry season and close to CRU3 observations then.(a) JJA rainfall climatology from ECMWF (left) and GPCC observations (middle) with stations (dots) and annual cycle (right) comparing ECMWF/CFS models and GPCC/GPCP observations. (b) JJA maximum temperature climatology from CFS (left) and CRU3 observations (middle) with 1000 m/2000 m elevation contours and annual cycle (right) comparing ECMWF/CFS models and CRU3 observations.
(a)
(b)


### 3.2. Spatial Performance

The correlation maps of CFS and ECMWF model outputs with respect to GPCC and CRU3 observations in the period 1981–2006 are given in Figure3 for March and April forecasts. For CFS rainfall, the forecasts reach statistical significance in the northeast and southern highlands but not in the east and west escarpments. For ECMWF the rain forecasts are only significant in the northeast and actually negative in the southwest highlands particularly in March. Maximum temperature forecasts are weaker than rainfall in both models. The CFS obtains significant values in the central highlands in March which fade out in April, while ECMWF Tx forecasts are only significant in the southern highlands and weak or negative in the north. Forecasts are better for R than Tx, suggesting that model resolution (~1°) is not an issue.Figure 3
Spatial maps of correlation between CFS and ECMWF model outputs (columns) 1981–2006 and GPCC rain/CRU3 temp observations at two lead times (rows) labelled by forecast month, for JJA rainfall (left) and maximum temperature (right). A few place names are given for reference. The first shaded level is insignificant.


### 3.3. Temporal Performance

Bar charts of highland area-averaged correlations between predicted and observedR and Tx for forecasts issued from January to June are given in Figure 4. CFS JJA rainfall forecasts are modest and comparable to ECMWF (Figure 4(a)) with r
>
0.40 after April, consistent with Ndiaye et al. [21] for the Sahel. ECMWF rainfall forecasts (Figure 4(b)) dip in March and rise thereafter. CFS area-averaged JJA maximum temperature forecasts are weak and rise in March (Figure 4(d)), while ECMWF Tx forecasts start well in January and slump from March to April (Figure 4(e)). Forecasts show slight improvement over time (Figure 4(c)) mainly for CFS maximum temperature, possibly owing to improved satellite and ocean measurements (model technology is fixed). The steadiness of early season forecasts is assessed in Figure 4(f), wherein it is seen that January forecasts remain steady for CFS outputs of R and Tx. However ECMWF January forecasts slump in February and suggest better value after March. It is thought that the seasonal weakening of Pacific ENSO [38] and ambiguous coupling with Atlantic and Indian Oceans [39] are the cause of instability.Bar charts summarizing model correlation with highlands area-averaged NMA-GPCC JJA rainfall (top) and CRU3 JJA maximum temperature (lower) for CFS ((a) and (d)) and ECMWF ((b) and (e)). (c) Model correlation in first and second half of sample. (f) Comparison of forecast persistence from January. Correlation values above 0.32 are significant at 90% confidence.
(a)
(b)
(c)
(d)
(e)
(f)Scatterplots of area-averaged CFS March and ECMWF April forecasts versus observed JJA seasonal anomalies are illustrated in Figures5(a)–5(d). It is evident that ECMWF forecasts have a narrower range for both R and Tx and thus tend toward the mean more than CFS. ECMWF rainfall forecasts exhibit a suitable 1 : 1 slope with the 1984 drought as an outlier. CFS rainfall forecasts are well distributed and show highest fit (27%), but the flat 0.29 slope indicates over-prediction. Surprizingly, Tx forecasts are too dispersed in both models and consequently have insignificant fit (10%). Considering the outliers: CFS forecast Tx is too warm in 1986 while ECMWF forecast Tx is too warm in 1994. Both models under-predict 2002, anticipating neutral conditions instead of drought. The scatterplot equations-of-fit suggest potential adjustments to operational model outputs.Scatterplots of highlands area-averaged model forecasts compared with JJA observations: (a) March CFS rain, (b) April ECMWF rain, (c) March CFS maximum temperature, and (d) April ECMWF maximum temperature. While rainfall forecasts are significant, temperature forecasts fall below.
(a)
(b)
(c)
(d)


### 3.4. Climate Signals

Skillful forecasts depend on model ability to simulate ocean-atmosphere coupling and transmit global circulation anomalies to northeast Africa [9, 40–43]. For model simulations in the 1981–2006 period, inter-annual forcing deserves attention. Quasi-Biennial Oscillation (QBO, 30 mb tropical zonal wind) and ENSO (Pacific SST EOF1) indices are correlated with key variables in a north-south slice over the study area (Figures 6(a) and 6(b)). Both exhibit quite similar patterns. In west phase QBO and warm phase ENSO, easterly winds (
r
<
0
) accelerate below 500 mb, driving away Congo moisture. Sinking motions warm the lower atmosphere, while westerly winds above 300 mb (
r
>
0
) shear the convection and inhibit Indian monsoon outflow. Given the adequate performance by CFS and ECMWF models noted above, it is likely that these signals are initialized and transmitted. Yet forecast skill is marginalized by confounding influences from the Atlantic and Indian Oceans, and an opposing ENSO response in southern Ethiopia where the equatorial trough shifts rapidly in spring [44]. It is beyond the scope of this paper to evaluate model diagnostics.Correlation of JJA zonal winds with (a) QBO and (b) ENSO indices, averaged over the highlands represented as a north-south slice (longitudes 35–40E). Secondary correlations with temperature, humidity, and vertical motion are labeled at the peak value.z-axis is the pressure in mb, land surface illustrated with country labels, and values < 0.3 masked.
(a)
(b)


### 3.5. Vulnerability

Considering the amplitude of vegetation (NDVI) response to climate impacts in the period 1981–2006, the standard deviation is calculated on monthly fields and departures (Figures7(a) and 7(b)). The former identifies annual range, the latter inter-annual fluctuations. Annual range is greatest near the Sudan border west of Lake Tana, where maximum temperatures exceed 35 C (cf. Figure 2(b)). Annual range is low next to the large lakes and in the eastern lowlands where it is always warm. Interannual fluctuations are greatest in the southern highlands and along the eastern escarpment on 40E. Year-to-year changes of NDVI are low across the northern highlands (Tigray, Amhara). Hence vulnerability to climate in the southern highlands, eastern escarpment and western lowlands, makes the uptake of forecasts there critical to food security.Standard deviation of (a) annual satellite NDVI and (b) interannual NDVI anomalies, showing areas vulnerable to fluctuations (brown) in the period 1981–2006. States borders/labels given.
(a)
(b)


### 3.6. Application

The Ethiopian Institute for Agriculture Research (EIAR) uses seasonal forecasts from the NMA/GHACOF and CFS/ECMWF modeling centers to develop an initial outlook for the season that guides farmers on how much area is planted and which hybrid seeds are used. The initial outlook is followed with bimonthly updates and advisories to optimize farming activities. The commercial sector carries bigger risks and is more responsive to technological inputs than the subsistence sector. As the season progresses, NDVI anomalies in cropped areas are analyzed (http://pekko.geog.umd.edu/usda/test/, cf. Figure 1(a)) and the EIAR obtains direct feedback from farm liaison officers. Agroclimate information networks are utilized to help the rural population avoid risks in dry years and secure resources in wet years. Interventions are made at planting time, in the event of flood or drought and to collect harvest data. At the EIAR experimental farm in Melkassa, staff monitor crops and develop ways to improve yields. This is critical, because an upward trend of ~0.1 T ha−1/yr is needed to keep pace with Ethiopia’s growing population.


## 3.1. JJA Pattern and Annual Cycle

The 1981–2006 climatology of rainfall and maximum temperature as simulated and observed are illustrated in Figures2(a) and 2(b). Both the CFS and the ECMWF models exhibit a cool wet bias compared with CRU3 and GPCC/GPCP observations in JJA season (e.g., Tx is ~2°C below observed, R
~
2 mm/day above). The model outputs could be “real” given that most observations are taken at urban airports located in warm dry valleys. The ECMWF JJA rain pattern is shifted too far east, suggesting that orographic uplift on the escarpment at 40E is overplayed. The CFS JJA maximum temperature pattern is well-located but cool. The annual cycle of model outputs follows the R and Tx observations (Figures 2(a) and 2(b)). ECMWF has too much rain in JJA (consistent with [23]), while the CFS reflects an earlier onset of rains that conforms to the observed seasonal shape. Both models follow the annual cycle of maximum temperature with a cool offset. ECMWF and CFS Tx climatologies match in the rainy season, but ECMWF is warmer in the dry season and close to CRU3 observations then.(a) JJA rainfall climatology from ECMWF (left) and GPCC observations (middle) with stations (dots) and annual cycle (right) comparing ECMWF/CFS models and GPCC/GPCP observations. (b) JJA maximum temperature climatology from CFS (left) and CRU3 observations (middle) with 1000 m/2000 m elevation contours and annual cycle (right) comparing ECMWF/CFS models and CRU3 observations.
(a)
(b)


## 3.2. Spatial Performance

The correlation maps of CFS and ECMWF model outputs with respect to GPCC and CRU3 observations in the period 1981–2006 are given in Figure3 for March and April forecasts. For CFS rainfall, the forecasts reach statistical significance in the northeast and southern highlands but not in the east and west escarpments. For ECMWF the rain forecasts are only significant in the northeast and actually negative in the southwest highlands particularly in March. Maximum temperature forecasts are weaker than rainfall in both models. The CFS obtains significant values in the central highlands in March which fade out in April, while ECMWF Tx forecasts are only significant in the southern highlands and weak or negative in the north. Forecasts are better for R than Tx, suggesting that model resolution (~1°) is not an issue.Figure 3
Spatial maps of correlation between CFS and ECMWF model outputs (columns) 1981–2006 and GPCC rain/CRU3 temp observations at two lead times (rows) labelled by forecast month, for JJA rainfall (left) and maximum temperature (right). A few place names are given for reference. The first shaded level is insignificant.


## 3.3. Temporal Performance

Bar charts of highland area-averaged correlations between predicted and observedR and Tx for forecasts issued from January to June are given in Figure 4. CFS JJA rainfall forecasts are modest and comparable to ECMWF (Figure 4(a)) with r
>
0.40 after April, consistent with Ndiaye et al. [21] for the Sahel. ECMWF rainfall forecasts (Figure 4(b)) dip in March and rise thereafter. CFS area-averaged JJA maximum temperature forecasts are weak and rise in March (Figure 4(d)), while ECMWF Tx forecasts start well in January and slump from March to April (Figure 4(e)). Forecasts show slight improvement over time (Figure 4(c)) mainly for CFS maximum temperature, possibly owing to improved satellite and ocean measurements (model technology is fixed). The steadiness of early season forecasts is assessed in Figure 4(f), wherein it is seen that January forecasts remain steady for CFS outputs of R and Tx. However ECMWF January forecasts slump in February and suggest better value after March. It is thought that the seasonal weakening of Pacific ENSO [38] and ambiguous coupling with Atlantic and Indian Oceans [39] are the cause of instability.Bar charts summarizing model correlation with highlands area-averaged NMA-GPCC JJA rainfall (top) and CRU3 JJA maximum temperature (lower) for CFS ((a) and (d)) and ECMWF ((b) and (e)). (c) Model correlation in first and second half of sample. (f) Comparison of forecast persistence from January. Correlation values above 0.32 are significant at 90% confidence.
(a)
(b)
(c)
(d)
(e)
(f)Scatterplots of area-averaged CFS March and ECMWF April forecasts versus observed JJA seasonal anomalies are illustrated in Figures5(a)–5(d). It is evident that ECMWF forecasts have a narrower range for both R and Tx and thus tend toward the mean more than CFS. ECMWF rainfall forecasts exhibit a suitable 1 : 1 slope with the 1984 drought as an outlier. CFS rainfall forecasts are well distributed and show highest fit (27%), but the flat 0.29 slope indicates over-prediction. Surprizingly, Tx forecasts are too dispersed in both models and consequently have insignificant fit (10%). Considering the outliers: CFS forecast Tx is too warm in 1986 while ECMWF forecast Tx is too warm in 1994. Both models under-predict 2002, anticipating neutral conditions instead of drought. The scatterplot equations-of-fit suggest potential adjustments to operational model outputs.Scatterplots of highlands area-averaged model forecasts compared with JJA observations: (a) March CFS rain, (b) April ECMWF rain, (c) March CFS maximum temperature, and (d) April ECMWF maximum temperature. While rainfall forecasts are significant, temperature forecasts fall below.
(a)
(b)
(c)
(d)


## 3.4. Climate Signals

Skillful forecasts depend on model ability to simulate ocean-atmosphere coupling and transmit global circulation anomalies to northeast Africa [9, 40–43]. For model simulations in the 1981–2006 period, inter-annual forcing deserves attention. Quasi-Biennial Oscillation (QBO, 30 mb tropical zonal wind) and ENSO (Pacific SST EOF1) indices are correlated with key variables in a north-south slice over the study area (Figures 6(a) and 6(b)). Both exhibit quite similar patterns. In west phase QBO and warm phase ENSO, easterly winds (
r
<
0
) accelerate below 500 mb, driving away Congo moisture. Sinking motions warm the lower atmosphere, while westerly winds above 300 mb (
r
>
0
) shear the convection and inhibit Indian monsoon outflow. Given the adequate performance by CFS and ECMWF models noted above, it is likely that these signals are initialized and transmitted. Yet forecast skill is marginalized by confounding influences from the Atlantic and Indian Oceans, and an opposing ENSO response in southern Ethiopia where the equatorial trough shifts rapidly in spring [44]. It is beyond the scope of this paper to evaluate model diagnostics.Correlation of JJA zonal winds with (a) QBO and (b) ENSO indices, averaged over the highlands represented as a north-south slice (longitudes 35–40E). Secondary correlations with temperature, humidity, and vertical motion are labeled at the peak value.z-axis is the pressure in mb, land surface illustrated with country labels, and values < 0.3 masked.
(a)
(b)


## 3.5. Vulnerability

Considering the amplitude of vegetation (NDVI) response to climate impacts in the period 1981–2006, the standard deviation is calculated on monthly fields and departures (Figures7(a) and 7(b)). The former identifies annual range, the latter inter-annual fluctuations. Annual range is greatest near the Sudan border west of Lake Tana, where maximum temperatures exceed 35 C (cf. Figure 2(b)). Annual range is low next to the large lakes and in the eastern lowlands where it is always warm. Interannual fluctuations are greatest in the southern highlands and along the eastern escarpment on 40E. Year-to-year changes of NDVI are low across the northern highlands (Tigray, Amhara). Hence vulnerability to climate in the southern highlands, eastern escarpment and western lowlands, makes the uptake of forecasts there critical to food security.Standard deviation of (a) annual satellite NDVI and (b) interannual NDVI anomalies, showing areas vulnerable to fluctuations (brown) in the period 1981–2006. States borders/labels given.
(a)
(b)


## 3.6. Application

The Ethiopian Institute for Agriculture Research (EIAR) uses seasonal forecasts from the NMA/GHACOF and CFS/ECMWF modeling centers to develop an initial outlook for the season that guides farmers on how much area is planted and which hybrid seeds are used. The initial outlook is followed with bimonthly updates and advisories to optimize farming activities. The commercial sector carries bigger risks and is more responsive to technological inputs than the subsistence sector. As the season progresses, NDVI anomalies in cropped areas are analyzed (http://pekko.geog.umd.edu/usda/test/, cf. Figure 1(a)) and the EIAR obtains direct feedback from farm liaison officers. Agroclimate information networks are utilized to help the rural population avoid risks in dry years and secure resources in wet years. Interventions are made at planting time, in the event of flood or drought and to collect harvest data. At the EIAR experimental farm in Melkassa, staff monitor crops and develop ways to improve yields. This is critical, because an upward trend of ~0.1 T ha−1/yr is needed to keep pace with Ethiopia’s growing population.


## 4. Summary

This study has evaluated summer rainfall and maximum temperature forecasts by ECMWFv3 and CFSv2 models (cf. [22, 23]) via spatial correlation maps and area-averaged temporal analyses. Reference data were comprised of gridded NMA + GPCC R and CRU3 Tx observations in the Ethiopian highlands 7–14N 36–40E. Both models simulate the JJA mean spatial pattern with a 10% cool wet bias, and their March-April forecasts correlate positively with summer observations from 1981 to 2006. Considering that a cost benefit is possible with model “fit” above half [45], such skill is reached for CFS and ECMWF April forecasts of JJA rainfall over most of the highlands (cf. Figures 3, 4(a), and 4(b)). Yet the limit of predictability is evident in Figure 5, 25–27% of variance for rainfall and 9-10% for maximum temperature. Further work is recommended to understand causes of instability in early season forecasts, determine why maximum temperature forecasts are weak, employ more robust reference data [46], and develop bias corrections for improved model skill. At the EIAR experimental farm, numerical and statistical forecasts will be compared and utilized to develop mitigating strategies that boost crop yields.



---

*Source: 894318-2014-10-14.xml*