metadata: authors: 0: author: Steinbuks,Jevgenijs
count: World
docna: 0: docna: Assessing the accuracy of electricity demand
            forecasts in developing countries
docty: Policy Research Working Paper
subtopic: Economic Theory & Research,Industrial Economics,Energy Policies & Economics,Economic Growth,Energy Demand,Energy and Environment,Energy and Mining,Global Environment
teratopic: Energy
abstracts: cdata!: This study assesses the accuracy of
            time-series econometric methods in forecasting electricity
            demand in developing countries. The analysis of historical
            time series for 106 developing countries over 1960-2012
            demonstrates that econometric forecasts are highly accurate
            for the majority of these countries. These forecasts
            significantly outperform predictions of simple heuristic
            models, which assume that electricity demand grows at an
            exogenous rate or is proportional to real gross domestic
            product growth. The quality of the forecasts, however,
            diminishes for the countries and regions, where rapid
            economic and structural transformation or exposure to
            conflicts and environmental disasters makes it difficult to
            establish stable historical demand trends.
repnme: repnme: Assessing the accuracy of electricity
            demand forecasts in developing countries
display_title: Assessing the accuracy of
            electricity demand forecasts in developing countries
listing_relative_url: /research/2017/02/27193230/assessing-accuracy-electricity-demand-forecasts-developing-countries
topicv3: Economic Growth m1326241 471,Energy and Extractives m1326595 507,Power and Electricity Sector m1327094 515,Environment, Natural Resources and Blue
            Economy m1327391 566,Population, Aging and Health m1327736 640
publishtoextweb_dt: 2017-02-15T00:00:00Z
historic_topic: Energy
no_of_pages: 35
author_keys: 0: author_key: 666646
model_extractions: WPS7974 Policy Research Working Paper 7974 Assessing the Accuracy of Electricity Demand Forecasts in Developing Countries _Jevgenijs Steinbuks_ Development Research Group Environment and Energy Team February 2017
Policy Research Working Paper 7974

# **Abstract**

This study assesses the accuracy of time-series econometric methods in forecasting electricity demand in developing countries. The analysis of historical time series for 106 developing countries over 1960–2012 demonstrates that econometric forecasts are highly accurate for the majority of these countries. These forecasts significantly outperform predictions of simple heuristic models, which assume that electricity demand grows at an exogenous rate or is proportional to real gross domestic product growth. The quality of the forecasts, however, diminishes for the countries and regions, where rapid economic and structural transformation or exposure to conflicts and environmental disasters makes it difficult to establish stable historical demand trends.

This paper is a product of the Environment and Energy Team, Development Research Group. It is part of a larger effort by the World Bank to provide open access to its research and make a contribution to development policy discussions around the world. Policy Research Working Papers are also posted on the Web at http://econ.worldbank.org. The author may be contacted at jsteinbuks@worldbank.org.

_The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about development issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent._ Produced by the Research Support Team
Assessing the Accuracy of Electricity Demand Forecasts in Developing Countries<sup>∗</sup> Jevgenijs Steinbuks Development Research Group, The World Bank JEL: Q47 Keywords: electricity demand forecasting, time-series econometric models

> ∗Acknowledgments: The author thanks Deb Chattopadhyay, Vivien Foster, Arthur Kochnakyan, Herman Stekler, Mike Toman, Joeri de Wit and the seminar participants at the Oxford Institute for Energy Studies and the World Bank for their helpful suggestions and comments. Responsibility for the content of the paper is the author’s alone and does not necessarily reflect the views of his institution or member countries of the World Bank.
# **1 Introduction**

Forecasting the future demand of electricity is an important issue for the utility companies, policy-makers, and private investors in developing countries. Reliable electricity demand forecasts are essential for long-term planning of future generation facilities and transmission augmentation.<sup>1</sup> As excess power is not easily storable, underestimating electricity demand results in supply shortages and forced power outages, which have detrimental effects on productivity and economic growth (Calderón and Servén 2004; Fisher-Vanden, et al., 2015; Allcott et al., 2016). However, overestimating demand may result in overinvestment in generation capacity and ultimately even higher electricity prices because, at least for traditional utilities, investment costs need to be recovered to maintain financial viability.

Forecasting long-term electricity demand is a difficult problem as it is subject to a range of uncertainties, which include, among other factors, underlying population growth, changing technology, economic conditions, and prevailing weather conditions (and the timing of those conditions). This problem can be particularly challenging in developing countries, where data are often elusive, political influences are often brought to bear, and historical electricity demand itself is more volatile owing to macroeconomic or political instability.

Despite the vast significance of having accurate and reliable electricity demand forecasts for utilities, investors and policy makers, the electricity demand forecasting literature comprises of a handful of studies. Table 1 summarizes this limited research on electricity production and consumption econometric forecasts.<sup>2</sup> Most of the studies focus on developed economies. Only five studies (Abdel-Aal and Al-Garni 1997, Sadownik and Barbosa 1999, Saab et al. 2001, Inglesi 2010, and El-Shazly 2013) forecast electricity demand for developing countries (Saudi Arabia, Brazil, Lebanon, South Africa, and the Arab Republic of Egypt, respectively). As regards data frequency, these studies are almost evenly split between short-term forecasts based on monthly data and long-term

> 1Though not the focus of this study it is worth noting that reliable electricity demand forecasts are also inportant for short-term load allocation because they help the utilities to optimize the amount of generated power, i.e., maximize their revenue and minimize operational (including environmental) costs.

> 2This summary focuses on medium- to long-term econometric projections and does not include high-frequency forecast studies of day-ahead electricity demand. It also omits noneconometric forecast studies based on soft computing techniques such as fuzzy logic, genetic algorithm, and neural networks, and bottom-up computational models such as MARKAL and LEAP. For a comprehensive review of these methods and their applications to energy forecasting, please refer to Suganthi and Samuel (2012).

2
|Method<br>ARIMA<br>ARDL<br>ARDL<br>ARIMA<br>UCM<br>ARDL / ECM<br>ECM<br>VAR / VECM<br>Linear Regression<br>ARDL / ECM<br>State Space Models<br>ARIMA<br>UCM<br>ARIMA<br>ARDL<br>istributed|
|---|
|Forecast<br>12 months<br>5 years<br>1-5 years<br>3 years<br>12 years<br>2 years<br>15 years<br>10 months<br>15 years<br>10 years<br>1-6 years<br>10 years<br>1 month<br>24 months<br>43 years<br>oregressive d<br>ion model.|
|Sample<br>1987-1993<br>1970-2007<br>1970-1990<br>1969-1990<br>1960-2008<br>1982-2010<br>1980-2005<br>1977-1991<br>1965-1999<br>1966-1999<br>1980-2007<br>1970-1999<br>1990-1994<br>1975-1989<br>1960-2007<br> ARDL: Aut<br> error correct|
|Frequency<br>Monthly<br>Yearly<br>Yearly<br>Monthly<br>Yearly<br>Yearly<br>Yearly<br>Monthly<br>Yearly<br>Yearly<br>Yearly<br>Monthly<br>Monthly<br>Monthly<br>Yearly<br>erage model. <br>CM: (Vector)|
|p.<br> av<br>)E|
|Country<br>Saudi Arabia<br>Italy<br>United States<br>United States<br>Turkey<br>Egypt, Arab Re<br>South Africa<br>United States<br>New Zealand<br>Australia<br>Taiwan, China<br>Lebanon<br>Brazil<br>Greece<br>Cyprus<br> integrated moving <br>ressive model. (V<br>model.|
|nd Al-Garni (1997)<br> (2009)<br>. (2002)<br>iu (1993)<br>Hunt (2011)<br>013)<br>)<br>(1995)<br>d Bodger (2005)<br> Smyth (2005)<br>2001)<br>d Barbosa (1999)<br>992)<br>2010)<br>A: Autoregressive <br>AR: Vector autoreg<br>served components|
|a<br>al.<br> al<br> L<br>d <br>(2<br>10<br>l. <br> an<br>nd<br>)<br> (<br>an<br>(1<br>s (<br>IM<br> V<br>ob|
|l <br> <br>t <br>d<br>n<br> <br>0<br>a<br> <br>a<br>9<br>l.<br> <br> <br>i<br>R<br>l. <br>|
|Aa<br> et<br> e<br> an<br> a<br>zly<br> (2<br>t <br>ed<br>n <br>00<br>t a<br>nik<br>zos<br>iad<br> A<br>de<br>Un|
|r<br>-<br>o <br>gi<br> <br>r<br>a<br>i <br> e<br>m<br>a<br>2<br>e<br><br>e<br>r<br>. <br>o<br>|
|o<br>el<br>c<br>a<br>is<br>ve<br>h<br>s<br>z <br>a<br>y<br>(<br> <br>w<br>k<br>a<br>s<br>m<br>:|
|Auth<br>Abd<br>Bian<br>Balt<br>Harr<br>Dila<br>El-S<br>Ingle<br>Jout<br>Moh<br>Nara<br>Pao <br>Saab<br>Sado<br>Tser<br>Zach<br>Note<br>lag <br>UCM|

3
forecasts based on yearly data. The largest part of these studies employs univariate time series methods with exogenous regressors. Few other studies use multivariate time series methods or state space econometric models. With the exception of Baltagi et al. (2002), none of these studies attempt to compare the forecast accuracy of different forecasting models.<sup>3</sup> Given significant variation in country coverage, time frame, forecast horizons, and econometric methods, the results of these studies are difficult, if not impossible, to reconcile.

The purpose of this study is to assess the accuracy of different econometric methods in forecasting electricity demand in developing countries. Based on the time series econometrics literature we first develop an econometric framework for forecasting electricity demand. We then obtain a number of electricity demand forecasts based on historical time series of 106 developing countries over the period 1960-2012. Finally, we evaluate the accuracy of the electricity demand forecasts resulting from different econometric methods and model specifications.

Our results demonstrate that time-series econometric forecasts yield highly accurate predictions for the evolution of electricity demand in the majority of developing countries. The forecasts based on the best performing method do significantly improve over the predictions of two heuristical models, commonly used by development practitioners, which assume that electricity demand grows at an exogenous rate or is proportional to real GDP growth. The quality of the forecasts, however, diminishes for the countries and regions, where rapid economic and structural transformation or exposure to conflicts and environmental disasters makes it to establish stable historical demand trends.

# **2 Forecasting Methods and Accuracy Tests**

This section briefly documents the econometric framework for forecasting electricity demand and evaluating its forecast accuracy. It first discusses implications of the stationarity property on forecastability of electricity demand time series. It then summarizes econometric methods employed for forecasting electricity demand. Finally, it describes measures of forecast errors for assessing forecast accuracy and comparing the quality of different forecasting methods.

> 3Baltagi et al. (2002) only focus on a small set of estimators within Autoregressive distributed lag (ARDL) model.

4
# **2.1 Testing for Data Stationarity**

As electricity generation and consumption data series are typically nonstationary (i.e., their mean and/or variance are varying with time), an important aspect of forecasting model selection concerns the appropriate treatment of nonstationary data. The difference-stationary processes contain stochastic trends that are integrated of order _k_ , so that differencing _k_ times yields a stationary series. The difference stationary processes have poor forecastability as forecast error variances grow linearly in the forecast horizon for these processes (Clements and Hendry 2001). Establishing whether the data generating process is the difference stationary one is therefore of particular concern.

To test whether the data are the difference stationary we perform the modified Dickey–Fuller test (also known as the DF-GLS test) proposed by Elliott et al (1996).<sup>4</sup> The test involves fitting a regression of the form where _yt_ are the electricity production series, _"t_ is the error term, _↵_ , _β_ and _δ_ are the parameters to be estimated, _k_ is the lag order of time _t_ , and ∆ is the difference operator. The DF-GLS test is performed on detrended data by Generalized Least Squares (GLS) and involves testing the null hypothesis _H_ 0 : _β_ = 0. If the test cannot reject the null hypothesis, this implies that _yt_ is a random walk, possibly with drift and the data are difference stationary. Our choice of lag order in regression (1) is based on the modified Akaike information criterion developed by Ng and Perron (2000).

# **2.2 Forecasting Methods**

Table 2 summarizes econometric methods employed for forecasting electricity demand. A brief formal representation of these methods is documented in Appendix A.1. For advanced textbook treatment of these methods, please refer to Harvey (1989), Hamilton (1994), Lütkepohl (2005), and Enders (2010).

> 4For robustness purposes we have also performed other tests for data stationarity, such as Augmented Dickey–Fuller test and Phillips and Perron (1988) unit root test. The results were little changed.

5
Table 2: Methods for Assessing Electricity Production Forecasts

|Method|Description|
|---|---|
|VAR3/VECM3|Trivariate vector autoregressive model /<br>Vector error correction model|
|VAR2/VECM2|Bivariate vector autoregressive model /<br>Vector error correction model|
|ARIMA|Autoregressive integrated moving average model|
|GARCH|Generalized autoregressive conditional<br>heteroskedasticity model|
|Holt-Winters|Holt–Winter’s linear smoothing model|
|UCM-RWD|Unobserved components model:<br>Random walk with a drift|
|UCM-LLTM|Unobserved components model:<br>Local level with deterministic trend|
|UCM-RWSC|Unobserved components model:<br>Random walk with a stochastic cycle|

These methods can be broadly grouped into three categories. Vector autoregressive model (VAR) and Vector error correction model (VECM) are the multivariate time series forecasting methods that are most appropriate when electricity demand is closely related to other macroeconomic fundamentals. Over the long term, electricity demand is influenced by economic and demographic growth, changes in energy intensity, and shifting input prices. Among these drivers, gross domestic product (GDP) is often the strongest correlate of electricity demand (Steinbuks et al., 2017). And the data for input prices and structural fundamentals affecting energy intensity are scarce for most of the developing countries. In light of the above, we employ trivariate methods, which assume that a country’s electricity demand is co-determined by GDP and population growth and bivariate methods, which assume that the country’s electricity demand is co-determined by its GDP growth only.

Autoregressive integrated moving average (ARIMA) and generalized autoregressive conditional heteroskedasticity (GARCH) models are univariate time series forecasting methods that work best when other drivers of electricity demand are exogenous and have a small effect on electricity demand. These models assume that the best predictors of electricity demand are its past realizations. Additionally, the GARCH model is particularly helpful for forecasting electricity demand in countries, where electricity supply is highly volatile.

Finally, Holt-Winters and unobserved components methods are the most suitable for forecasting electricity demand that evolves around a linear trend, 6
which can be either deterministic or stochastic. Additionally, the random walk with a stochastic cycle model (RWSC) may further improve forecasting accuracy in countries, where electricity demand exhibits cyclical behavior.

Autoregressive time series models (both multivariate and univariate) and the Holt-Winters method are applied to forecast both stationary and non-stationary electricity demand time series. Unobserved components models are only applied to forecast non-stationary electricity demand series. For all autoregressive time series models, we also estimate different specifications, assuming different lag structures (for details, please refer to Appendix). Altogether we estimate 33 model specifications for stationary electricity demand series and 36 model specifications for non-stationary series.

# **2.3 Measures of Forecast Accuracy of Individual Methods**

We employ two popular measures of forecast errors for assessing forecast accuracy of an individual method: symmetric mean absolute percent error (sMAPE) and root mean squared error (RMSE). sMAPE is defined as the average absolute percent error of electricity consumption forecasts, _y_<sup>_F_</sup> _,_ minus actuals divided by the average of absolute values of forecasts and actuals across all forecasts made for a given horizon: By using the symmetric MAPE, we avoid the problem of large errors when the actual values are close to zero, and the problem of the large difference between the absolute percentage errors when actuals are greater than forecasts and vice versa (Makridakis and Hibon, 2000).

The RMSE is a quadratic scoring rule which measures the average magnitude of the error. RMSE is defined as the difference between forecast and corresponding observed values that are each squared and then averaged over the sample: As forecast errors are squared before they are averaged, the RMSE gives a relatively high weight to larger errors. The RMSE is, therefore, most useful when large errors are particularly undesirable.

7
# **2.4 Measures of Forecast Accuracy of Competing Methods**

An important question that occurs in assessing the accuracy of electricity demand forecasts is how to formally compare the quality of different forecasting methods. Makridakis and Hibon (2000, p. 457) argue that “the absolute accuracy of the various methods is not as important as how well these methods perform relative to some benchmark.” We choose two benchmarks, the random walk model (Näıve), and the fixed GDP multiplier model (Näıve2). The former is a standard benchmark in the forecasting literature, which sets predicted electricity demand to the last available data value of stationary series. The latter benchmark assumes that electricity demand grows at the exogenous rate, which is the same rate as country’s GDP growth.<sup>5</sup> The choice of this benchmark is motivated by common practices by development professionals. Given the paucity of data and the methodological challenges, they frequently derive electricity demand forecasts from GDP-based demand growth forecasts as proxies for the growth in demand for electricity (Bhattacharyya and Timilsina 2010, Steinbuks et al. 2017).

To assess the accuracy of electricity demand forecasts, we calculate the median relative absolute error (MdRAE), which is the absolute error for the proposed model relative to the absolute error for a random walk model. It is defined as It ranges from 0 (a perfect forecast) to 1.0 (equal to the random walk), to greater than 1 (worse than the random walk). The RAE is similar to Theil’s U2, except that it is a linear rather than a quadratic measure. It is designed to be easy to interpret, and it lends itself easily to summarizing across horizons and series as it controls for scale and the difficulty of forecasting. The median RAE is recommended for comparing the accuracy of alternative models as it also controls for outliers (for information on the performance of this measure, see Armstrong and Collopy, 1992). We also compute the median percentage better measure, which reports the median of the percentage difference between sMAPE forecasting error of proposed model and one of the two benchmark

> 5For a more detailed description of these models, please refer to Appendix A.2.

8
models. Finally, we perform the Diebold and Mariano (1995) test to assess whether differences between competing forecasts are statistically significant or simply due to sampling variability.<sup>6</sup>

# **3 Electricity Demand Measurement, Data and the Forecast Horizon**

The ultimate goal of this study is to forecast electricity demand, i.e., the total final consumption.<sup>7</sup> However, in many developing countries, particularly in South Asia and Sub-saharan Africa regions, these data are either not available or available for a relatively short time frame due to difficulties with an accurate accounting of electricity at the end use level.<sup>8</sup> In light of these limitations, we have to rely on the more accurate electricity production (output) data for forecasting purposes. As electricity is a nonstorable and poorly tradable commodity, the output is a reasonable proxy for the total final consumption. However, we have to acknowledge that using electricity output data may lead to biased forecasts in a handful of developing countries with high exposure to electricity trade.

As regards data sources, the electricity generation (output) data come from the OECD/IEA Extended World Energy Balances database (IEA, 2016). The data on population and real GDP come from Penn World Tables, version 8 (Feenstra et al., 2013). The resulting dataset covers 106 developing countries over the period between 1960 and 2012.

Finally, we have to specify the within sample forecast horizons for assessing the accuracy of the forecasting methods. These are set to five and ten years, conditional on at least ten observations in the forecast validation sample. Additionally, we report out of sample forecasts over the period 2013-2022. For each country in the dataset, the out of sample forecasts are chosen based on the fore-

> 6For a more detailed description of the Diebold and Mariano (1995) test please refer to appendix section A.3.

> 7Bhattacharyya and Timilsina (2010) point out that the reliance on consumption data for the demand forecasting implies that only the satisfied demand is captured the suppressed demand is not taken into consideration. This problem can be potentially important in the presence of electricity market distortions and, correspondingly, unrealized demand (e.g., load shedding). As estimating unrealized demand typically requires high-quality micro-level panel data of enterprises and households, which are typically not available, addressing this problem is beyond the scope of this paper.

> 8These difficulties include the inaccurate recording of electricity consumption due to the poor technical capacity of electric utilities (Jamasb 2006), the absence of reliable electricity meters (Victor and Heller 2007), and large unaccounted losses from electricity theft (Smith 2004, Joseph 2010).

9
casting method corresponding to lowest within sample 5 year forecast horizon sMAPE. Appendix Table A3.1 shows the historical and forecasted electricity demand growth rates for each country. Country-specific forecast plots are also shown in the appendix.

# **4 Evaluating Accuracy of Methods**

This section describes the evaluation of different forecasting methods’ accuracy. In subsection 4.1 we compare different forecasting methods based on the chosen measures of predictive accuracy (for a description of these measures see subsection 2.3). In subsection 4.2 we examine the effectiveness of the best performing method across different categories of developing countries.

## **4.1 Comparisons across error measures**

Tables 3 and 4 report frequencies of best-performing methods according to sMAPE and RFSE criteria, respectively.<sup>9</sup> For both measures of forecasts accuracy, the GARCH model has the highest incidence of delivering best predictions over both 5- and 10-year forecast horizons, followed by the bivariate VAR / VEC model over the 5-year forecast horizon and the trivariate VAR / VEC model over the 10-year forecast horizon. None of the chosen forecasting methods appears clearly superior to other methods. However, VAR/VEC and ARIMA/GARCH models cumulatively account for a dominant share of best performing models. Other methods (Holt-Winters and Unobserved Components models) tend to perform better in a relatively small number of cases.

> 9For VAR/VEC and ARIMA/GARCH models, the best performing method is a specification with the number of lagged terms that minimizes sMAPE and RFSE forecast errors.

10
Table 3: Frequency Tabulation of Best Performing Methods: sMAPE criterion

|Mdl|5 year f|orecast horizon|10 year|forecast horizon|
|---|---|---|---|---|
|oe|Count|Frequency|Count|Frequency|
|VAR3 / VEC3|15|14.15%|30|28.57%|
|VAR2 / VEC2|21|19.81%|20|19.03%|
|GARCH|39|36.79%|34|32.35%|
|ARIMA|13|12.25%|9|8.55%|
|HOLT-WINTERS|6|5.66%|8|7.62%|
|UCM-RWD|3|2.83%|2|1.90%|
|UCM-RWC|9|8.49%|2|1.90%|
|Total|106|100%|105|100%|

Table 4: Frequency Tabulation of Best Performing Methods: RMSE criterion

|Mdl|5 year f|orecast horizon|10 year|forecast horizon|
|---|---|---|---|---|
|oe|Count|Frequency|Count|Frequency|
|VAR3 / VEC3|15|14.15%|33|31.41%|
|VAR2 / VEC2|23|21.69%|19|18.09%|
|GARCH|29|27.35%|33|31.41%|
|ARIMA|17|16.02%|7|6.65%|
|HOLT-WINTERS|7|6.60%|10|9.52%|
|UCM-RWD|5|4.72%|2|1.90%|
|UCM-LLTM|1|0.94%|0|0.00%|
|UCM-RWC|9|8.49%|1|0.95%|
|Total|106|100%|105|100%|

Tables 5 and 6 show how well the forecasting methods perform compared to benchmark models, Näıve and Näıve2. For each forecast horizon, these tables report the median percentage better measure (see subsection 2.4) as well as the percentage of times the difference between the forecast errors is statistically significant based on the Diebold and Mariano (1995) forecast accuracy test.

Table 5 compares the accuracy of forecasting methods relative to the Näıve model, which assumes that electricity demand is a random walk. We see that the best performing model based on sMAPE criterion yields considerable improvement over Näıve model. The median sMAPE forecast error of the Näıve model is 77 percent higher than forecast error of the best performing model over the 5-year forecast horizon and 74 percent higher over the 10 year forecast 11
Table 5: Comparison of various methods with Näıve as the benchmark

|Mdl|5 year fore|cast horizon<br>|10 year for|ecast horizon<br>|
|---|---|---|---|---|
|oe|Median<br>% Better|% signifcant<br>(_p_= 0_._05)|Median<br>% Better|% signifcant<br>(_p_= 0_._05)|
|Lowest sMAPE|77%|85.0%|74%|67.5%|
|VAR3 / VEC3|19%|83.3%|16%|68.3%|
|VAR2 / VEC2|7%|96.5%|11%|72.5%|
|GARCH|37%|84.6%|10%|69.3%|
|ARIMA|13%|83.9%|-8%|70.1%|
|HOLT-WINTERS|-2%|86.0%|-11%|78.1%|
|UCM-RWD|-9%|86.8%|-22%|82.5%|
|UCM-LLTM|-10%|88.4%|-23%|82.5%|
|UCM-RWC|-40%|94.8%|-58%|87.9%|

horizon. And the difference between forecast errors is statistically significant (assuming 5 percent level) for 85 percent of countries over the 5 year forecast horizon and for 67.5 percent of countries over the 10-year forecast horizon. As regards specific forecasting methods, VAR/VEC and GARCH methods yield more accurate forecasts than the Näıve model over both 5- and 10-year forecast horizons, with median accuracy improvement ranging between 7 and 37 percent. To the contrary, the Holt-Winters method and Unobserved Components Models yield less accurate forecasts over both 5- and 10-year forecast horizons, with median accuracy decline ranging between 2 and 58 percent. Finally, the ARIMA model produces more accurate forecasts than the Näıve model over the 5-year forecast horizon, with median accuracy improvement of 13 percent. However, the ARIMA model yields less accurate forecasts than the Näıve model over 10year forecast horizon, with median accuracy decline of 8 percent. Regardless of the direction of forecast error differences, they are mostly statistically significant across all methods, ranging between 83.3 to 96.5 percent of countries over the 5-year forecast horizon, and between 68.3 and 87.9 percent of countries over the 10-year forecast horizon.

Table 6 compares the accuracy of forecasting methods relative to Näıve2 model, which assumes that electricity demand grows at the same rate as GDP. The results are qualitatively similar to those reported in Table 5, and the quantitative improvements over forecasts of Näıve2 model are even more pronounced. The median sMAPE forecast error of the Näıve2 model is 184% percent higher than forecast error of the best performing model over the 5-year forecast hori12
Table 6: Comparison of various methods with Näıve2 as the benchmark

|Mdl|5 year fore|cast horizon<br>|10 year for|ecast horizon<br>|
|---|---|---|---|---|
|oe|Median<br>% Better|% signifcant<br>(_p_= 0_._05)|Median<br>% Better|% signifcant<br>(_p_= 0_._05)|
|Lowest sMAPE|184%|88.0%|124%|73.0%|
|VAR3 / VEC3|68%|91.2%|52%|73.5%|
|VAR2 / VEC2|57%|95.2%|43%|73.7%|
|GARCH|121%|85.1%|54%|74.5%|
|ARIMA|69%|91.8%|23%|76.3%|
|HOLT-WINTERS|45%|90.2%|17%|81.7%|
|UCM-RWD|35%|90.5%|1%|79.5%|
|UCM-LLTM|31%|90.8%|0%|77.1%|
|UCM-RWC|-16%|94.2%|-40%|91.2%|

zon and 124 percent higher over the 10-year forecast horizon. The performance of specific forecasting methods over the Näıve2 forecasting model is also improved. Specifically, VAR/VEC, GARCH, ARIMA, and Holt-Winters methods all yield more accurate forecasts than the Näıve2 model over both 5- and 10year forecast horizons, with median accuracy improvement ranging between 17 and 121 percent. As regards Unobserved Components models, both RWD and LLTM methods deliver more accurate forecasts over the 5-year forecast horizon, whereas their forecast accuracy over the 5 year forecast horizon is of the same magnitude as that of the Näıve2 model. Finally, the RWC model yields less accurate forecasts than the Näıve2 model over both 5- and 10-year forecast horizons, with median accuracy decline between 16 and 40 percent. Similar to results reported in Table 5, the differences in predicted forecasts between forecasting methods and the Näıve2 model are mostly statistically significant across all methods, ranging between 85.1 to 95.2 percent of countries over the 5-year forecast horizon, and between 73 and 91.2 percent of countries over the 10-year forecast horizon.

# **4.2 Comparisons across developing country groups**

Tables 7 - 10 compare effectiveness of the best performing method (based on sMAPE criterion) across developing countries based on their regional, income, generation capacity and energy intensity characteristics. Table 7 reports the average sMAPE and MdRAE measures of forecast accuracy across regions over 13
Table 7: Comparison of Forecast Errors across Regions

|Ri|5 year for|ecast horizon|10 year f|orecast horizon|
|---|---|---|---|---|
|egon|sMAPE|MdRAE|sMAPE|MdRAE|
|AFR|0.09|0.44|0.11|0.63|
|EAP|0.05|0.54|0.05|0.53|
|ECA|0.06|0.45|0.07|0.49|
|LAC|0.05|0.52|0.08|0.62|
|MENA|0.05|0.42|0.08|0.52|
|SAR|0.02|0.25|0.05|0.41|

the 5- and 10-year forecast horizons. All in all, the best performing method is highly accurate with average sMAPE varying between 2 and 9 percent over the 5-year forecasting horizon and between 5 and 11 percent over the 10-year forecasting horizon, respectively. Consistent with the results from the previous section, the best performing method is also more accurate than the Näıve model, with average MdRAE varying between 0.25 and 0.54 over the 5-year forecasting horizon, and between 0.41 and 0.63 over the 10-year forecasting horizon, respectively.

It follows from Table 7 that the forecast accuracy is the highest for the countries of the South Asia region over both 5- and 10-year forecasting horizons and across both types of error accuracy measures. The forecast accuracy is the lowest for the Sub-Saharan Africa region based on the sMAPE criterion over both 5- and 10- year forecasting horizons, with other regions having broadly comparable forecast errors. The forecast accuracy based on the MdRAE criterion is the lowest for the East Asia and Pacific and the Latin America regions over the 5-year forecasting horizon, and for the Sub-Saharan Africa and the Latin America regions over the 10-year forecasting horizon.

To frther elucidate the observed differences in the forecast accuracy across regions, this study also reports the average sMAPE and MdRAE measures of forecast accuracy for individual countries, grouped across regions over the 5- and 10-year forecast horizons (see Appendix Table A3.2). For most countries, both sMAPE and MdRAE errors are small, which indicates that the best performing method is both highly accurate and yields considerable improvements over the Näıve model. However, the forecasting accuracy is greatly diminished for countries that have recently undertaken major investments (Ethiopia, Cameroon, Myanmar) or disinvestments (Lithuania) in electricity generation assets; coun14
tries that have volatile electricity demand and / or rely heavily on electricity imports (Albania, Benin, Botswana); or countries affected by major conflicts (Iraq, Libya, Syrian Arab Republic) or environmental disasters (Haiti).

Table 8 shows the average sMAPE and MdRAE measures of forecast accuracy across country income groups over the 5- and 10-year forecast horizons. It follows from Table 8 that electricity demand forecasts are less accurate for the lower income countries. For low income countries, the average sMAPE is 9 and 12 percent over the 5- and 10-year forecast horizons, respectively. These errors are twice as high compared to high-income countries. The accuracy of forecasting methods relative to the Näıve model is also considerably diminished for lower income countries. For low-income countries, the value of MdRAE is 0.79 over the 10-year forecast horizon, which indicates that the best performing method is just 21 percent more accurate than the Näıve model.

Table 8: Comparison of Forecast Errors across Income Groups

|I|5 year for|ecast horizon|10 year|forecast horizon|
|---|---|---|---|---|
|ncome|sMAPE|MdRAE|sMAPE|MdRAE|
|Low|0.09|0.49|0.12|0.79|
|Low-Middle|0.05|0.39|0.07|0.44|
|Upper-Middle|0.06|0.50|0.08|0.59|
|High|0.05|0.46|0.06|0.49|

Table 9 shows the average sMAPE and MdRAE measures of forecast accuracy across installed capacity categories over the 5 and 10 year forecast horizons. Forecast accuracy is the highest for the countries with large installed capacity and diminishes significantly as the size of the installed capacity falls. For countries with the largest installed capacity (over 100GW), the average sMAPE is 3 and 4 percent over the 5 and 10 year forecast horizons, respectively. These errors are twice as low as compared to countries with medium installed capacity size (1 to 10 GW). For countries with the smallest installed capacity (less than 1GW), electricity generation is particularly difficult to forecast, with the average sMAPE of 9 and 10 percent over the 5- and 10-year forecast horizons, respectively. The countries with large installed capacity also have higher accuracy of forecasting methods relative to the Näıve model. The value of MdRAE for countries with the largest installed capacity (over 100GW) is 0.34 and 0.23 over the 10-year forecast horizon, which is 1.5-2 times smaller as compared to 15
countries with the smallest installed capacity size (less than 1GW).

Table 9: Comparison of Forecast Errors across Installed Capacity Categories

|ItlldCit|5 year fo|recast horizon|10 year|forecast horizon|
|---|---|---|---|---|
|nsae apacy|sMAPE|MdRAE|sMAPE|MdRAE|
|less than 1GW|0.09|0.50|0.10|0.57|
|1GW-10GW|0.06|0.47|0.08|0.59|
|10GW-100GW|0.02|0.41|0.04|0.47|
|more than 100GW|0.03|0.34|0.04|0.23|

Table 10 shows the average sMAPE and MdRAE measures of forecast accuracy across energy intensity categories over the 5 and 10 year forecast horizons. Forecast accuracy is the highest for the most energy intensive countries (more than 12$/kgoe) with average sMAPE of 2 and 4 percent over the 5 and 10 year forecast horizons, respectively. Compared to other countries these errors are 2 to 3 times smaller over the 5 year forecast horizon and 1.5 to 2 times smaller over the 10 year forecast horizon. However, the more energy intensive countries also have the lower accuracy of forecasting methods relative to the Näıve model, at least for the shorter term forecast horizon. The value of MdRAE for the most energy intensive countries is 0.75 over the 5 year forecast horizon, which is twice as high as compared to the most energy efficient countries. To the contrary, the most energy efficient countries have the lowest forecast accuracy relative to the Näıve model over the 10 year forecast horizon, with MdRAE of 0.68.

Table 10: Comparison of Forecast Errors across Energy Intensity Categories

|EnerIntensit|5 year for|ecast horizon|10 year|forecast horizon|
|---|---|---|---|---|
|gy y|sMAPE|MdRAE|sMAPE|MdRAE|
|<3$/kgoe|0.04|0.34|0.09|0.68|
|3$/kg-6$/kgoe|0.06|0.41|0.08|0.47|
|6$/kg-9$/kgoe|0.07|0.43|0.09|0.56|
|9$/kg-12$/kgoe|0.06|0.50|0.06|0.46|
|>12$/kgoe|0.02|0.75|0.04|0.52|

16
# **5 Conclusions**

Accurate projections of electricity demand are essential for planning power systems and appraising investment projects in developing countries. Nonetheless, demand forecasting issues are not rigorously studied and are not always given adequate attention among development practitioners. This study demonstrates that time-series econometric methods yield highly accurate forecast predictions for the majority of developing countries. Econometric forecasts significantly outperform simple heuristical rules used by practitioners, who frequently assume that electricity demand grows at some exogenous rate or is proportional to real GDP growth. These improvements notwithstanding, relying on time-series econometric methods alone may produce inaccurate forecasts in some developing countries. We show that econometric forecasts of electricity demand are challenging for developing countries that are in the process of rapid economic and structural transformation or are prone to conflicts and environmental disasters. These include, among other, the countries in Sub-Saharan Africa region, the low-income countries, and the countries with small electricity generation systems. For those countries, in particular, a more rigorous forecasting approach, using a combination of micro-econometric and computational modeling methods would be preferred.

# **References**

- [1] Abdel-Aal, R. E., and Al-Garni, A. Z. (1997). Forecasting Monthly Electric Energy Consumption in Eastern Saudi Arabia Using Univariate Time-series Analysis. _Energy_ , 22(11), 1059-1069.

- [2] Allcott, H., Collard-Wexler, A., and O’Connell, S. D. (2016). How Do Electricity Shortages Affect Productivity? Evidence from India. _American Economic Review_ , 106(3), 587-624.

- [3] Armstrong, J. S., and Collopy, F. (1992). Error Measures for Generalizing about Forecasting Methods: Empirical Comparisons. _International Journal of Forecasting_ , 8(1), 69-80.

- [4] Baltagi, B. H., Bresson, G., and Pirotte, A. (2002). Comparison of Forecast Performance for Homogeneous, Heterogeneous and Shrinkage Estimators: Some Empirical Evidence from US Electricity and Natural Gas Consumption. _Economics Letters_ , 76(3), 375-382.

17
- [5] Bhattacharyya, S. C., and Timilsina, G. R. (2010). Modelling Energy Demand of Developing Countries: Are the Specific Features Adequately Captured? _Energy Policy_ , 38(4), 1979-1990.

- [6] Bianco, V., Manca, O., and Nardini, S. (2009). Electricity Consumption Forecasting in Italy using Linear Regression Models. _Energy_ , 34(9), 14131421.

- [7] Calderón, C., and Servén, L. (2004). The Effects of Infrastructure Development on Growth and Income Distribution. _World Bank Policy Research Paper 3400._

- [8] Clements, M. P., and Hendry, D. F. (2001). _Forecasting Non-Stationary Economic Time Series._ MIT Press.

- [9] Diebold, F.X., and Mariano, R.S. (1995). Comparing Predictive Accuracy. _Journal of Business and Economic Statistics_ 13, 253-63.

- [10] Dilaver, Z., & Hunt, L. C. (2011). Modelling and Forecasting Turkish Residential Electricity Demand. _Energy Policy_ , 39(6), 3117-3127.

- [11] Elliott, G. R., Rothenberg, T. J. and Stock, J. H. (1996). Efficient Tests for an Autoregressive Unit Root. _Econometrica_ 64, 813–836.

- [12] El-Shazly, A. (2013). Electricity Demand Analysis and Forecasting: A Panel Cointegration Approach. _Energy Economics_ , 40, 251-258.

- [13] Enders, W. (2008). _Applied Econometric Time Series_ . John Wiley & Sons.

- [14] Feenstra, R.C., Inklaar, R. and Timmer, M. (2013). The Next Generation of the Penn World Table. _NBER Working Paper 19255_

- [15] Fisher-Vanden, K., Mansur, E. and Wang, Q. (2015). “Electricity Shortages and Firm Productivity: Evidence from China’s Industrial Firms.” _Journal of Development Economics_ , 114, 172-188.

- [16] Gardner, E. S., Jr. (1985). Exponential Smoothing: The State of the Art. _Journal of Forecasting_ 4, 1–28.

- [17] Hamilton, J.D. (1994). _Time Series Analysis_ . Princeton University Press.

- [18] Harris, J. L., and Liu, L. M. (1993). Dynamic Structural Analysis and Forecasting of Residential Electricity Consumption. _International Journal of Forecasting,_ 9(4), 437-455.

18
- [19] Harvey, A.C. (1989). _Forecasting, Structural Time Series Models and the Kalman Filter_ . Cambridge University Press.

- [20] IEA (2016). Extended World Energy Balances. IEA World Energy Statistics and Balances (database). DOI: http://dx.doi.org/10.1787/data-00513en (Accessed on 23 June 2016).

- [21] Inglesi, R. (2010). Aggregate Electricity Demand in South Africa: Conditional Forecasts to 2030. _Applied Energy_ , 87(1), 197-204.

- [22] Jamasb, T. (2006). Between the State and Market: Electricity Sector Reform in Developing Countries. _Utilities Policy_ , 14(1), 14-30.

- [23] Johansen, S. (1995). _Likelihood-Based Inference in Cointegrated Vector Autoregressive Models_ . Oxford University Press.

- [24] Joseph, K. L. (2010). The Politics of Power: Electricity Reform in India. _Energy Policy_ , 38(1), 503-511.

- [25] Joutz, F. L., Maddala, G. S., and Trost, R. P. (1995). An Integrated Bayesian Vector Auto Regression and Error Correction Model for Forecasting Electricity Consumption and Prices. _Journal of Forecasting_ , 14(3), 287-310.

- [26] Luger, R. (2004). Exact Tests of Equal Forecast Accuracy with an Application to the Term Structure of Interest Rates. _Bank of Canada Working Paper 2004-2._

- [27] Lütkepohl, H. (2005). _New Introduction to Multiple Time Series Analysis_ . Springer.

- [28] Makridakis, S., Andersen, A., Carbone, R., Fildes, R., Hibon, M., Lewandowski, R., Newton, J., Parzen, E., and Winkler, R. (1982). The Accuracy of Extrapolation (Time Series) Methods: Results of a Forecasting Competition. _Journal of Forecasting_ , 1(2), 111-153.

- [29] Makridakis, S., Chatfield, C., Hibon, M., Lawrence, M., Mills, T., Ord, K., and Simmons, L. F. (1993). The M2-competition: A Real-time Judgmentally Based Forecasting Study. _International Journal of Forecasting_ , 9(1), 5-22.

19
- [30] Makridakis, S., and Hibon, M. (2000). The M3-Competition: Results, Conclusions and Implications. _International Journal of Forecasting_ , 16(4), 451476.

- [31] Mohamed, Z., and Bodger, P. (2005). Forecasting Electricity Consumption In New Zealand Using Economic and Demographic Variables. _Energy_ , 30(10), 1833-1843.

- [32] Narayan, P. K., and Smyth, R. (2005). Electricity Consumption, Employment and Real Income in Australia: Evidence from Multivariate Granger Causality Tests. _Energy Polic_ y, 33(9), 1109-1116.

- [33] Ng, S., and Perron, P. (2000). Lag Length Selection and the Construction of Unit Root Tests with Good Size and Power. _Econometrica,_ 69, 1519–1554.

- [34] Pao, H. T. (2009). Forecast of Electricity Consumption and Economic Growth in Taiwan by State Space Modeling. _Energy_ , 34(11), 1779-1791.

- [35] Phillips, P. C. B., and Perron, P. (1988). Testing for a Unit Root in Time Series Regression. _Biometrika,_ 75, 335–346.

- [36] Saab, S., Badr, E., & Nasr, G. (2001). Univariate Modeling and Forecasting of Energy Consumption: The Case of Electricity in Lebanon. _Energy_ , 26(1), 1-14.

- [37] Sadownik, R., and Barbosa, E. P. (1999). Short-Term Forecasting of Industrial Electricity Consumption in Brazil. _Journal of Forecasting_ , 18(3), 215-224.

- [38] Smith, T. B. (2004). Electricity Theft: A Comparative Analysis. _Energy Policy_ , 32(18), 2067-2076.

- [39] Steinbuks, J., de Wit, J., Kochnakyan A. and Foster, V. (2017). Forecasting Electricity Demand: An Aid for Practitioners. _Live Wire_ 2017/1. World Bank, Washington, DC.

- [40] Suganthi, L., and Samuel, A. A. (2012). Energy Models for Demand Forecasting—A Review. _Renewable and Sustainable Energy Reviews_ , 16(2), 1223-1240

- [41] Tserkezos, E. D. (1992). Forecasting Residential Electricity Consumption in Greece using Monthly and Quarterly Data. _Energy Economics_ , 14(3), 226-232.

20
- [42] Victor, D. G., and Heller, T. C. (Eds.). (2007). _The Political Economy of Power Sector Reform: The Experiences of Five Major Developing Countries_ . Cambridge University Press.

- [43] Zachariadis, T. (2010). Forecast of Electricity Consumption in Cyprus up to the Year 2030: The Potential Impact of Climate Change. _Energy Policy_ , 38(2), 744-750.

21