metadata: authors: 0: author: Mendez Ramos,Fabian
count: World
docna: 0: docna: Variance and Skewness in Density Predictions
            : A World GDP Growth Forecast Assessment
docty: Policy Research Working Paper
subtopic: Inflation,Energy and Environment,Energy Demand,Energy and Mining,Financial Structures,Consumption,Fiscal & Monetary Policy
teratopic: Energy,Finance and Financial Sector Development,Macroeconomics and Economic Growth
abstracts: cdata!: The paper introduces a Bayesian
            cross-entropy forecast (BCEF) procedure to assess the
            variance and skewness in density forecasting. The
            methodology decomposes the variance and skewness of the
            predictive distribution accounting for the shares of
            selected risk factors. The method assigns probability
            distributions to baseline-projections of an economic or
            social random variable -- for example, gross domestic
            product growth, inflation, population growth, poverty
            headcount, among others -- combining ex-post and ex-ante
            market information. The generated asymmetric density
            forecasts use information derived from surveys on
            expectations and implied statistics of predictive models.
            The BCEF procedure is applied to produce world GDP growth
            forecasts for three-year horizons using information spanning
            the period of October 2005-August 2015. The scores indicate
            that the BCEF density forecasts are more accurate and
            reliable than some naïve—symmetric and normal distributed
            confidence interval—predictions, illustrating the
            value-added of the introduced methodology.
repnme: repnme: Variance and Skewness in Density
            Predictions : A World GDP Growth Forecast Assessment
display_title: Variance and Skewness in Density
            Predictions : A World GDP Growth Forecast Assessment
listing_relative_url: /research/2017/08/27900161/variance-skewness-density-predictions-world-gdp-growth-forecast-assessment
topicv3: Energy and Extractives m1326595 507,Inflation m1326709 0870F4727ACED1CF8525815C00471347,Macroeconomic and Structural Policies
            m1327380 439,Central Banks m1327453 1235,Population, Aging and Health m1327736 640
publishtoextweb_dt: 2017-08-14T00:00:00Z
historic_topic: Energy,Finance and Financial Sector Development,Macroeconomics and Economic Growth
no_of_pages: 50
author_keys: 0: author_key: 1247803
model_extractions: Policy Research Working Paper 8165 Variance and Skewness in Density Predictions A World GDP Growth Forecast Assessment _Fabian Mendez-Ramos_ Development Research Group Macroeconomics and Growth Team August 2017
Policy Research Working Paper 8165

# **Abstract**

The paper introduces a Bayesian cross-entropy forecast (BCEF) procedure to assess the variance and skewness in density forecasting. The methodology decomposes the variance and skewness of the predictive distribution accounting for the shares of selected risk factors. The method assigns probability distributions to baseline-projections of an economic or social random variable—for example, gross domestic product growth, inflation, population growth, poverty headcount, among others—combining ex-post and ex-ante market information. The generated asymmetric density forecasts use information derived from surveys on expectations and implied statistics of predictive models. The BCEF procedure is applied to produce world GDP growth forecasts for three-year horizons using information spanning the period of October 2005–August 2015. The scores indicate that the BCEF density forecasts are more accurate and reliable than some naïve—symmetric and normal distributed confidence interval—predictions, illustrating the value-added of the introduced methodology.

This paper is a product of the Macroeconomics and Growth Team, Development Research Group. It is part of a larger effort by the World Bank to provide open access to its research and make a contribution to development policy discussions around the world. Policy Research Working Papers are also posted on the Web at http://econ.worldbank.org. The author may be contacted at fmendezramos@worldbank.org.

_The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about development issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent._ Produced by the Research Support Team
VARIANCE AND SKEWNESS IN DENSITY PREDICTIONS: A WORLD GDP GROWTH FORECAST ASSESSMENT

# Fabian Mendez-Ramos<sup>1</sup>

(Version: November 1, 2019)

## C11, C53, C54

**_Keywords:_** Non-symmetric density forecasts, forecast uncertainty, Bayesian cross-entropy, scoring rules, the continuous ranked probability score, world GDP growth, implied volatility, consensus forecasts, the balance of risks, variance decomposition, skewness decomposition.

I am thankful to Israel Osorio Rodarte, Raul Ibarra Ramírez, Aqib Aslam, Norman Loayza, colleagues from the Development Prospects Group: Delfin S. Go, Ayhan Kose, Franziska L. Ohnsorge and participants of the DECRG Seminar Series in Kuala Lumpur for helpful comments, inputs, and suggestions at various stages of the study. I am also thankful to Dr Michael McCracken and to several anonymous reviewers for helpful and constructive comments in previous versions of the paper. Contact address: Fabian Mendez Ramos, Development Research Group, The World Bank. Email: fmendezramos@worldbank.org.
“... if one model outforecasts another, the poorer of the two cannot be sensibly used for policy purposes”.

Granger (1986).

# 1. INTRODUCTION

The relationship between informed decisions and knowledge about foreseen risks remains a central theme in the areas of policymaking and investment. The lack of suitable predictive models and basic rules of thumb leaves policy-makers and investors make decisions without proper knowledge of upside and downside risk dimensions associated with a random phenomenon. Especially in economics, the construction of appropriate predictive models should be a binding task before the generation of a policy or investment assessment. If, in terms of central measures and confidence intervals, a model is not able to predict trustworthy outcomes, the model itself is not useful for policy benchmarking as its predictions are misleading.

In this paper, we introduce a Bayesian cross-entropy forecasting (BCEF) procedure to generate density forecasts<sup>2</sup> solving an information-theoretical problem under noisy and scattered information. Our method combines ex-post and ex-ante knowledge of the variable of interest and risk factors that affect it. The empirical application generates density forecasts of world GDP growth associating selected risk factors: terms spreads, absolute deviations of inflation targets, energy prices, and the S&P 500 index prices. The BCEF method is Bayesian in the sense that allows the use of prior probability distributions to initially assume the structure of parameters that affect the distribution of our forecasted variable.

There are four key advantages of using the BCEF procedure to generate density forecasts. First, prior information of underlying unknown and unobserved parameters—or latent variables such as error terms—can be incorporated into a defined system to find less noisy levels of information in the noisy parameters. Second, the Bayesian entropy procedure recovers and processes information when the underlying sampling model is partly or incorrectly known, and

> 2 From here onwards, density forecast, probabilistic forecast, or fan chart are used interchangeably.

2
the data is limited, partial, or incomplete (Golan, Judge, and Miller 1996). Third, the BCEF procedure complements already established forecasting econometric methods and assesses the   uncertainty of a point forecast most likely outcome quantifying the magnitude of the variance and skewness of the confidence interval.<sup>3</sup> Fourth, the BCEF method also quantifies the risk-factor shares affecting the assessed variance and skewness of the density forecast. In this regard, the introduced procedure includes decompositions of ex-ante variance and skewness<sup>4</sup> of the predicted random variable. Both the variance and the skewness decompositions highlight the importance of ex-ante risk factor information in shaping the density forecasts.

The BCEF method is an information-theoretical procedure that constructs less information-noisy density forecasts given three significant restrictions: the constraint of a parametric functional form (of the density forecast), the consistency and behavioral constraints, and the selected prior information. Our BCEF procedure combines the minimum cross-entropy method (Golan, Judge, and Robinson 1994, Golan 2017) with ex-ante available data to predict future outcomes in the form of density forecasts. Thus, the introduced methodology adopts the Kullback-Leibler—or relative- or cross-entropy—function. By construction, the method relies on a principle that optimizes the error distribution of the variables of interest: minimization of the noise between the prior and the posterior distribution of the error terms. The solution of the BCEF method provides a posterior distribution of error terms assigned to parameters that describe the distribution of a study variable: world GDP growth in our empirical exercise. These posterior distributions are the closest to the prior distribution of the error components constrained in satisfying all the consistency and behavioral conditions of the model. As a minimum cross-entropy procedure, the BCEF procedure is

> 3 Although in the BCEF procedure the point forecasts are taken as given, the accuracy of the density forecasts are still meaningful as they are evaluated in ex-post manner using proper scoring rules. The evaluation of the density forecasts via scoring rules gives the incentives to elicit accurate predictive distributions. The variance and skewness of the forecasted confidence intervals play an important role in the final ex-post score. See Appendix 1 for more details about the incentive mechanisms to release honest density forecasts behind proper scoring rules. 4 We use the terms ex-ante variance and ex-ante skewness to denote a conditional variance and skewness given information that considers both ex-ante and ex-post observations.

3
optimal in the sense that it minimizes information losses in moving from a prior distribution assumption to a posterior distribution estimate.<sup>5</sup> Whereas forecasters disagree in assigning prior distributions based on their judgment and personal information, these disagreements are in part a reflection of underlying uncertainty (Al-Najjar and Shmaya 2015). The BCEF procedure is flexible to exploit heterogeneity in beliefs embedded in assumed prior probability distributions of the parameters defining the study variable. We can get BCEF estimates under an almost infinite number of prior distributions, which, in most cases, the results will converge to similar posterior estimates.<sup>6</sup> A critical characteristic of our BCEF method is that it uses ex-ante information to generate density forecasts. The used ex-ante information is extracted from surveys on expectations and implied statistics of forecasting models. While the use of ex-ante information to predict uncertainty in future outcomes is becoming more common, there is still some disagreement regarding its predictive power. Rich and Tracy (2010) build up uncertainty statistics to analyze inflation forecasts from the Survey of Professional Forecasters (SPF). Their results found little evidence that disagreement among forecasters is a good proxy for inflation uncertainty. Using the same SPF data, in contrast, Giordani and Söderlind (2003) find that difference among forecasters is a good measure for inflation and output growth uncertainties. More recent literature uses ex-ante forecasts errors based on SPF sources to improve the accuracy of uncertainty estimates around macroeconomic forecasts (Clark, McCracken, and Mertens 2018).

The empirical application of the introduced BCEF method generates monthly density forecasts of world GDP growth based on ex-post information spanning the period of October 2005–August 2015. The BCFE density forecasts for world GDP growth assess three annual horizons. The accuracy of the BCEF density forecasts is evaluated ex-post using the continuous

> 5 In the language of Bayesian inference, the BCEF procedure capture a measure of the information gained when one revises one’s beliefs from the prior probability distribution to the posterior probability distribution. 6 Using incorrect priors will result in solutions that may be far from the underlying truth. In statistical terms, the solution will be biased (Golan 2017, p. 314).

4
ranked probability score (CRPS). Besides, the BCEF forecasts are compared with “naïve” density forecasts. This naïve forecasts are time-inexpensive and only require the standard deviation of historical forecast errors of world GDP growth predictions and a normality distribution assumption. Unweighted and weighted CRPS scores show that the BCEF fan charts are more accurate and reliable than the “naïve” density forecasts. In terms of forecast accuracy and reliability, the results show there is an incentive to invest in a method like the BCEF presented in this paper.

Furthermore, by construction, the BCEF results include variance and skewness decompositions of the density forecasts. For instance, the August 2015 density forecast indicates that around 68 percent of the world GDP growth uncertainty  measured via the  variance of the predictive distribution is directly explained with the four selected risk factors, whereas “other” risk factor components explain the rest of this predictive variance. In terms of the skewness decomposition, crude oil prices and “other” risk factor components are the elements that contribute more to the direction and magnitude of the world GDP growth predictive skewness across the three forecast horizons.

We describe the BCEF procedure in the following sections of the paper. Econometric considerations, unobserved parameters, distributional assumptions, prior information, and the BCEF minimization problem is explained in Section 2. The BCEF method is tested in Section 3, producing density forecasts to predict world GDP growth. The practical exercise targets world GDP growth, which is a variable that faces—ex-ante and ex-post—concerns of being incorrectly known, incorrectly quantified, or both. Risk factors affecting world GDP growth, prior values, the estimated density forecast series, and their evaluation scores are also discussed in Section 3. Finally, the main conclusions of this study are presented in Section 4.

# 2. THE MODEL

The introduced BCEF technique allows for the quantification of risks in the prediction of a studied variable. The BCEF procedure can be applied to predict economic variables; however, it is also suitable in other fields where unobserved information prevails. Examples of studied or 5
target variables are GDP growth, poverty headcount rates, population growth, consumer price inflation, stock market prices or returns, consumption or investment levels, among others. The generation of BCEF density forecasts uses two categories of constraints: behavioral and statistical ones. The behavioral restrictions rely on assumptions about the statistical shape and parameter performance of the predictive density forecasts. The statistical—or consistency— constraints rely on formal probability statements that govern random variables.

# 2.1 THE LINEAR ASSUMPTION

The BCEF procedure is a non-linear inversion procedure. It provides a basis for information recovery while helping to make conservative inferences about an unknown and unobservable number, vector, or function (Golan, Judge, and Miller 1996). In the case of the estimation of predictive density functions, the target variable—and its statistics—might be unobserved and indeed not accessible for direct measurement. The assembled system incorporates not only expost information but also ex-ante knowledge about dispersion and skewness measures of associated risk factors of the target random variable.

The forecasted variable, , is assumed to be a function of a subset of risk factors, YY ZZ⊆ . The selection of the subset of risk factors, **,** might depend on empirical results and ΘΘ ZZ theoretical underpinnings associated with . Note that denotes the complete set of risk YY ΘΘ factors; }, where states for the complement of . The forecasted dispersion ΘΘ= {ZZ, ZZ<sup>cc</sup> ZZ<sup>cc</sup> ZZ and skewness measures of are assumed to depend on—ex-post and ex-ante—information of YY . The basic idea is that markets perform not only using ex-post information, but they also ZZ behave incorporating ex-ante information once it is made available. The uncertainty of is YY approximated using dispersion and skewness statistics. Both statistics are modeled in separate forms to support their specific constraints. For instance, while the variance is always a positive number, the third central moment statistic can take negative and positive values. The BCEF procedure looks to exploit the potential uneven and noisy trajectories of ZZ and using a minimization information-theoretical setup. The combination of the ex-post and YY 6
<!-- Start of picture text --> R= tr<br>Z=(Z )<br>{Z }<br>{}<br>G={t <-+<T}<br>(Q.F,{F ,P)<br><!-- End of picture text -->
<!-- Start of picture text --> {Z } {Z }<br>var(Z ) (Z )<br>var(Y ) v(¥ ) var(Z )<br>V(Z)<br>var(Y ) v(¥ )<br>EY)<br>mode(Y )<br>var(Y ) v(¥ )<br>mode )<br>var (Y ) v(Y )<br><!-- End of picture text -->
PROPOSITION 2: Under the linear assumption in (1), the predicted conditional skewness of the study variable is a linear combination of the risk factor skewness plus an error term as defined in (3).

Besides, note that the same coefficients 1 2 } that come from the econometric {aa , aa , ... , aaRR assumption in (1) constrain the estimation of Equation (2) and (3). Thus, for consistency purposes, (2) and (3) must be estimated simultaneously. As presented in Subsection 2.5, the BCEF procedure is a constrained non-linear programming model. The simultaneous recovery of the coefficients in the set restricts the number of values that the parameters can take. AA While the coefficients in estimate the direct effects on the statistics of the forecasted AA variables, the cross-effects of the risk factors on the target variable, , are also captured and YY added up in the error terms, 2 and 3. For analytical purposes, the coefficients in Equations εεσσ εεγγ (2) and (3) should be read as first-order effects of the risk statistics over the forecasted variable, . YY 9
# 2.2 LATENT STATISTICS

The BCEF procedure looks to predict the variance and skewness of for period , for YY TT+ h any horizon h∈H .<sup>7</sup> However, the information of these statistics is not observable or uncertain, i.e., the forecaster does not have data on the series |F ��TT 1 and �vvaarr�YYtt+h tt tt=tt �<sup>γγ</sup> �<sup>YY</sup> ��TT 1. In our empirical exercise, the series of predicted world GDP growth variance tt+h<sup>|F</sup> tt tt=tt and skewness are not available. This lack of information makes complicated the econometric estimation of (2) and (3). While simultaneously estimating (2) and (3), the BCEF method recovers the i) the set of risk-share coefficients 1 2 }; ii) the cross-moment AA= {aa , aa , ... , aaRR effects and “other” risk-factor components affecting the predictive variance and skewness, �<sup>εε</sup> �TT and �TT , and iii) the conditional variance and skewness of the study 1 1 σσ<sup>2</sup> ,tt+h tt=tt �εεγγ,tt+h tt=tt variable, |F ��TT 1 and |F ��TT 1. �vvaarr�YYtt+h tt tt=tt �γγ�YYtt+h tt tt=tt

# 2.3 DISTRIBUTIONAL ASSUMPTIONS

Even though several parametric distributions are available to model density forecast, we selected a distribution that mainly facilitates the estimation of variance and skewness statistics as we are interested in studying nonsymmetric behaviors. Thus, to generate the predictive probability density function of the study variable, , we assume the two-piece normal (TPN) YY function.<sup>8</sup> The variable is predicted using three statistics: a central measure like the mode, YY a dispersion measure like the variance, and a skewness measure like the third central moment statistic. The TPN assumption (Wallis 2014) is flexible enough to allow skewed distributional shapes of the predicted random variable. In some cases, the TPN assumption can be a restriction. This distributional constraint can be especially stringent in variables with very high or shallow levels of excess kurtosis, i.e., leptokurtic and platykurtic shapes, respectively.

> 7 Note that the actual or most recent period is denoted with . 8 We acknowledge that the distributional assumption of the TPN distribution is restrictive. For instance, bi-modal TT distributions are excluded in this assumption. However, we consider that the non-symmetric option that the TPN distribution brings is still of much value for density forecasting purposes and forward-looking uncertainty analysis.

10
Ex-ante dispersion and skewness information are inputs to identify the forecast uncertainty dimensions of . This ex-ante information is complemented with an already YY elicited forecast mode—or most likely outcome of —and the TPN distributional assumption. YY The magnitude of the ex-ante variance that the forecaster elicits, |F �, reflects his vvaarr�YYtt+h tt level of knowledge and confidence in the likelihood around central expected outcomes. The ex-ante skewness of the predicted variable, |F �, summarizes the forecaster’s knowledge γγ�YYtt+h tt and beliefs that the most likely outcome would materialize on one or the other side of the distribution.

There are two complementary angles to the usefulness of eliciting dispersion and skewness measures: technical and policy-oriented. On the technical side, the assessment of dispersion and skewness statistics is central to accurately quantify the upside and downside uncertainties of the most likely outcome of the predicted variable . The technical side also YY allows the evaluation of uncertainty in recursive form and the testing of the predictive power of the elicited density forecasts. From the policy analysis angle, the predictive dispersion and skewness statistics provide the forecaster’s sentiment around the most likely outcome. In policy analysis, these two statistics offer a fundamental understanding of the future behavior of , YY complementing the decision-making process under uncertainty.

> 9 Even though the study variable is assumed to follow a TPN distribution, its density forecast cannot be YY estimated via maximum likelihood as we only collect few elicited central-measure predictions of . YY

11
# 2.4 PRIOR INFORMATION

The econometric steps for estimating Equations (2) and (3) deal with two main restrictions. First, the ex-ante variance and ex-ante skewness of the predicted variable are unobserved or uncertain. Second, there might be empirical evidence, previous literature, or beliefs affecting the sign of the coefficients in Equations (2) and (3). To deal with these two significant constraints, the BCEF procedure solves a minimization information-theoretical setup where prior information is integrated into the setting of the problem. In specific, our BCEF method follows a minimum cross-entropy setting, as presented in Golan, Judge, and Miller (1996), and Judge and Mittelhammer (2011).

The BCEF procedure incorporates prior information related to and . The use of prior ZZ YY information is an implication of the principles that one should fully specify to make probabilistic statements of what is unknown (Geweke and Whiteman 2006). The subjective probability distribution that a decision-maker assesses upon a variable should embody a synthesis of all his or her information relevant to its outcome (Bunn 1984, chapter seven). In this regard, we incorporate prior sampling information of the underlying phenomena affecting as well as the YY perceptions of the forecaster. Note that this personal information about the distribution of a phenomenon can be symmetric or skewed, although in any case, the prior assessment should always be coherent (De Finetti 1937).

The BCEF method needs three steps to set prior information. Each of these three steps corresponds to three classes of unobserved parameters, which are components of Equations(2)– (3). The three categories of unobserved parameters are risk factor coefficients, error terms of the variance and skewness equations, and the predicted variance and skewness statistics. Furthermore, we assume that all the unobserved parameters behave in linear form including additive error terms for all periods and horizons h∈H as presented in Equations (6)– tt∈TT (10).

12
<!-- Start of picture text --> var(Y ) = var (Y ) +€&<br>y(¥ jay(¥ +E<br>p={a —svarl¥ —).r(¥ YY<br>{ (Yo jx(Y Df<br>{x he() (ale ale]<br>(em) (le ete}<br>={a<br><!-- End of picture text -->
<!-- Start of picture text --> (—|a |,co ) E(-|a |, )<br>{e } {e }<br>E(o )<br>{ i {y (¥ —)}<br>mE) x )<br>={v<br>= {2k }<br><!-- End of picture text -->
<!-- Start of picture text -->
-(:<br>=<br><!-- End of picture text -->

<!-- Start of picture text --> m(E)<br>n(E )<br>(x(E I<br>n(é))<br>(x(E ) a(E))<br>r(E)>a($ )<br><!-- End of picture text -->
{2 EA( -)m( {zz ~~(=~~ )
to the realization, but that fail to foresee the precise economic growth of a country. In recent years, evidence shows a wide gap in inaccuracies of a variety of forecasters in predicting the economic growth of several countries (Figure 1).

<!-- Start of picture text --> Panel A. Concurrent-Year Forecast Horizon Panel B. Next-Year Forecast Horizon<br>18 18<br>12 12<br>6 6<br>0 0<br>-6 -6<br>Releasing period (month) Releasing period (month)<br>90th percentile Median 10th percentile<br>GDP growth HFE GDP growth HFE<br>(percentage points) (percentage points)<br>Jan 2005 Jul 2006 Jan 2008 Jul 2009 Jan 2011 Jul 2012 Jan 2014 Jul 2015 Jan 2005 Jan 2007 Jan 2009 Jan 2011 Jan 2013 Jan 2015<br><!-- End of picture text --> FIGURE 1. HISTORICAL FORECAST ERRORS OF REAL GDP GROWTH RATES FOR A VARIETY OF COUNTRIES Source: Consensus Economics, Global Economic Prospects, and World Development Indicators. Note: Monthly observations over January 2005–August 2015 period. Forty-one economies with real GDP growth forecasts are considered: Argentina, Australia, Bulgaria, Brazil, Canada, Switzerland, Chile, China, Colombia, Czech Republic, Germany, Euro area, Spain, Estonia, France, the United Kingdom, Hong Kong SAR-China, Croatia, Indonesia, India, Italy, Japan, the Republic of Korea, Lithuania, Latvia, Mexico, Malaysia, Netherlands, Norway, New Zealand, Peru, Poland, Romania, the Russian Federation, Singapore, Slovak Republic, Sweden, Thailand, Turkey, the United States, and Venezuela. Historical forecast error is defined as predicted minus realized value. A positive HFE value indicates overprediction. The next-year horizon historical forecast errors (Panel B) shows that the country growth point forecasts made during January 2007–December 2008 were mostly The most likely outcome—mode—of world GDP growth is taken from predictions made in a structural model that includes inputs from experts on country economic growth (World Bank 2016). However, note that the looking-forward conditional variance and skewness of world GDP growth are assumed not observed. The ex-ante information on GDP growth and its risk factors uses forward-looking information covering the scope of annual predictions spanning the period of 2005–2017. The BCEF method recovers predictive densities of global growth over the full monthly period of October 2005–August 2015 with three annual horizons: 17
the current year, next year, and two-year ahead horizons. For instance, the density forecasts estimated in August 2015 predicts world GDP growth for 2015, 2016, and 2017.

# 3.1 RISK FACTORS AND PRIORS

We select a subset of risk factors, **,** directly affecting world GDP growth. Four random ZZ variables help us to account for substantial risks of world GDP growth. These variables account for absolute deviations from consumer price inflation targets, global energy prices proxied by crude oil prices, global financial conditions quantified by the S&P 500 index price, and world term spread conditions measured by country-individual term spreads.

The density predictions for GDP growth are constructed reflecting upon basic economic mechanisms between output growth and underlying risks. The linear relationship of GDP growth, , and its associated risk factors, , imply a constrained behavior of the variance and YY ZZ skewness as discussed in Proposition 1 and 2. In this application, the assumed responses of global growth to the associated risk factor movements are supported using previous studies.<sup>15</sup> The sign restrictions on the responses of World GDP growth to the changes of its global risk factors are presented in Table 1. The sign of the risk-coefficients in the linear Equation (1) defines the reactions of the skewness to predicted global growth, as seen in Equation (3). In this regard, the global growth skewness is marginally synchronized with the directional change of an associated risk factor.

TABLE 1 − SIGN RESTRICTIONS OF THE RESPONSES OF WORLD GDP TO MOVEMENTS OF GLOBAL CONDITIONS

||World Term|Spreads<br>World (CPI) Infation:<br>Absolute Deviations|World Energy Prices|S&P 500 Index Prices|
|---|---|---|---|---|
|||from Targets|||
|World GDPgrowth|(+)|(-)|(-)|(+)|

> 15 Apart from the literature revision, we also estimated variety of econometric representations for ex-post world GDP growth using OLS and robust White-Huber standard errors. These OLS estimations obtain coefficients of the risk factors with the same signs as the ones presented in Table 1. The OLS estimates are available upon request.

18
The general financial conditions affecting global growth are represented using cross-country term spreads and the S&P 500 index prices. High-interest rate term spreads are assumed positively linked with output growth. This is in line with the following authors: Harvey (1989), Estrella and Hardouvelis (1991), Haubrich and Dombrosky (1996), Stock and Watson (2003), Wheelock and Wohar (2009), Aretz and Peel (2010), and Kao, Kuan, and Chen (2013), among others. The stock market performance measured by the S&P 500 index prices—and other equity indexes—is assumed to be positively associated with anticipated output growth; as per Mitchell and Burns (1938), Fama (1981), Fischer and Merton (1984), Harvey (1989), Jay Choi, Hauser, and Kopecky (1999), Hassapis and Kalyvitis (2002), and Tsouma (2009), among others.

Energy sector performance—measured via crude oil and gas prices—is assumed to be negatively associated with output growth. This association is in line with Hamilton (2005), Jimenez-Rodriguez and Sánchez (2005), Blanchard and Gali (2007), Blanchard and Riggi (2013), among others. Absolute deviations of CPI inflation targets are negatively associated with GDP growth performance. In this empirical application, the assumed benchmark inflation target is fixed at two percent across the study countries.

Based on data origination, the four selected risk factors affecting world GDP growth are divided into two categories: i) survey-based; and ii) market-based. The survey-based risk factors are a) the term spread, and b) the CPI inflation. The survey-based observations are collected by Consensus Economics. The market-based risk factor information is collected using Bloomberg. implied volatilities of option prices. This information is put together for the S&P 500 index, and the West Texas Intermediate (WTI) and Brent crude oils. As previously mentioned, elicited most likely forecasts of world GDP growth are obtained from World Bank (2016).

19
<!-- Start of picture text --> Current year horizon Median: Current year horizon Current year horizon Median: Current year horizon<br>Next-year horizon Median: Next-year horizon Next-year horizon Median: Next-year horizon<br>8 1<br>6 .5<br>4 0<br>2 -.5<br>0 -1<br>Panel C. Ex-Ante Variance of Term Spread Panel D. Ex-Ante Skewness of Term Spread<br>Current year horizon Median: Current year horizon Current year horizon Median: Current year horizon<br>Next-year horizon Median: Next-year horizon Next-year horizon Median: Next-year horizon<br>8 1<br>6 .5<br>4 0<br>2 -.5<br>0 -1<br>Oct 2005 Jan 2007 Apr 2008 Jul 2009 Oct 2010 Jan 2012 Apr 2013 Jul 2014 Oct 2015 Oct 2005 Jan 2007 Apr 2008 Jul 2009 Oct 2010 Jan 2012 Apr 2013 Jul 2014 Oct 2015<br>Oct 2005 Jan 2007 Apr 2008 Jul 2009 Oct 2010 Jan 2012 Apr 2013 Jul 2014 Oct 2015 Oct 2005 Jan 2007 Apr 2008 Jul 2009 Oct 2010 Jan 2012 Apr 2013 Jul 2014 Oct 2015<br><!-- End of picture text --> FIGURE 2. FORWARD-LOOKING CONDITIONAL VARIANCE AND SKEWNESS OF TERM SPREAD AND ABSOLUTE DEVIATIONS FROM INFLATION TARGETS Source: Consensus Forecasts and author’s calculations.

Note: The global term spread statistics are constructed using fifteen countries: 1) Australia; 2) Canada; 3) France; 4) Germany; 5) India; 6) Indonesia; 7) Italy; 8) Japan; 9) the Republic of Korea; 10) the Netherlands; 11) Spain; 12) Sweden; 13) Switzerland; 14) the United Kingdom; and 15) the United States. The global inflation statistics are derived using the twenty most significant economies in terms of GDP size: 1) Australia; 2) Brazil; 3) Canada; 4) China; 5) France; 6) Germany; 7) India; 8) Indonesia; 9) Italy; 10) Japan; 11) the Republic of Korea; 12) Mexico; 13) the Netherlands; 14) the Russian Federation; 15) Spain; 16) Sweden; 17) Switzerland; 18) Turkey; 19) the United Kingdom; and 20) the United States. The global variance series are scaled down using the sample standard deviation of the original series. The global skewness series is scaled down using the maximum value of the corresponding original series.

Ex-ante variance and skewness information of the term spread and absolute deviations of inflation targets from a variety of countries are used to proxy the corresponding global measures (Figure 2). Implied volatilities of options contracts are the source of information for estimating the ex-ante variance and ex-ante skewness of the S&P 500 and crude oil prices (Figure 3).

20
The selected implied volatilities come from option contracts with 3, 12, and 24-month maturities to represent the uncertainty for the current year, next year, and two-year ahead horizons. The procedure for recovering the variance and skewness of the equity and commodity prices from implied volatility series—assuming the Black-Scholes-Merton option pricing model—is discussed in Appendix A6.

Panel A. Ex-Ante Variance of Crude Oil Prices Panel B. Ex-Ante Skewness of Crude Oil Prices <!-- Start of picture text --> 3-month horizon Median: 3-month horizon 3-month horizon Median: 3-month horizon<br>12-month horizon Median: 12-month horizon 12-month horizon Median: 12-month horizon<br>24-month horizon Median: 24-month horizon 24-month horizon Median: 24-month horizon<br>5 1<br>4 .8<br>3 .6<br>2 .4<br>1 .2<br>0 0<br>Panel C. Ex-Ante Variance of the S&P 500 Index Price Panel D. Ex-Ante Skewness of the S&P 500 Index Price<br>3-month horizon Median: 3-month horizon 3-month horizon Median: 3-month horizon<br>12-month horizon Median: 12-month horizon 12-month horizon Median: 12-month horizon<br>24-month horizon Median: 24-month horizon 24-month horizon Median: 24-month horizon<br>4 1<br>3 .8<br>.6<br>2<br>.4<br>1<br>.2<br>0<br>FIGURE 3. EX-ANTE VARIANCE AND SKEWNESS OF CRUDE OIL AND S&P 500 INDEX PRICES<br>Oct 2005 Jan 2007 Apr 2008 Jul 2009 Oct 2010 Jan 2012 Apr 2013 Jul 2014 Oct 2015 Oct 2005 Jan 2007 Apr 2008 Jul 2009 Oct 2010 Jan 2012 Apr 2013 Jul 2014 Oct 2015<br>Oct 2005 Jan 2007 Apr 2008 Jul 2009 Oct 2010 Jan 2012 Apr 2013 Jul 2014 Oct 2015 Oct 2005 Jan 2007 Apr 2008 Jul 2009 Oct 2010 Jan 2012 Apr 2013 Jul 2014 Oct 2015<br><!-- End of picture text --> Source: Bloomberg and author’s calculations.

Note: Ex-ante variance and skewness of crude oil and the S&P 500 index prices are estimated via implied volatilities of option contract prices. Bloomberg data uses a variation of the option pricing model of BlackScholes-Merton. The option contracts series are at the money. The variance series is scaled down using the sample standard deviation of the original series. The skewness series is scaled down using the maximum value of the corresponding original series.

21
A challenging step of the BCEF method involves the identification of prior series for the unobserved ex-ante variance and ex-ante skewness of the world GDP growth, . In this regard, YY we modeled the prior ex-ante variance and ex-ante skewness of world GDP growth as pointed out in the Appendix A3<sup>16</sup> using the sign restrictions presented in Table 1 and the available forward-looking variance and skewness series of the risk factors.

Furthermore, based on point forecasts of world GDP growth (World Bank 2016), Table 2 shows that there is generally an overprediction pattern in the elicited predictions. Thus, opposite to the overpredicted forecasts, the initial guess for the prior global ex-ante skewness is to assign more weight to the downside section of the world GDP growth predicted distribution; i.e., to assign negative skewness values.

TABLE 2 – HISTORICAL FORECAST ERRORS OF WORLD GDP GROWTH PREDICTIONS

|Horizons:|Concurrent Year<br>(T)|T+1|T+2|Combined Horizons|
|---|---|---|---|---|
|Mean|0.22|0.98|2.31|0.82|
|Median|0.16|0.73|1.49|0.51|
|Std. Dev.|1.20|1.85|1.78|1.66|
|Observations|13.00|11.00|4.00|28.00|

Source: Global Economic Prospects (World Bank 2016).

Note: Realizations of world GDP growth based on WDI. Historical forecast error defined as predicted minus realized value. A positive HFE value indicates overprediction.

In this specific world GDP growth forecast, the mechanics of optimistic baseline forecasts assessed with more significant downside than upside risks are a summary and a statistical translation of the well-known phrase “hope for the best and prepare for the worst.” Finally, once priors for the alpha coefficients and the future variance and skewness of real GDP are selected, the posterior distributions that solve the cross-entropy optimization model presented in Subsection 2.5 are ready to be estimated.

> 16 There are multiple methods to obtain priors (Golan 2017). The procedure presented in Appendix A3 to proxy the prior distributions of the study parameters is only an alternative approximation.

22
# 3.2 RESULTS

The main results of the application to calculate the density forecasts for world GDP growth based on the BCEF method are summarized in Table 3 and Figure 4. Table 3 shows the prior and posterior coefficients that solve the BCEF problem stated in Equation (16). Our priors aarr,h of the coefficients at the concurrent period, h= 0 weight more  in absolute terms  the aarr,h crude oil prices and the world term-spreads, while the measure of inflation target deviations and the S&P 500 index are the risk factors with coefficients with smaller magnitudes. At horizons h= 1 and h= 2, these prior values for the coefficients are more homogeneous. aarr The posterior estimates of are quite heterogenous between risk factors and across time aarr horizons (Table 3).

TABLE 3 – POSTERIOR AND PRIOR AVERAGE DIRECT RESPONSES OF WORLD GDP TO MOVEMENTS OF GLOBAL RISKS

|arCoefcients|Horizon|World Term Spread|World (CPI)<br>Infation Absolute<br>Deviations from<br>Target|World Energy<br>(Crude Oil) Prices|S&P 500 Index|
|---|---|---|---|---|---|
|Posterior|h= 0|0.35|-0.29|-0.51|0.30|
|Posterior|h= 1|0.43|-0.46|-0.50|0.47|
|Posterior|h= 2|0.36|-0.36|-0.53|0.53|
|Prior|h= 0|0.51|-0.41|-0.69|0.24|
|Prior|h= 1|0.51|-0.55|-0.63|0.62|
|Prior|h= 2|0.54|-0.53|-0.66|0.69|

Note: These coefficients are used in Equations (2), and (3) to shape the parametric TPN predictive distribution of world GDP growth.

These posterior estimates of the risk factor coefficients feed into Equations (2) and aarr (3) to recover the average series of ex-ante variance and skewness of GDP growth for three forecasting horizons. As seen in Panel A of Figure 4, the estimated ex-ante variance of world GDP growth has peaks around June 2008 and April 2012 across all the three studied time horizons. The series representing the ex-ante skewness of world GDP growth is in general negative, highlighting the bias of the density forecasts to the downside section of the distribution. These ex-ante skewness series vary substantially between horizons.

23
<!-- Start of picture text --> — Current year ¢ ¢ Next-year —~Two-year ahead ——Median (3 horizons)<br>3.5<br>233<br>( , q<br>I — iN eo oh 7 \ Pe NS e~<br>0.5 re, ——~ RAL &<br>Oo +++ +++" t+" +--+" + +H.<br>SSSSESRBSSESEg2LeErEggIeaarsee<br>FHSSSHSSHSSHFGFGISISSSSRSH<br>SESDEDEDERDEDEDEDECEDE<br>6=6#=6=6276 2756756 < 6X6 HGH<br><!-- End of picture text --> <!-- Start of picture text --> — Current year +++ Next-year —=Two-year ahead ——Median (3 horizons)<br>0.05<br>0 , 7 vay Waa! EYQ aATATAT<br>WMA KNa, °| t weyDal isYEatte aeoe °<br>02 cs * oy<br>-0.25 7 ;<br>-03<br>SSSSssSsssSSSrsgeesrizes<br>FUSPTISFSISPFISGSRSISGas<br>CEGEDERCERDERDECEGEDEDE<br>oo oo Moi Moi ol MoM ole<br><!-- End of picture text -->
<!-- Start of picture text --> 6.0 percentage, yoy 6.0<br>5.0 mm40% mm75% mm@90% Baseline 50<br>4.0 4.0<br>3.0 30°<br>2.0 2.0<br>L.0 1.0<br>0.0 0.0<br>-1.0 -1.0<br>-2.0 -2.0<br>8 8 € SF 8 B SF 2B FS 8B<br>a a a a a a a a a a<br><!-- End of picture text --> <!-- Start of picture text --> 6.0 percentage, yoy 6.0<br>50 mm40% mm75% mm90% Baseline 5.0<br>4.0 4.0<br>39 3.0<br>2.0 2.0<br>1.0 1.0<br>0.0 0.0<br>-1.0 -1.0<br>-2.0 “2.0<br>2 z S a Q 3 x 2 2 ><br>RX ®S R FR HR FR KR FR AR AR<br><!-- End of picture text --> <u>yZ_)</u> (Y )
<!-- Start of picture text --> 7.0 percentage, yoy 7.0<br>6.0 mm40% mm75% mm90% Baseline 6.0<br>5.0 5.0<br>4.0 4.0<br>3.0 3.0<br>2.0 2.0<br>1.0 1.0<br>0.0 | : : 1 1 ' 0.0<br>-1.0 -1.0<br>-2.0 -2.0<br>2 2 £¢ = gg £2 ¢ £8 ¥g 8<br>a a a A R R Ri qi R R<br><!-- End of picture text -->

<!-- Start of picture text --> 7.0 percentage, yoy 7.0<br>6.0 mm40% mm75% mm90% Baseline 6.0<br>5.0 5.0<br>4.0 4.0<br>3.0 3.0<br>2.0 2.0<br>1.0 1.0<br>0.0 | : : 1 ' 0.0<br> -1.0 -1.0<br> -2.0 -2.0<br>2 2 £¢ = gg £2 ¢ £8 ¥g 8<br>a a a A R R Ri qi R R<br><!-- End of picture text -->
TABLE 5 – CONTINUOUS RANKED PROBABILITY SCORE OF DENSITY FORECASTS

||Average for<br>TPN Density Forecasts<br>(BCEF method)|Weighted Average for TPN<br>Density Forecasts<br>(BCEF method)<br>Average for<br>Normal Density Forecasts<br>(Varying HFE Standard<br>|<br> <br>Weighted Average for<br>Normal Density Forecasts<br>(Varying HFE Standard<br>|
|---|---|---|---|
|||<br>Deviation Across Horizons)|<br>Deviation Across Horizons)|
|Panel A. Monthly Averag<br>|e Forecast Scores Across the<br>|Period of 2005–2015 and Three Annual Time-Horizons<br> <br>|h={0,1,2}.<br>|
|January|1.6|0.55<br>1.79|0.62|
|February|1.55|0.58<br>1.79|0.67|
|March|1.51|0.61<br>1.79|0.72|
|April|1.55|0.66<br>1.79|0.77|
|May|1.51|0.69<br>1.79|0.82|
|June|1.45|0.7<br>1.76|0.85|
|July|1.44|0.73<br>1.76|0.89|
|August|1.47|0.79<br>1.76|0.94|
|September|1.46|0.82<br>1.79|1|
|October|1.46|0.86<br>1.76|1.03|
|November|1.47|0.9<br>1.76|1.08|
|December|1.45|0.93<br>1.76|1.12|
|<br>Panel B. Annual Average|<br>Forecast Scores Across Mont|<br> <br>hly Observations Over 2005–2015 and Three Annual Ti|<br>me-Horizonsh={0,1,2}.|
|2005|1.04|0.62<br>1.5|0.9|
|2006|1.26|0.61<br>1.6|0.78|
|2007|2.6|0.92<br>2.65|0.99|
|2008|2.69|1.43<br>2.82|1.46|
|2009|1.6|1.03<br>1.97|1.15|
|2010|1.27|0.63<br>1.55|0.77|
|2011|1.23|0.56<br>1.47|0.68|
|2012|1.07|0.49<br>1.38|0.65|
|2013|0.89|0.4<br>1.32|0.62|
|2014|0.93|0.59<br>1.29|0.83|
|2015|0.72|0.57<br>0.97|0.77|
|Panel C. Horizon Averag|e Forecast Scores Across Mon|thly Observations Over 2005–2015.||
|h= 0|1.15|0.97<br>1.34|1.14|
|h= 1|1.73|0.89<br>2.07|1.07|
|h= 2|1.65|0.30<br>1.95|0.36|
|Panel D. Average Foreca|st Scores Across Monthly Obs|ervations Over 2005–2015 and Three Annual Time-Ho|rizonsh={0,1,2}.|
|Aggregated Score|490.90|243.15<br>583.41|289.45|

Note: The weighted scores use masses derived following the rule presented in Equation (32). The weights penalize the forecasts under the following principle: when a forecast is elicited, it is expected that it has better accuracy if it is produced closer to the realization period than when it is released with a further horizon. In this sense, the closer to the realization period, the more punished (weighted) the elicited forecast. HFE stands for historical forecast error. The monthly estimated density forecasts cover the period of October 2005–August 2015. Forecasts generated in 2014 are evaluated only with the realizations of 2014 and 2015. Predictions made in 2015 are scored using realizations of 2015. Scores of naïve density forecasts generated using the rule of thumb of a fixed HFE standard deviation across the three forecasting horizons (results are available upon request) were estimated and not presented in this table. These latter scoring results remained the same as the ones shown in this table: BCEF density forecasts outperform naïve density predictions.

The CRPS scores overwhelmingly show more accurate density forecasts—lower score values—for the BCEF density forecasts than for the naïve predictions. The unweighted and weighted CRPS results are robust over four views: across horizons, months, years, and in the aggregated form. In this regard, the CRPS scores also indicate that the BCEF fan charts are 27
more reliable to measure the uncertainty around the studied world GDP growth baseline than the naïve  normal distributed  density forecasts. The takeaway point here is that the BCEF forecasts at least overperform, in terms of prediction accuracy, the simple, inexpensive, and time-saving symmetric normal distributed density forecasts. Although in some settings, this will be a minimal consideration, the CRPS scores show that forecast accuracy and reliability are incentives to invest in a method like the BCEF presented in this paper.

# 4. CONCLUDING REMARKS

Informed decisions require to consider potential upside and downside outcomes. For policymaking and investment purposes, predictive models that consider the modeling of asymmetric density forecasts provide superior reliability and decision-advantage. As per the procedure presented in this paper, the generation of forecasting procedures that allow the incorporation and update of prior information are encouraged to obtain more flexible models to enhance the reliability of elicited density forecasts. Revision of established prequential systems is essential in practice to improve forecast scores. The proper scoring rule framework is also encouraged to track the reliability and transparency of the predictions.

In this paper, we introduce a minimum cross-entropy procedure to generate asymmetric density forecasts in systematic form. The methodology relies on prior information of parameters and unobserved statistics, expectations of risk factors statistics, and the two-piece normal distribution assumption. Despite its constraints, the TPN distribution provides some flexibility to assess the upside and downside sections of the predictive density forecasts. The BCEF method allows the selected risk factors to depict the shape of the density forecasts, i.e., the description of the uncertainty around the most likely outcomes of the study variable.

The introduced BCEF method models a variance and skewness decomposition of the predictive distributions. The combination of the study variable and selected risk factors offers a simple linear regression framework to exploit ex-ante information regarding predictive variance and skewness. This ex-ante information is used to predict a confidence interval around 28
the most likely outcome of a study variable. By construction, the BCEF procedure gives optimal results in the sense that it minimizes information losses when moving from the prior distribution assumptions to the posterior distribution estimates subject to consistency and behavioral constraints, and the data.

In the empirical application of predicting world GDP growth, the associated ex-ante market information emerges from term spreads, absolute deviations from inflation targets, energy prices, and the S&P 500 index prices. The recovered posterior series of the global GDP growth variance and skewness are used to construct hypothetical BCEF density forecasts. These BCEF world GDP growth forecasts are monthly-elicited spanning the period of October 2005–August 2015 and are evaluated using the continuous-ranked probability score. The unweighted and weighted average CRPS scores show that the Bayesian entropy fan charts outperform symmetric normal distributed  naïve  density forecasts, which are generated only using historical forecast error information. Noticeably, the estimated monthly BCEF-generated density forecasts for world GDP growth show predominantly negative values of skewness— downside skewed density forecasts—across the period of October 2005–August 2015. These negative skewness spotlight two aspects of the model: its easiness to incorporate prior information such as historical overprediction of world GDP growth, and its ability to minimize noise under the relative entropy setup.

Implementing proper scores—like the continuous-ranked probability score—provides strong incentives for the forecaster to elicit his or her best and honest density predictions. Evaluation of density forecasts through proper scoring rules provides transparency and the opportunity to distinguish the heterogeneity of forecasting model accuracy and forecaster skill. The recording of elicited predictions and their scoring rule evaluations will allow the comparison of forecasters and methods that aim to improve better investment and policymaking decisions.

# 5. REFERENCES

> Al-Najjar, N. I., and E. Shmaya. 2015. "Uncertainty and Disagreement in Equilibrium Models." _Journal of Political Economy_ 123 (4): 778-808.

29
<!-- Start of picture text --> Bessler, D. A., and J. L. Kling. 1990. "Prequential Analysis of Cattle Prices "<br>Bickel, J. E. 2007. "Some Comparisons among Quadratic, Spherical, and Logarithmic Scoring Rules."<br>": National Bureau of Economic Research.<br>Interpretation of Changes in the Macroeconomic Effects of Oil Prices."<br>Brier, G. W. 1950. "Verification of Forecasts Expressed in Terms of Probability."<br>Brown, T. A. 1974. "Admissible Scoring Systems for Continuous Distributions."<br>Clark, T. E., M. W. McCracken, and E. Mertens. 2018. "Modeling Time-Varying Uncertainty of Multiple-<br>----------, 1984. "Present Position and Potential Developments: Some Personal Views: Statistical Theory:<br>The Prequential Approach."<br>De Finetti, B. 1937.<br>Diebold, F. X., J. Hahn, and A. S. Tay. 1999. "Multivariate Density Forecast Evaluation and Calibration in<br>Estrella, A., and G. A. Hardouvelis. 1991. "The Term Structure as a Predictor of Real Economic Activity."<br>Fama, E. F. 1981. "Stock Returns, Real Activity, Inflation, and Money."<br>Geweke, J., and C. Whiteman. 2006. "Bayesian Forecasting."<br>Giordani, P., and P. Sdderlind. 2003. "Inflation Forecast Uncertainty."<br>Gneiting, T. 2011. "Quantiles as Optimal Point Forecasts."<br>Gneiting, T., F. Balabdaoui, and A. E. Raftery. 2007. "Probabilistic Forecasts, Calibration and Sharpness."<br>Gneiting, T., and A. E. Raftery. 2007. "Strictly Proper Scoring Rules, Prediction, and Estimation."<br>Gneiting, T., A. E. Raftery, A. H. Westveld III, and T. Goldman. 2005. "Calibrated Probabilistic Forecasting<br>using Ensemble Model Output Statistics and Minimum CRPS Estimation."<br>Gneiting, T., and T. L. Thorarinsdottir. 2010. "Predicting Inflation: Professional Experts versus No-Change<br><!-- End of picture text -->
<!-- Start of picture text --> Go, D. S., H. Lofgren, F. Mendez-Ramos, and S. Robinson. 2016. "Estimating Parameters and Structural<br>Change in CGE Models using a Bayesian Cross-Entropy Estimation Approach."<br>Golan, A., G. Judge, and S. Robinson. 1994. "Recovering Information from Incomplete or Partial<br>Multisectoral Economic Data."<br>Granger, C. W. J. 1986. "Forecasting Accuracy of Alternative Techniques: A Comparison of U.S.<br>Grimit, E. P., T. Gneiting, V. Berrocal, and N. A. Johnson. 2006. "The Continuous Ranked Probability Score<br>for Circular Variables and its Application to Mesoscale Forecast Ensemble Verification."<br>Hassapis, C., and S. Kalyvitis. 2002. "Investigating the Links Between Growth and Real Stock Price Changes<br>Haubrich, J. G., and A. M. Dombrosky. 1996. "Predicting Real Growth using the Yield Curve."<br>Hersbach, H. 2000. "Decomposition of the Continuous Ranked Probability Score for Ensemble Prediction<br>Jay Choi, J., S. Hauser, and K. J. Kopecky. 1999. "Does the stock market predict real activity? Time series<br>Johnstone, D. 2007. "Economic Darwinism: Who has the Best Probabilities?"<br>----------, 2007b. "The Value of a Probability Forecast from Portfolio Theory."<br>Johnstone, D., V. R. R. Jose, and R. L. Winkler. 2011. "Tailored Scoring Rules for Probabilities."<br>Kao, Y.-C., C.-M. Kuan, and S. Chen. 2013. "Testing the Predictive Power of the Term Structure Without Data<br>Kling, J. L. 1987. "Predicting the Turning Points of Business and Economic Time Series."<br>Kling, J. L., and D. A. Bessler. 1989. "Calibration-Based Predictive Distributions: An Application of<br>Prequential Analysis to Interest Rates, Money, Prices, and Output."<br>Kramer, W. 2006. "Evaluating Probability Forecasts in Terms of Refinement and Strictly Proper Scoring<br><!-- End of picture text -->
Lichtenstein, S., B. Fischhoff, and L. D. Phillips. 1982. "Calibration of Probabilities: The State of the Art to Matheson, J. E., and R. L. Winkler. 1976. "Scoring Rules for Continuous Probability Distributions." Mitchell, W. C., and A. F. Burns. 1938. "Statistical Indicators of Cyclical Revivals." In Nelson, R. G., and D. A. Bessler. 1989. "Subjective Probabilities and Scoring Rules: Experimental Evidence." Rich, R., and J. Tracy. 2010. "The Relationships Among Expected Inflation, Disagreement, and Uncertainty: Sanders, F. 1963. "On Subjective Probability Forecasting." Savage, L. J. 1971. "Elicitation of Personal Probabilities and Expectations " Stock, J. H., and M. W. Watson. 2003. "Forecasting Output and Inflation: The Role of Asset Prices." Tsouma, E. 2009. "Stock Returns and Economic Activity in Mature and Emerging Markets." Wallis, K. F. 2014. "The Two-Piece Normal, Binormal, or Double Gaussian Distribution: Its Origin and Winkler, R. L. 1996. "Scoring Rules and the Evaluation of Probabilities." Winkler, R. L., and A. H. Murphy. 1970. "Nonlinear Utility and the Probability Score." var(Y ) = var(a )
<!-- Start of picture text --> =E|(e ) |-€ [le ) |<br>“=| ) |<br>-[Ee ]#2d Elz ele 4D<br>ez |elz<br>E|Z | IZ<br>E|Z | FIZ JE[Z—<br>+Ele |-E | |<br>E|Z E|Z ] Ele |<br>E|Z | [Z|<br>E|Z | E[Z jelzZ |<br>tle | [| [Z E[Z Jele |<br>var|Z | cov|Z | + var[e |<br>cov|Z |<br>cov|Z | tvarle ]+2> cov|Z |<br>y(aX) = E|(aX )<br>[a x|<br>[EX X|<br><!-- End of picture text -->
<!-- Start of picture text --> var(Y )<br>ry )=el(y -E(Y )) |<br>=E[(¢ ) |<br>~3E [(e 1 (( ) )<br>Wo<br>El (e ) | ele | Ez |<br>+3E|e | E|Z |<br>3E|e | E|Z |<br>=Ele E/Z |<br>E|Z |eEle |<br>E|Z |-Ele |<br>3 E|(¢ ) | (( ) )<br>=3(Ele J+y EZ J)<br>(x var|Z | cov|Z |<br>+var |e |+2> cov|Z 1)<br>=3E|e |x var|Z |<br><!-- End of picture text -->
<!-- Start of picture text --> +3 Ele |x cov|Z |<br>+3 Ele |var[e |<br>+3026 [ |> co Z |<br>E[Z - [Z|<br>E|Z | cov|Z |<br>EZ | evarle<br>EZ | cov|Z |<br>=3-Ele |> eva |Z |<br>+3E|e |> cov|Z |<br>+3E|e |° r |<br>+6+E| | v|Z |<br>| ea eva Z |<br>¢cov[Z 4 |<br>Jeo fe |<br>°E|Z | + cov[Z |<br>=3-Ele |> eva |Z |<br>+3 Ele |> cov|Z |<br>+3E|e |° r |<br>+6°E| | co [Z |<br>[Z ea ¢var|Z |<br>¢cov[Z ]-E(Z |<br>Jsvar[ |<br>eE[Z = | +cov|Z |<br>=3-Ele |> °va [Z |<br>+3 Ele |> cov|Z |<br>+3E|e |° r |<br>+6°E| | cov|Z |<br>E|Z ]ea evar |Z |<br>E|Z ]ea evar |Z |<br>¢cov[Z ]-E(Z |<br><!-- End of picture text -->
<!-- Start of picture text -->
[ — fevart |<br>eE[Z | +cov|Z |<br>fe yo] Ele [+elz !)<br>[|<br>[x |<br>lele JEJeE [ Dd |<br>le |<br>IZ |<br>le |*2 IZ |<br>le Jeo Z|<br>yje | -3+Ele |e> cov|Z |<br>-—6¢E | x *cov|Z |<br>E|Z | a ¢var|Z |<br>¢cov[Z ]-E(Z |<br>eE[Z | cov|Z |<br>(var (YY (YDS<br><!-- End of picture text -->
<!-- Start of picture text --> ={@ }<br>~ (Y — )} {7 (¥ —)}<br>(Yo )=X (__ ) var(Z )<br>(YY )=X ( )x{Z )<br>{oar (Y —) {7 (Y¥ —)}<br>(Y D(var (y oa (y —))<br>tcc yf EER Vr D)<br>(Y )=> ( ) var(Z<br>(¥ XY ( )r(Z )<br>(a ) var(Z )<b<br><!-- End of picture text -->
to evaluate the reliability of elicited forecasts. In this paper, the evaluation of elicited density forecasts is made via scoring rules. A scoring rule is a statistic that quantifies in an ex-post manner the accuracy of released forecasts. A scoring rule is defined as a numerical ranking based on the predictive probability distribution, and the materialized event. A scoring rule could be proper or improper. Proper scoring rules provide the incentive for the forecaster to elicit her best and accurate predictions. On this matter, the implementation of transparent scoring rule frameworks is encouraged to distinguish forecasting skill heterogeneity. Also, the sequential compilation of scoring rule outcomes will allow for future forecast enhancements.

The objective of the proper scoring rules framework is twofold: to evaluate the density forecasts and to incentivize the generation of better—probabilistic—predictions. In terms of evaluation, the scoring rule measures the quality of elicited density forecasts. It also rewards the forecaster for his or her probability predictions and ranks density forecast outcomes. From the perspective of the generation of forecasts, a scoring rule has two purposes: to encourage the forecaster to make careful assessments, and to ensure that the forecaster retains his or her honesty during the process, i.e., to ensure that the forecaster elicits his or her real and best density forecast.

The literature of assessments of probabilistic predictions comes from a long-standing tradition and with a wealth of examples. The pioneering work of Brier (1950) establishes the quadratic score to evaluate probabilistic forecasts using a raining example. Brier (1950) looks into the principle of the proper scoring rules to devise a verification scheme that cannot influence the forecaster in an undesirable way. Even using the proper quadratic score (Brier), Sanders (1963) studies various weather elements and provides evidence that skillful probability statements can indeed be formulated subjectively. Winkler and Murphy (1968) argue that a subjective probability statement quantifies the confidence of an individual in a correct and particular proposition, e.g., an economist’s confidence in the statement that “next-year inflation will be two percent.” The assessment of the elicitation of probabilities and the discussion of proper scoring rules are widely debated in Savage (1971), which concludes that scoring rules enable us, in principle, to discover people’s opinions.

38
As an alternative to proper scoring rules, the quality, evaluation, and improvement of density forecasts can be assessed through probabilistic forecast calibration procedures. The calibration method makes use of the cumulative probability integral transform to evaluate elicited density forecasts in an ex-post manner. Although calibration and scoring rules are complementary procedures in assessing the quality of forecasts, the scoring rule framework still allows the forecasters to be rewarded for their probability predictions and to directly rank forecasters or forecasting systems. Lichtenstein, Fischhoff, and Phillips (1982) summarize the literature on the calibration of point, and density forecasts up to the 1980s. Dawid (1982) posits a theorem to assess calibration and also discusses recalibration processes and coherence versus calibration. In a classic work, Bunn (1984) also explains the calibration and recalibration procedures of discrete predictive densities, as well as the evaluation of probabilistic forecasts through scoring rules. Examples of calibration and re-calibration assessments to improve univariate density forecasts are found in Kling (1987), Kling and Bessler (1989), Schervish (1989), Bessler and Kling (1990), etcetera. More recently, Diebold, Hahn, and Tay (1999) have extended the univariate calibration assessment to a multivariate framework using examples that involve highfrequency exchange rate density forecasts.

# A4.1 PROPER SCORING RULES

The reliability of the predicted density forecasts can be evaluated using proper scoring rules. The scoring rules allow the ranking of density forecast accuracy in assigning a numerical score. The score is based on two characteristics: a) the predictive distribution; and b) the event, state of nature, or numerical value that is realized. Proper scoring rules incentivize the forecaster to elicit his or her best predictive density forecasts and to be honest (Murphy and Epstein 1967). The continuous-ranked probability score (CRPS) evaluates predictive cumulative density functions as proper scoring rules (Gneiting and Raftery 2007). Properties of the proper scoring rules and the definitions of the CRPS are presented in the following paragraphs.

39
The actual forecaster’s predictive probability for the target random variable is denoted YY with and the forecaster’s reported predictive probability with **.** Both and can be pp uu pp uu probability density functions or probability mass distributions. The scoring rule is a function that assigns a score and maps into real numbers R. denotes the realization of the SS(pp, ss) ss random variable, . A scoring rule is a (weakly) proper scoring rule—in an ex-ante sense—if YY the forecaster’s expected score eliciting his real predictive probability—density or mass— function , , is smaller than his expected score eliciting any pp EEppSS(pp, YY) = ∫SS(pp, ss)pp(ss)mmss another probability function , , e.g., Equation (30) will hold for uu EEppSS(uu, YY) = ∫SS(uu, ss)pp(ss)mmss all probability functions and . Finally, the scoring rule is strictly proper when the equality pp uu holds if and only if almost surely (Gneiting 2011). pp= uu (30) EEppSS(pp, YY) ≤EEppSS(uu, YY).

If the forecaster prefers to use his true density function , instead of cheating and pp reporting any other function , the forecaster will be enforced by the proper scoring rule to uu set , i.e., the forecaster obtains his best-expected score for reporting his true predictive uu= pp probability. Besides that, the negative orientation of (30) concerning the forecast accuracy originates from a conventional change of minimizing the forecaster’s expected penalty to maximizing the forecaster’s expected score (Bunn 1984). Thus, the lower the score in (30), the better the forecaster is compensated. Hence, the desire to minimize the expected score from a strictly proper scoring rule motivates the forecaster to provide probabilities that are well-calibrated and sharp (Winkler 1986). Proper scoring rules motivate the forecasters to minimize their expected scores only by reporting individual probabilistic forecasts honestly. The use of proper scoring rules in forecast density evaluations minimizes the concerns that assessed subjective probabilities are, in some sense, arbitrary.

The objective of the scoring rule can also be interpreted in an ex-post sense. Assume that has sequential realizations, �TT 1, for a variety of horizons h∈H . Two different YY NNTT �sstt+h tt=tt prequential forecasting systems (PFSs) for collect the issued probability forecast sequences YY �<sup>pp̂</sup> �TT 1, and �TT 1, ∀h∈H . They can be compared using the scoring rule by tt+h tt=tt �uûtt+h tt=tt SS(∙) 40
1 estimating their average scores, pp̂ (∙) = ∑TT 1 � and sŝ (∙) = SS̅TT TT tt=tt SS�pp̂tt+h, sstt+h SS̅TT 1 ∑TT 1 � , ∀h∈H . Then, the subject evaluating the forecast sequences would TT tt=tt SS�uûtt+h, sstt+h prefer the PFS to the if and only if pp̂ sŝ , at any period ≥1 and for all pp ss (pp≽ss) SS̅TT ≤SS̅TT TT∈N horizon h∈H , and otherwise (Gneiting 2011). ss By definition, a proper scoring rule assumes that the forecaster is risk-neutral. The basic framework of the proper scoring rule assumes a linear utility reward for the agent (Nelson and Bessler 1989). Some rules, such as the quadratic, logarithmic, and spherical ones, are motivated by this risk-neutral assumption. Derivations of the proper scoring rules under the risk-neutral forecaster hypothesis can be seen in Winkler and Murphy (1968), Winkler (1996), Bunn (1984), Krämer (2006), Bickel (2007), Gneiting, Balabdaoui, and Raftery (2007), among others. Non-risk-neutral forecasters might have different implications on the forecaster’s incentives to elicit his or her real and honest probability predictions. Examples of risk-averse and risk-lover forecasters are found in Winkler and Murphy (1970), Winkler (1996), Bickel (2007), Johnstone (2007a), Johnstone (2007b), and Johnstone, Jose, and Winkler (2011).

A4.2 WEIGHTED AVERAGES As an alternative to using the arithmetic mean score, (∙) , the evaluation of density SS̅NNTT forecasts across time can be studied via the weighted average of individual scores, (∙), SS̅ss,TT ≥1. The weighted average score of the sequence of issued density forecasts �TT 1, ∀TT∈N �uûtt+h tt=tt is denoted by sŝ (∙) , ≥1 , h∈H and stated in Equation (31). While cycles and SS̅ss,TT ∀TT∈N forecast horizons are essential components in the forecasting exercises, weighted average scores incorporate these sequential behaviors providing a personalized mass for each score. (31) SS̅ss,TTsŝ �<sup>�uûtt+h�</sup> TTtt=tt1, �ssaa,tt+h�TTtt=tt1�<sup>= ∑</sup> TTtt=tt1 sstt+h SS�uûtt+h, ssaa,tt+h�, ∀TT∈N≥1, h∈H . The weights can be estimated considering some cyclicality or seasonality depending sstt+h on two primary considerations: i) the exact date when the forecast is released—seasonal component—; and ii) how many horizons ahead the forecast was made for. For instance, a 41
~~TS~~ Vi,m & {1,2,...,12},h€ {0,1,...,H|H EN }.
<!-- Start of picture text --> weight value<br>0.07<br>QHorizon T GHorizon T+1 mHorizon T+2<br>0.06<br>0.00<br>JAN FEB MAR APR MAY JUN JUL AUG SEP OCT NOV DEC<br><!-- End of picture text -->
Panel A. Predictive Probability Density Function and the Realized Value Line Panel B. Predictive Cumulative Distribution Function and the Realization Cumulative Distribution Function <!-- Start of picture text --> 0.35 1<br>Predictive density 0.9 Predictive CDF<br>0.3 Realized value Realized CDF<br>0.8<br>0.25 0.7<br>0.6<br>0.2<br>0.5<br>0.15<br>0.4<br>0.1 0.3<br>0.2<br>0.05<br>0.1<br>0 0<br>-2 -1 0 1 2 3 4 5 6 7 8 -2 -1 0 1 2 3 4 5 6 7 8<br>Y random variable Y random variable<br>PDF of Y CDF of Y<br><!-- End of picture text --> FIGURE 8. PROBABILITY FUNCTIONS OF AN ISSUED DENSITY FORECAST AND THE REALIZED VALUE On the other hand, the CDF for the value of the realized event-variable is denoted as ss YYaa = . Note that is the indicator function 1 . Under this notation, )≥0} PPaa(ss) PP(YYaa ≤ss) PPaa(ss) {(ss−ssaa the CRPS definition can be stated as presented in (33). Figure 8 shows an example of elicited probability density and cumulative distribution functions for the released predicting variable YY and the realized variable . YYaa ∞ 2 (33) ) = ∫−∞�<sup>PP(ss) −PP</sup> � . CCRRPPSS= CCRRPPSS(PP, ssaa aa<sup>(ss)</sup> mmss The CRPS ranks the distance between the issued event probabilities, and the PP(ss) materialized event probability, . Then, the CRPS measures the difference between the PPaa(ss) predicted and the occurred cumulative distributions. The CRPS best and minimal value of zero is only achieved when , that is, in the case of a perfect categorical PP(ss) = PPaa(ss) (deterministic) foresight. The CRPS has the dimension of the random variable , which is YY entered via the integration over . Besides, the CRPS can be understood as the limit of a mmss ranked probability score (RPS)—Epstein (1969) and Murphy (1971)—with an infinite number of classes and each with zero width.

The orientation of the CRPS is negative; that is, we prefer the scores with the smallest values. The lower the CRPS score, the better the prediction characteristics of the forecaster. Most 44
({(Po) } oy}!

CRPS(PO) ) Y~TPN (u ) ~~<u>7</u>~~ yIY]=E[(Y -E[Y) | = ~~V%~~

ee( ~~-—)~~ ao( ~~-—)~~ )[ ~~(-~~ - 1
<!-- Start of picture text --> —| ( —) ()<br>—-) -|<br>(<br><!-- End of picture text -->

<!-- Start of picture text -->
||<br>| |<br>(—| (-) |<br>—| ()<br>- [-|— (—) ()<br>_—_& J<br>EH) HO) —<br>_<br>& ]<br><!-- End of picture text -->
~~—-~~ n((: ~~—)~~ ( io (t ) ~~—-n~~ (() ~~Ji~~ je (4) ( ~~-)~~ i ee 9) “og (Ins aC ~~—)~~ ( Jo (1 ) cfs j=" (un ~~=~~ ) ~~at~~ } (« (@ )) var[s |= lel" Jaa ( <u>Je</u> et)
[ee ( )) ai] «el ( )) r(s )= [el ( +2] vel® (© )) ~~V6~~

[S|

{var|S |}