PROBABILITY MODELS FOR MONETARY POLICY DECISIONS

To be written. I. THE STATE OF CENTRAL BANK POLICY MODELING I.1. Not probability models. I.2. Ad hoc “econometrics” layered on a “long-run” core. I.3. Lack of a language to discuss uncertainty about models and parameters. I.4. Why models are nonetheless used, and are useful. The need to bring data to bear on policy discussions. There’s a lot of data, and it’s hard to maintain accounting and statistical consistency without a model. I.4. 1. Decentralization, model size. I.5. The impact of flexible inflation targeting. II. DIRECTIONS FOR PROGRESS There are promising new directions, but as we get closer to actually using these methods in real-time decision-making we will need to solve some implementation problems and broaden understanding of the contributions and limitations of these methods. II.1. Existing work. II.1. 1. Using bad models. Schorfheide, Geweke, Brock-Durlauf-West? II.1. 2. MCMC methods for parameter uncertainty. Smets and Wouters Date: July 9, 2003. c ©2003 by Christopher A. Sims. This material may be reproduced for educational and research purposes so long as the copies are not sold, even to recover costs, the document is not altered, and this copyright notice is included in the copies. 1 PROBABILITY MODELS 2 II.1. 3. Bayes factors, odds ratios, model averaging. Brock-Durlauf-West, Smets and Wouters II.1. 4. Perturbation Methods. II.2. Problems and prospects. II.2. 1. Using bad models: wasted energy? II.2. 2. MCMC methods. II.2. 3. Nonlinearities. Stickiness, welfare evaluation, the zero bound. III. ODDS RATIOS AND MODEL AVERAGING Though most central banks focus much more attention on one primary model than on others, the banks’ economists and policy makers are well aware that other models exist, in their own institutions as well as outside them, and that the other models might have different implications and might in some sense fit as well or better than their primary model. So one aspect of the Bayesian DSGE modeling approach that central bank economists find most appealing is its ability to deal with multiple models, characterizing uncertainty about which models fit best and prescribing how to use results from multiple models in policy analysis and forecasting. But practical experience with Bayesian approaches to handling multiple models has frequently turned out to be disappointing or bizarre. One standard applied Bayesian textbook (Gelman, Carlin, Stern, and Rubin, 1995) has no entry for “model selection” in its index, only an entry for “model selection, why we do not do it”. 1Some discussions of Bayesian methods by econometricians have asserted that Bayesian model selection methods can be useful in choosing among “false models”. This is basically not true. It is true that non-Bayesian approaches cannot put probability distributions across models, conditional on the data, any more than they can put probability distributions on continuously distributed parameters. But, as Schorfheide (2000) explains, choosing among, or putting weight on, false models by fit criteria or probability calculations that assume one of the models is true can lead to large errors. Bayesian methods are likelihood-based and therefore as subject to this kind of error as any other approach to inference. Schorfheide proposes ways to mitigate this problem. PROBABILITY MODELS 3 There are several related ways the Bayesian model comparison methods tend to misbehave. • Results are sensitive to prior distributions on parameters within each model’s parameter space, even when the priors attempt to be “uninformative”. • Results can be sensitive to seemingly minor aspects of model specification. • Results tend to be implausibly sharp, with posterior probabilities of models mostly very near zero or one. When we work with a single model, with a prior distribution specified by a continuous density function over the parameter space, under mild regularity conditions in large samples posterior distributions over parameters cease to depend on the prior. There are no such results available for model comparison, when we hold the set of models fixed as sample size increases. I will argue that these pathologies of Bayesian model comparison are not inherent in the methodology, but instead arise from the ways we generate and interpret collections of parametric models. Once this is understood, the model comparison methodology can be useful, but as much for guiding the process of generating and modifying our collections of models as for choosing among or weighting a given collection of models. III.1. When the Set of Models is Too Sparse. Even in simple situations with a small number of parameters, model comparison methods will misbehave when the discrete collection of models is serving as a proxy for a more realistic continuous parameter space. For example, suppose we observe a random variable Xt distributed as N(μ, .01). One theory, model 0, asserts that μ = 0, while another, model 1, asserts that μ = 1. With equal prior probabilities, if the observed X is bigger than . 55 or smaller than . 45, the posterior odds ratio on the two models is greater than 100 to 1. This is a correct conclusion if we in fact know that one of the two models must be true. The practical problem is that often we will have proposed these two models as representatives of “μ is small” and “μ is what is predicted by a simple theory” classes of models. It then is worrisome when an observation such as X = . 6, which might seem to slightly favor the “μ is big” class, but actually is highly unlikely under either model 0 or model 1, implies a result that suggests near-certainty, rather than skepticism. PROBABILITY MODELS 4 Here it is quite clear what the problem is and how to fix it: treat μ as a continuous parameter and report the entire likelihood. Or, having noticed that likelihood concentrates almost entirely in a region far from either of our representative models, generate new representative models that are more realistic. If we had a collection of models with μ = 0, . 1, . 2, . , . 9, 1, odds ratios across models would produce roughly the same sensible implications as use of a continuous parameter space. This is the kind of example that Gelman, Carlin, Stern, and Rubin (1995) have in mind when they cite, as a condition for posterior probabilities on models being “useful”, the requirement that “each of the discrete models makes scientific sense, and there are no obvious scientific models in between”. III.2. Changing Posteriors by Changing Priors. Suppose we have a set of models i = 1, . , q for a vector of observations X, with model i specifying the pdf of X as pi(X | θi), and the prior pdf on θi as πi(θi), over parameter space Θi. We will assume for simplicity that priors on parameter spaces are independent across models. Take the prior probabilities on models to be equal. The posterior weight on model i is then the marginal data density