T T Mboweni: Artistic science or scientific art? - The role of forecasting
in monetary policy formulation
Address by Mr T T Mboweni, Governor of the South African Reserve Bank, at the Reuters
Economist of the Year Award, Johannesburg, 6 September 2007.
* * *
Honoured Guests,
Ladies and Gentlemen
1. Introduction
It is said that there are two kinds of economists: those who cannot forecast, and those who
don't know that they cannot forecast. We have to be thick-skinned to be economists as we
are often the butt of jokes. Apparently there are more jokes about economists than any other
profession, except perhaps lawyers. It would appear that the negative perceptions that are
held about economists can be blamed to a large extent on economic forecasters who, we are
told, have accurately forecast eight of the last three recessions.
But forecasting is not always a joke, and the quality of the contestants of the Reuters
Economist of the Year competition is testament to that. Forecasting is a serious and integral
part of economic life. Any decision, whether an investment decision or a policy decision, or in
fact any decision in life that involves taking a view on the future, has to be made on the basis
of some forecast. So if we regard forecasting as a joke, then the joke is on us. Unfortunately
we do not have perfect foresight and therefore we will never be able to forecast perfectly.
The best we can do is to strive to create forecasting models that are close approximations of
reality which in turn provides a coherent and disciplined framework for making decisions. In
my comments to you this morning, I will discuss the role of forecasting and how we use
forecasts in the monetary policy decision-making process.
2. Models and forecasts
There are different ways we can go about generating forecasts. Although there may be some
forecasters who engage in pure guesswork or thumb-sucking, most forecasters would be
informed by models with varying degrees of sophistication. These could vary from simple
extrapolation of the past, to analysing current developments and assessing their implications
for the future, to a more complex dynamic stochastic general equilibrium model, which is the
latest fad among model builders. Forecasting success, however, is not guaranteed by the
level of sophistication of the model.
The type of model we use would, to some extent, depend on the time horizon that we are
interested in, as different models are better suited to different forecast horizons. In the short
run, momentum of data may be more important than longer-term structural and behavioural
relationships. We therefore see different types of forecasting strategies in the markets. For
example, many traders have time horizons of a few minutes. To them tomorrow is very long
term. Those needing short-term forecasts will probably use chartist or bottom-up
spreadsheet techniques. These models have little basis in economic theory, and are unlikely
to perform well over longer-term horizons. Our structural models in the Bank, for example,
use quarterly data, so by definition they cannot be used for predicting one month ahead. For
this we would use autoregressive integrated moving average (ARIMA) models, which are
also momentum-type models with no underlying economic theory. Predictions based on
ARIMA are used for short-term predictions, and since they are based purely on historical
trends, they are not very good when it comes to predicting turning points.
Some forecasters rely on simple correlations noted in the market. As we all know,
correlations do not imply a causal relationship or even any direct relationship. The dangers of
spurious correlations are well-known. David Hendry, the renowned Oxford University
econometrician, in his appropriately-titled paper: "Forecasting: alchemy or science?"
illustrated this perfectly when he showed that there was a better relationship between
inflation and the cumulative rainfall in Scotland, than between inflation and monetary
aggregates. I don't know if this means that we should be employing Scottish weather
forecasters in the Bank to make our inflation forecasts. I am told, however, that weather
forecasters were created in order to make economic forecasters look good.
3. Why are forecasts wrong?
Even if we do build sophisticated structural models incorporating good behavioural
relationships, the forecasts are still likely to be wrong. There are various types of forecast
errors. Firstly, there may be misspecification of the model. This could mean that we have
excluded one or more variables, or that we have specified the wrong type of function. Our
modelling team is continually developing our models to try and overcome this type of
problem. Secondly, there may be structural breaks in the economy which are difficult to take
account of when estimating over a long time horizon. This means that there may be a bias to
the estimated coefficients of the model. This is particularly relevant to South Africa given the
transformation of the economy over the last decade or more.
A third problem relates to data. Historical data are subject to measurement error and are also
revised after publication. In a number of instances we don't have a consistent series going
back far enough to ensure a more reliable estimate of the parameters. In addition, some
variables, such as the wealth or expectations variables, have to be proxied. Inflation
expectations are central to the inflation formation process, yet modelling inflation
expectations is a major challenge.
While misspecified models, bad methods and inaccurate data are often blamed for serious
forecast errors, David Hendry argues that they are not the main cause of systematic
mistakes. Rather it is the unanticipated large changes or shocks within the forecast period
that are the primary source of errors. These are totally unpredictable or idiosyncratic events
that we simply cannot predict, and in these instances the fault then lies in not rapidly
adjusting the forecasts once they become inconsistent with the exogenous shock. When
shocks occur, the best we can do is to adjust our forecasts accordingly. This is why we
should be constantly monitoring new developments.
There are instances where the risks to the forecast may be known but are unquantifiable.
The current round of financial market volatility is a case in point. Although it was not a totally
unpredictable shock, the timing was always uncertain and we still don't know what the
ultimate impact will be. Lawrence Meyer, a former Federal Reserve Governor, in commenting
on the Federal Reserve's reactions to the financial market turbulence, said that the recent
shift in policy stance "tells us how difficult it is to translate financial turbulence into
macroeconomic forecast."
A model-based forecast is only as good as the key assumptions that it is based on. In the
forecasting process of the Bank, a lot of time and effort is dedicated to the process of
deciding on the exogenous assumptions of the model. Staff at various levels make inputs,
but ultimately, about two to three weeks in advance of the MPC meeting, the MPC members
meet to finalise the assumptions. Making assumptions about exogenous variables is an
important component of the forecasting process. But it is not always easy. Take for example
the international oil price for example. It is one of the most important exogenous assumptions
in our model and we have to formulate a view of the price over the coming three years.
Unpredictable geopolitical events, hurricanes etc., will surprise us over the period and force
us to change our views. No matter how well we analyse the underlying market conditions, we
are likely to be wrong because of unpredictable events.
The difficulty of forecasting the oil price is well illustrated when comparing the oil price
forecasts of a number of different forecasters. For the 2008 forecast, the spread between the
highest and lowest forecast in the market is almost US$22 per barrel, and for 2009 the
spread is slightly wider. If the oil experts are so uncertain, how confident can we be? Yet this
variable has a critical bearing on the inflation outcome and the accuracy of our forecasts.
One way we try and cope with this uncertainty is to consider various oil price scenarios, so
that we can see the sensitivity of the forecast to possible changes in the exogenous
variables. We then have to make a judgement call as to which is the most likely scenario. As
is the case with policy making, forecasting is very often a science as well as an art.
4. The use of forecasts in monetary policy decision-making
The Bank does not rely on a single model for its forecast. In line with most central banks we
have a suite of models that can be used for different purposes and various time horizons. As
I noted earlier, we use ARIMA models or vector autoregressive (VAR) models for short-term
forecasting and for estimating impulse responses. Other models include a disaggregated
model which builds up the inflation forecast from the individual components of CPIX
independently, which helps us identify the categories where price changes and inflationary
pressures have started to emerge. Finally, we have our main quarterly core model which has
18 structural equations.
Policy, like investment decisions, has to be made on a forward-looking basis. As
inflationtargeters, we need to set interest rates on the basis of our expectations of inflation over the
next two years or so. This is particularly the case given the lags in adjustment to interest rate
changes. The closer the model represents reality, the better it will be. However, as outlined
earlier, no model fully captures all the interrelationships in the economy or captures
expectations appropriately. Furthermore, as I outlined earlier, idiosyncratic shocks cannot be
forecast.
Much is made in the markets every month when new inflation data comes out. To us the
latest data point is only important to the extent that it may contain clues to the future and that
it gives us a new data point for the longer-term projection. The latest data point is in fact
history. We can compare this to driving a car. We do not drive a car by focusing only on the
rear-view mirror. That is a sure recipe for disaster. True, we have to look in the rear-view
mirror every now and again in case there is a bad driver bearing down on us, but we should
be generally looking ahead. Unfortunately, as with policy-making, the road ahead is not
always clear.
For these reasons, it is not possible to use the forecast in a mechanical way. Although in
theory models should incorporate all available information at any given point in time, there is
still a certain amount of judgement that must be used. So as policy-makers, we cannot
devolve our responsibilities to the outcomes of the model. Much consideration has to be
given to a thorough analysis of the risks to the outlook. Even the most sophisticated models
need to be supplanted by anecdotal and other off-model information. As Federal Reserve
Chairman Ben Bernanke recently remarked, "for all the advances that have been made in
modeling and statistical analysis, practical forecasting continues to involve art as well as
science."
Our most recent forecast that we reported on in the August MPC statement suggests that we
will be outside the target range until the second quarter of next year, and then inflation will
gradually decline. However this forecast is only as good as the assumptions we have put in
to the model. The sensitivity of the forecast to oil price and exchange rate changes are well
known, and any unexpected changes in these or other variables will cause the forecast to
change. So there can be no guarantee that the forecast will be the same next time.
Forecasts cannot be followed in a mechanical fashion since relevant off-model information
has to be accounted for and the risks to the forecasts assessed.
5. Forecasting and transparency
The global trend in recent years is towards increased transparency in monetary policy
formulation. This helps to make monetary policy more predictable which, in turn, helps
reduce volatility in markets and make forward planning easier. We can debate about where
to draw the line as to the limits of transparency, but one of the new trends has been for
central banks to publish their forecasts as well as their forecasting models. For some time we
have been showing our central forecast and the associated fan chart in the Monetary Policy
Review (MPR) which we publish twice a year. This is the forecast presented to the MPC at
the most recent MPC meeting prior to the publication of the MPR.
Furthermore, in our statements which are released after each MPC meeting, we give details
of the forecast so that market participants have a good idea of the timing and level at which
inflation is expected to change direction. We also indicate where we expect inflation to be at
the end of the forecast period.
More recently we have also published our core model following numerous requests from
private sector analysts and forecasters. We have also provided an assessment of the
performance of the model over time which shows that the Bank's model has performed well,
particularly relative to other private sector forecasts. The obvious question to ask is, should
these analysts not also be transparent with their models? Would it not improve the quality of
debate or assist in the development of better models if we could see how other forecasters
generate their forecasts? I have noted a deafening silence when I raise this issue with
market economists.
6. Conclusion
I will not attempt to forecast the winner of the competition. I congratulate all of the
contestants. The winner is the forecaster who has most accurately forecast the month-ahead
inflation over the past year. Although forecasting near-term inflation is useful, as the
Financial Times columnist, Sir Samuel Brittan argued, this type of forecast tells us more
about the present and the recent past than about the future. From a monetary policy
perspective, we are primarily concerned with forecasting over a longer time horizon. There is
nothing monetary policy can do about the latest inflation numbers. Monetary policy should be
judged on whether appropriate actions are being taken today to ensure that inflation will be
within the target range in 18-24 months time. For this we need to have a good medium-term
forecast to provide us with a coherent framework.
Despite all the problems associated with forecasting, they remain integral to policy decisions,
and we will continue to use them. The models have become increasingly sophisticated, but
the future is inherently uncertain. At the same time the economic and political environment
has also become more challenging and difficult to predict. Unfortunately, as is sometimes
said, the future is not what it used to be.
Thank you.