Glenn Stevens: On the use of forecasts
Address by Mr Glenn Stevens, Governor of the Reserve Bank of Australia, to the Australian
Business Economists Annual Dinner, Sydney, 24 November 2011.
* * *
Thank you for the invitation to return to this platform.
This being a forecasting conference, you will have spent much of your time contemplating
the outlook for 2012. The Reserve Bank set out its views on the outlook only a few weeks
ago, and I will not canvass any changes to them on this occasion. Instead, I propose to
return to a theme I have covered on some previous occasions, namely, the nature and use of
forecasts for policy purposes. A couple of important points that have been illustrated over the
past two or three years are worth drawing out.
To begin, I would like to draw some observations from another kind of forecast. I do a bit of
aviation in my spare time. Hence I am a serious user of weather forecasts and there are very
detailed forecasts prepared on a frequent basis for the aviation community, in order to make
flying more predictable and safer.
I want to give an example. A couple of months ago, a pilot I know had planned to fly in a light
aircraft from Bankstown airport in Sydney's west to Armidale, about 90 minutes flight time to
the north, pick up some people and return to Bankstown.
On the relevant day (25 September), pertinent excerpts from the forecasts for the two
aerodromes were as follows:
YSBK (Bankstown)
15015G25KT 9999 LIGHT SHOWERS OF RAIN FEW 012 SCT020 BKN030
TEMPO 2500/2506 3000 SHOWERS OF RAIN BKN008
YARM (Armidale)
16012 9999 LIGHT SHOWERS OF RAIN SCT030
INTER 2501/2508 4000 SHOWERS OF RAIN BKN010
PROB30 INTER VRB20G30KT 3000 THUNDERSTORMS WITH RAIN SCT 045CB
It was windy and wet in Sydney that day, one of those days when strong south easterly
winds bring in moisture from the Tasman Sea and dump it on the Sydney basin. The
Bankstown forecast indicated that for much of the day, there were expected to be periods of
reduced visibility and heavy, low cloud. Conditions could thus be quite marginal for a landing
on the return flight, and it was possible they would be below the legal minimum for a landing
off an instrument approach during those periods of time. This meant a requirement to have
extra fuel on board in case of the need to hold prior to landing, waiting for the weather to
improve. The Armidale forecast showed some similar periods of weather, also requiring extra
fuel. More significantly, the Armidale forecast indicated that there was a probability
(assessed as 30 per cent) of thunderstorms in the area, which could persist for up to half an
hour at a time. Thunderstorms at the airport amount to very dangerous conditions in which to
attempt a landing because of the potential for very strong winds and windshear near the
ground, not to mention heavy rain or hail. Even large aircraft avoid landing in such
circumstances. Again, this meant a requirement to have additional fuel in case of the need to
hold prior to landing, waiting for the storm to pass.
Apart from that, the general conditions between the two airports were forecast to include
isolated thunderstorms, rain and areas of low cloud, all produced by the south easterly
airstream operating across much of NSW. The route goes more or less along the Great
Divide, which means that terrain effects on weather conditions are an issue to keep in mind.
These conditions did not necessarily preclude the flight, which could have been legally
commenced, provided the requisite additional fuel was carried. It is very likely that it could
have been safely completed. For professional pilots, who fly every day in a multi person crew
environment in high performance aircraft, dealing with these conditions would be seen as
reasonably routine, if a bit tedious. The main question was whether it was prudent for an
amateur single pilot flying a light aircraft to conduct the flight on this occasion. It was
observable that at the intended time of departure from Bankstown, conditions there were at
least as bad as forecast, while a phone call from Armidale indicated that there were in fact
storms present at that time. It did not look like a day on which the forecasters had been too
pessimistic. The pilot in question decided to stay on the ground.
The point of this little diversion into aviation is to make a few observations about the nature
and use of forecasts, which I think have some relevance in the economic sphere.
The first is that the weather forecasters had understood what I would call the "big forces". In
this case, there was a high pressure system over Tasmania, and a low pressure system off
the north-east coast of NSW. This combination fed very moist air over the south-east of the
continent, resulting in cloud and rain.
In fact, meteorologists know a lot about how weather works. They have pretty long time
series of observations and increasingly frequent real time observations of conditions. They
have highly developed models. The combination of real time data, understanding of how the
dynamics of weather occur and their experience, enabled the forecasters to get the big
picture right, and give a very useful forecast for those planning on venturing into the skies
that day.
The second point of note is that some elements of the forecast were probabilistic in nature.
This was explicit in the use of the term "PROB30". Weather forecasters know they are
dealing with a very complex, non linear system, and are careful to present their forecasts
accordingly. They are able to observe the unstable atmospheric conditions that are
conducive to storms - mainly heat and moisture, with a role played by terrain as well. They
cannot say for sure that there will be storms over a particular location, but they know enough
about likely conditions in an area to assess a probability. On the day in question, there would
almost certainly have been some storms to avoid somewhere along the route.
The third observation is that forecasts are used in particular ways. In aviation, lives can
depend on the way a forecast is used. Professional pilots in large, well equipped aircraft that
fly above most of the weather still carefully study forecasts and make the requisite
amendments to their plans. They carry additional fuel, have a plan B for an alternate airport
and so on as needed, in response to the forecast conditions. Many of us have had the
experience of fog induced delays in Canberra in winter, for example, where the runway is not
fully visible at the legal minimum on approach and so the aircraft cannot land and must go
somewhere else. International flights into Sydney very occasionally end up in Brisbane or
Melbourne because of fog, having been required by the forecast to carry the necessary
additional fuel. They may have carried it all the way across the Pacific, at non-trivial cost.
Despite the criticism aimed at weather forecasters, the forecasts I have seen in use for
aviation are generally pretty good. And the saying that economic forecasters are there to
make weather forecasters look good has something going for it. Of course one big difference
in economics is that some decisions based on forecasts may alter the outcomes - as in the
case of economic policy decisions, or spending decisions by businesses and households -
whereas our response to a weather forecast will not actually alter the weather. That factor
makes economic forecasting more difficult than weather forecasting. Still, some aspects of
the process of weather forecasting are valuable in the economic sphere.
One is that the most useful economic forecasts, like weather forecasts, are those that are
based on a good sense of the "big forces", as well as on an understanding of the dynamics of
how economies typically behave. In addition, we should admit that economic forecasts have
a margin of error - they are a point in a distribution of possible outcomes.
On the latter point, often much is made about small changes to forecasts, or small
differences in two forecasters' numbers. But when consideration is given to the real margin
for error around central forecasts, such differences are often, for practical purposes,
insignificant. For example, in the case of a year ended forecast for growth of real GDP four
quarters ahead, experience over the past couple of decades is that the probability of a point
forecast being accurate to within half a percentage point is about one in five. For year
average forecasts the accuracy is better, but still the margins for error are non trivial. So any
point forecast will very likely not be right. The likelihood of some outcome other than the
central forecast is actually quite high. When comparing forecasts, if we are not talking about
differences of at least half a percentage point, the argument is not worth having.
In any event, the question is not really whether the forecasts will turn out to be exactly right.
The question is whether they form a reasonable basis for sensible analysis or decisions at
the time. When the forecast turns out to be not exactly correct, as is very likely, that is
actually not much of a basis on which to criticise the decision makers who used the forecast
(or, for that matter, the forecaster).
For monetary policy operating a medium term inflation target, we are naturally interested in
our ability to forecast inflation. Experience over the inflation targeting era (since 1993)
suggests that the probability of the CPI outcome being within half a percentage point of the
central forecast is roughly two in five at either a one year or a two year horizon. For
underlying inflation, the probability of the forecast being within half a percentage point is
about two in three at one year and just over one in two at two years. The smaller forecasting
errors for underlying inflation reflect the inherently more stable properties of the underlying
measure, which of course is by design.
Hence, if the central forecast for CPI inflation at a two year horizon was 21⁄2 per cent, the
chances of the outcome being between 2 and 3 per cent, based on this historical experience,
would be about two in five. The chances of being between 11⁄2 and 31⁄2 per cent would be
three in five. I note in passing that, if this is a reasonable description of forecast accuracy, it
suggests that the configuration of the inflation target is a pretty good one (though I hasten to
add that, when it was first set out, we did not really have a great deal of confidence in the
accuracy of inflation forecasts).
It would, in my judgement, be vastly preferable for discussions of forecasts to be couched in
more probabilistic language than tends to be the case in practice, and for there to be more
explicit recognition that the particular numbers quoted are conditional on various
assumptions. Careful observers will have noted that the latest forecasts published by the
Bank actually have a range for growth and inflation at the horizon. Moreover, there is more
extensive discussion these days of the ways in which things could turn out differently from
the central forecast. This goes at least some way to recognising the inherent uncertainties in
the forecasting process, and is also important in relating the forecast to the policy decision.
Taking account of the accuracy statistics I have quoted above, we can characterise the
RBA's latest published outlook as suggesting that, absent large shocks to oil prices or the
Australian dollar, or further extreme weather events, or the world economy taking a serious
turn for the worse (say because of events in Europe), Australia's inflation rate in 2012 has a
pretty good chance of being between 2 and 3 per cent. The chances of a similar outcome in
2013 are also reasonable, though with slightly greater probability that inflation would end up
above 3 per cent in that year than seems the case for 2012. That is, the point forecast is a
little higher in the second year. Even so, a margin of uncertainty is inevitable. A big change in
any of the variables subject to assumptions would quite easily push outcomes away, and
maybe a long way away, from the forecast.
This degree of uncertainty can of course be quite disconcerting. It is only natural to desire
certainty. Everyone wants to know what will happen. We all want to believe that someone,
somewhere, does know and can tell us what to expect. But the truth is that the best we can
do when talking about the future is to speak about likelihoods and possible alternative
outcomes.
This is not a counsel of despair. It is not as though we can say absolutely nothing about likely
performance. We know something about average rates of growth through time, and we know
something about the long run forces that work to produce them (productivity and population
growth). We know that there have been, and will be again, periods of recession and
recovery, though our ability to forecast the timing of those episodes is limited. We know from
experience some things about the nature of inflation, including its characteristic persistence,
and the things that can push it up or down. We know some of the "big forces" at work on the
global and local outlooks - a once in a century terms of trade event, for example, and a once
in a century deleveraging event in major countries. We know that our country is exposed to
both forces - the expansionary effects of the rise in the terms of trade, and the dampening
effects of a mild degree of deleveraging in our household sector (and indirect effects of the
more intense deleveraging in some other countries). We also know that the terms of trade
change is a large shift in relative prices, which will bring about changes to economic
structure.
So there is a good deal we can say about the things that are relevant to our future, and
economists' understanding about these forces will be helpful in making sense of what occurs
over time. We simply have to recognise the limits on our capacity to predict their net impact
with any precision.
This in turn has implications for the way policymakers use numerical forecasts. In the case of
monetary policy, forming a forecast is unavoidably part of the process, simply because the
evidence suggests that monetary policy changes take time to have their full effect. So we
have to use forecasts - but not unquestioningly. We have to form a view about the big forces
at work, but also operate with due recognition of the limitations of numerical forecasts. The
extent to which policy should respond to forecasts will therefore always have some element
of judgement.
The conduct of policy over the past few years has exhibited these features. The Bank's
assessment of the very broad major forces at work has been central. Policy was tight in a
period in which the economy was very fully employed, confidence was high, the terms of
trade were rising and inflation was picking up. The very large and rapid easing of monetary
policy late in 2008 and early 2009 was a response to a major change in the outlook, which
occurred because the "big forces" changed direction very quickly, due to the financial events
at that time. Among other things, this saw strong growth in Asia go into sharp retreat,
appetite for risk and willingness to lend sharply curtailed and confidence slump. The changes
to monetary policy beginning in the latter months of 2009, designed to restore "normal
settings", occurred when it had become clear that the risk of a major economic contraction in
Australia had passed. In fact the "big forces" in the expansionary direction had reasserted
themselves, after an unexpectedly short absence: resurgent Asian growth helped to push the
terms of trade to new highs. In that world, leaving interest rates at 50-year lows would have
been imprudent.
Over the past 18 months or so, policy changes have been much less frequent, but the
process of decision making has nonetheless not been dull. A year ago, the then current data
on inflation showed nothing particularly alarming. The analysis of the Bank's staff suggested,
however, that the fall in inflation we had been seeing since 2008 would probably not
continue, but instead inflation would probably begin to rise, albeit quite gradually. The Board
took the view that, on the basis of that outlook and in the circumstances prevailing, it would
be prudent for policy to exert some mild restraint, and so it decided late last year to raise the
cash rate by 25 basis points.
The inflation data for the first half of 2011 do indeed show some increase in underlying
inflation (though after a sequence of revisions, this is not quite as large as it looked a few
months ago). As of May this year, the central forecast was for inflation to pick up further over
the ensuing couple of years, eventually rising to be clearly in excess of 3 per cent. This
carried a simple message. As the Bank said in the May Statement on Monetary Policy:
"The central outlook sketched above suggests that further tightening of monetary
policy is likely to be required at some point for inflation to remain consistent with
the 2-3 per cent medium-term target."
But there was still a matter of judging how to respond to that message. The Board did not
tighten policy at that time, nor did it do so three months later when the forecasts for inflation
still looked similar to those in May. It certainly considered whether that course of action would
be appropriate, but elected to sit still, watching unfolding events. Eventually, last month, far
from tightening, the Board actually eased policy slightly - though by then, of course, the
forecasts had changed materially from those of six months earlier.
This was not a repudiation of the forecasts, nor a sign that forecasts are not useful. The
process of forming forecasts remains key to the forward-looking conduct of policy. In electing
to take time in considering their response to signs of an increase in inflation, and central
forecasts of further increases, the policymakers were simply recognising the inherent
uncertainties of the situation and the difficulties the forecasters face, and giving those factors
due consideration.
It is, in my view, entirely appropriate that there be this degree of limited discretion for the
policymakers in their response to changes in numerical central estimates. It is not that
forecasts should be ignored. But neither should the decision be rigidly and mechanically
linked to forecasts. Were that to be so, the policymakers would in effect have delegated the
policy decision to the forecasters, which is not what policymakers are supposed to do.
So the relationship between the formal forecasts and the policy decision can sometimes be a
subtle one. Ultimately, the policymakers have to make a judgement call, based partly on
what the central forecast says but conditioned also by the degree of confidence they have in
it. At the same time it must be emphasised that policymakers can be in a position to make
the sorts of judgements I have just been describing only if they have generally acted in a
timely, forward-looking way in earlier decisions.
In my view, these judgements over the past year were the right calls. But in truth we will not
know for a while - such are the lags in monetary policy. As always, more data will help the
process of evaluation, though they might also provide evidence of new shocks (that is, things
the forecasters could not predict). Such is the nature of the forecasting game.
In conclusion, to those of you here who do not have to make forecasts, I hope you realise
how fortunate you are! To those who do, I offer my sympathy - and best wishes for clear
vision over the year ahead.