The Value of Forecasting in an Uncertain
World
A speech delivered to the New Zealand Manufacturers and Exporters
Association (NZMEA) in Christchurch
On 15 May 2017
By Dr John McDermott, Assistant Governor and Head of Economics
2 The Terrace, PO Box 2498, Wellington 6140, New Zealand
Telephone 64 4 472 2029 Online at www.rbnz.govt.nz
"We are all forecasters." Tetlock and Gardner, Superforecasting.1
It has often been said that there is no escaping forecasting. Every decision we make
in life involves forming a view about what the future might hold. Likewise, there is no
escape for the Bank. We have a Policy Targets Agreement that requires us to keep
future inflation between 1 and 3 percent and therefore we need to form a view of
where we currently are, how future developments might influence the path of output
and inflation, and thus how we should respond.2
We use our forecasting to establish a clear and well-founded plan for what monetary
policy settings are necessary to achieve our inflation target, to communicate this plan
to the public and to support the effectiveness and transmission of monetary policy.
We present our forecasts in the quarterly Monetary Policy Statement. A great deal of
effort goes into producing these forecasts.
More often than not, the world does not turn out as we forecast. However,
forecasting is still a valuable and necessary part of the monetary policy process.
Producing forecasts and allowing them to be publically subject to challenge enables
us to build a solid foundation for policy decisions, learn from developments, and
improve our policy outcomes. Forecasting is not supposed to be prophecy; rather, it
is about being precise about our thinking.
Uncertainty and the challenges for forecasters
A major challenge for forecasters is 'radical uncertainty', that is, we fundamentally
don't know what the future holds. Examples of this uncertainty are 'black swan'
events that fall outside any existing experience. The term is derived from the
northern hemisphere belief that all swans were white, based on the empirical
evidence that every swan ever observed was white.3 For the first European migrants
to Australia, discovering black swans was a completely unexpected event, lying
outside of all evidence and experience up until that point.
For forecasters, such unpredictable events are a particular challenge. We have a
wide range of information about the past and current state of the economy, and are
setting policy to influence the future direction of the economy. But the economy is
populated with thousands of households and businesses responding to their own
particular circumstances and opportunities, so it is never certain how the economy
will evolve over time. Even without black swan events, the chain reactions of
economic activity are so complex and so changeable that the range of possible - let
alone probable - outcomes is vast.
Tetlock and Gardner argue that the best forecasters understand that the world is
uncertain and therefore react to it flexibly.4 Good forecasters are careful, curious,
and comfortable with numbers, and, crucially, when the facts change they change
their minds. And that is how we strive to be. Conversely, the worst forecasters
remain dogmatically attached to their forecasts, unchanging in the face of the
evidence.
Forecasting in uncertainty: dynamic programming and conditional forecasts
So how do we make policy for an uncertain future? The approach that the Bank
takes is to develop forecasts that are highly conditional on the information currently
available. That is, our forecasts are subject to revision when additional information
comes to light. These forecasts are then used in what Alan Blinder would describe
as a 'dynamic programming' plan.5 This means thinking about what you want to
achieve in the future and working backwards to form a plan today that will deliver
that goal. In our case the goal is inflation near 2 percent in two or three years' time,
and the plan is given by the forecast of the Official Cash Rate (OCR). If events
broadly unfold as envisaged then we carry out that plan, if not, then we adjust the
plan.
A feature of our inflation forecasts are that they always end up at the mid-point of the
target band. When planning our actions we necessarily plan to succeed in achieving
this target. If we thought we would not achieve this we would modify our OCR
forecasts.
As such, the forecasts show our judgement of the highest probability path to achieve
the goal, but this path is not fixed. As new information becomes available, we assess
it and where appropriate incorporate it into our understanding and our plans.
Consequently, our policy settings are, and should be, responsive to any
developments that might occur, and should move rapidly when required. For
example, the sharp fall in inflation expectations in March 2016 (amongst other
developments) led to us substantially revise our 90-day interest rate projection6 and
move away from a neutral stance to an easing bias (figure 1).7
Figure 1: Inflation expectations and selected 90 day interest rate projections
Source: RBNZ estimates.
The value of producing quantitative forecasts: internal discipline and the
learning process
So why do we produce specific forecasts given that our forecasts are conditional and
subject to radical uncertainty? Being numerically precise about our view of the future
allows us to test ideas, which in turn accelerates our ability to learn and understand
what is going on. Over time, this helps to ensure that our forecasts are as robust as
possible and form a solid basis for monetary policy decisions, minimising potentially
costly forecast errors and supporting good policy outcomes.
Testing is an integral part of robust analysis. As Nate Silver puts it, "The more
eagerly we commit to scrutinising and testing our theories, the more readily we
accept that our knowledge of the world is uncertain, the more willingly we
acknowledge that perfect prediction is impossible, the less we will live in fear of our
failures, and the more freedom we will have to let our minds flow freely."8
When an outturn is different from that forecast, the Bank seeks to discover why, what
it means for the outlook for monetary policy, and what we could do to make a better
forecast next time.9 We use our depth of experience to examine the data, distinguish
the signal from the noise, and identify where to adapt our approach or where to
maintain our existing views.
For example, residential investment at the end of 2016 was considerably weaker
than the Bank had forecast. In seeking to understand this forecast error, the Bank
considered the composition of residential investment. We found that a sharp fall in
transfer costs (based on decreasing house sales) was the cause of weak residential
investment, rather than weakness in actual construction activity. Accounting for this
information should support better forecasts for this part of the economy in the future
(figure 2).
Figure 2: Residential building activity and house sales
$m 000s
1700 28
1600
26
1500
24
1400
22
1300
20
House sales
1200
18
1100
Residential building
16
1000
work put in place
14
900
800 12
2006 2008 2010 2012 2014 2016
Source: RBNZ estimates.
Producing forecasts: avoiding biases and supporting robust outcomes
Forecasting can also help reduce human biases that are often present in decision
making. Our brains are designed to react to surprises by making sense of them. In
seeking to do so, we often rewrite our perceptions from prior to these events.10 Even
black swans may feel retrospectively predictable. This is known as 'hindsight bias' or
the 'knew it all along effect'.11
As forecasters, we need to be particularly aware of this bias. To learn from our
errors, we need to recognise that they are errors. By recording our quantitative
forecasts and accompanying reasoning, we prevent ourselves from rewriting our
original position and ensure we take on the full insights from any forecast errors we
make. Over time the Bank has built an extensive institutional knowledge base of
New Zealand economic data and experience of monetary policy formulation. Each
time we forecast, we examine the latest data, identify where new information should
lead us to change views, and incorporate lessons learned to ensure that our
forecasts keep evolving and are as accurate as possible.12 To use Kahneman's
terminology, our forecasts force us to think more slowly.
Forecasting is not the only essential part of monetary policy formulation. The Bank
has a well-defined process for monetary policy decisions, based on a Monetary
Policy Committee (MPC) that provides advice to the Governing Committee.13 The
MPC includes two external members to add to the Committee's diversity of
backgrounds and views. These external members provide a 'reality check' on the
Bank's thinking, and subject us to valuable independent scrutiny.14
The MPC structure is supported by a set of working practices that ensure that
colleagues closest to the data and forecasting processes present their work to the
MPC, and that subsequent discussions are broad-ranging and unconstrained in
order to avoid groupthink or other biases.15 This means that the analytics of our
Economics and Financial Markets Departments are blended with the experience of
senior staff and Committee members, achieving better results through incorporating
judgement in the forecasting process.
How do we create forecasts at the Reserve Bank?
The Bank monitors a wide range of data to gauge the current state of the economy,
distilling the underlying trends and signals from the noise and assessing what the
emerging picture means for the stance of monetary policy. In addition to formal
model-based analysis, monetary policy formulation is supported by the feedback we
12
In essence, this is a Bayesian approach combining priors with new information using judgement. For a
readable introduction see James V. Stone (2015), Bayes' Rule with Matlab: A Tutorial to Bayesian Analysis,
Sebtel Press.
13
The Governor retains statutory responsibility for OCR decisions, but the Bank has in practice delegated
the decision on the appropriate policy settings to the Governing Committee.
14
The MPC has included external members since 2001. External members are limited to serving a 1 to 3
year term to ensure that they remain truly external. The two current and previous members are: Tony
Caughey; Conor English; Richard Townshend; Luke Moriarty; Pip Dunphy; Earl Rattray; Liz Coutts;
Bronwyn Monopoli; Terry McFadgen; Malcolm Bailey; Kerrin M Vautier; and Brent Layton.
15
For more detail see C J McDermott, 'How the Bank formulates and assesses its monetary policy
decisions', a speech delivered to the Manawatu Chamber of Commerce in Palmerston North, 13 July 2016;
and Adam Richardson, 'Behind the scenes of an OCR decision in New Zealand', RBNZ Bulletin 79(11) July
2016. For example, by convention the Governor speaks last in MPC meetings so as not to unduly influence
the direction of the debate before a diverse range of views are heard from staff.
receive from regular meetings with private businesses, industry associations and
government agencies. We also undertake longer-term research projects to improve
our understanding of specific features of the economy.16
At the heart of our forecasting is our structural model, the New Zealand Structural
Inflation Model (NZSIM). This is a theoretically rich framework that describes key
behavioural relationships in the New Zealand economy and how they influence
inflation. NZSIM generates the core forecasts for informing monetary policy
decisions, and helps us identify the fundamental economic drivers and empirical
relationships to explain the observed economic data.17
The information collated over the forecast round is fed into NZSIM, and the
theoretical structure of the model ensures that our forecasts are consistent. For
example, the national income accounting discipline is preserved, we can distinguish
between domestic and imported inflation which sum up to CPI inflation, and we are
able to trace the forces driving economic growth.
During the monetary policy decision-making process, the NZSIM forecasts are cross-
checked with other modelling, and where appropriate modified to take into account
additional information not directly represented in the model. That is, our forecasts are
'model-assisted' rather than 'model-produced'.18 NZSIM also can consider alternative
scenarios, meaning that we can test competing ideas and assess how monetary
policy should respond if circumstances develop differently.
NZSIM necessarily abstracts from reality, and (as with any modelling) requires some
difficult tradeoffs to be made, balancing the amount of detail that can be incorporated
without sacrificing the ease of interpretation and resulting policy recommendations.
The combination of tractable model outputs and the expert judgement of Bank
colleagues and Committee members forms the basis for building nuanced and
detailed policy recommendations.
Over the last few decades, increases in computing power have enabled the Bank to
vastly expand its modelling and research capability, including making NZSIM
possible. However, one significant limiting factor to forecast improvement is data:
macroeconomic statistics have not fundamentally changed over the 30 years we
have been using inflation targeting.19 For instance, New Zealand is one of only two
advanced countries that use a quarterly consumer price index rather than monthly.
This is likely to be a fruitful avenue for future improvement that would greatly improve
the Bank's forecasting ability.
The benefits of publishing our forecasts: improving the effectiveness of
monetary policy
The RBNZ Act20 requires the Bank to publically set out its monetary policy stance
and underlying reasoning, including a review of previous decisions. This information
is published in our quarterly Monetary Policy Statement, along with our economic
forecasts and accompanying explanation. The Bank was the first central bank to
publish its planned interest rate path, starting in 1997, and we continue to rank as
one of the most transparent central banks.21 Publishing these macroeconomic
forecasts and our planned OCR path provides a clear 'line in the sand' as to the
Bank's thinking. It also helps synthesise our views in a more approachable format,
enabling commentators, market participants, and others to understand the Bank's
monetary policy stance.
Communication is a key part of the monetary policy transmission mechanism as it
helps people to understand the Bank's thinking and expected policy direction.22 By
building people's understanding of how the Bank is likely to react to news, the
predictability of monetary policy decisions is enhanced and policy uncertainty is
reduced. We are a long way from the days when central bankers cultivated their
ability to "mumble with great incoherence".23
To inform the market about the uncertainty inherent in projections, some central
banks publish fan charts showing the range of possible outcomes around the highest
probability scenario. While this approach shows the extent of uncertainty, it does not
help people understand how the central bank's forecasts may evolve if some of the
contingencies change. Consequently, in recent Monetary Policy Statements we have
published a range of potential scenarios to help people understand how the
forecasts would change should the economy develop differently. We believe that this
approach is potentially more helpful.
Figure 3 illustrates how market participants understand the way the Bank is likely to
incorporate economic and financial information into its projections. The market
revisions (red line) are generally good at anticipating changes to our 1-year ahead
90-day interest rate projection (blue line), which suggests that they understand the
conditional nature of the forecasts and the Bank's likely response (or 'reaction
function') to new information.
Figure 3: MPS and market revisions
Source: RBNZ estimates.
Case Study: the increase in the OCR in 2014
The increase in the OCR in the first half of 2014 provides a good case study of our
conditional approach to forecasting in a 'radically uncertain' world. At the start of
2014 the global economy appeared to be recovering from the 2008/9 global financial
crises, with the IMF asking whether the "tide was rising".24 The New Zealand
economy was growing strongly: the terms of trade had reached their highest level in
40 years; construction activity was robust; immigration was boosting housing and
consumer demand; and monetary policy was providing substantial stimulus to the
economy. Under these circumstances, we judged that there was a high probability
that inflationary pressures were rising, contingent on the global and domestic
economy developing as forecast and our knowledge of the structure of economic
relationships. This view about rising inflation pressures was shared by the private
sector forecasters and the IMF. Consequently, the OCR, which was set at an
accommodative level of 2.5 percent, was raised by a total of 1.0 percentage points at
successive OCR reviews in the first half of 2014.
However, by September 2014 it had become apparent that the forecast
contingencies were not developing as expected. Firstly, uncertainty around the
central scenario had risen: the global economy was not strengthening as anticipated
and New Zealand terms of trade had fallen by about 7 percent since the start of
2014. Secondly, uncertainty about the structure of the domestic economic
relationships was becoming apparent as CPI inflation remained moderate despite
the inflationary pressures. Under these circumstances the Bank judged it appropriate
to put a hold on interest rates, await the flow of new economic data, and investigate
the evolution of domestic economic relationships.
In the event, this decision proved correct. Global economic growth continued to be
sluggish through 2015 and 2016. The Bank's analysis of the domestic economic
relationships concluded that resource pressures were not as strong as previously
estimated, in part due to the record level of migration that the New Zealand economy
was experiencing.25 Low inflation had also changed the price-setting behaviour of
New Zealand businesses, who were placing a greater weight on recent low inflation
(rather than expectations of future inflation) in their wage-setting and pricing
decisions.26 Incorporating these lessons into our forecasting framework meant that
by mid-2015 the most probable scenario had changed to a continuation of low
inflation (figure 4). The Bank commenced a new cycle of reducing the OCR, to an
eventual low of 1.75 percent in November 2016. Today, in May 2017, the most likely
scenario is for the OCR to remain stable for some time, although uncertainty remains
high.
Figure 4: selected 90-day forward rates
Source: RBNZ estimates.
Conclusion
Forecasting is a valuable part of the monetary policy process, helping the Bank to
plan for the future, make well-founded policy decisions, communicate its
understanding and intentions, and accommodate new information as it appears. It
requires the Bank to be rigorous, unbiased, and open to new ideas in formulating
and implementing monetary policy.
Because we operate in a world of radical uncertainty, we do not know what might
happen in the future; we are not fortune tellers. However, because we want our
forecasts to be relevant and useful, we present them numerically, an approach that
comes with risks. As Tetlock and Gardner put it, "use the number and you risk being
unfairly blamed. Stick with phrases as fuzzy as a puff of smoke and you are safe".27
We do not want to be safe in this context as we welcome scrutiny, both internally and
externally, as it builds understanding and fosters accountability.
Finally, good forecasting happens when the numbers are supported by evidence and
reasoning, and subject to public scrutiny. When things - inevitably - do not turn out
as projected we seek to learn from those episodes and continually work to improve
our forecasts and policy-making.
27
Tetlock and Garnder, op.sit.