Lars E O Svensson: Evaluating monetary policy
Speech by Mr Lars E O Svensson, Deputy Governor of the Sveriges Riksbank, at Uppsala
University, Uppsala, 13 March 2009.
The original speech, which contains various links to the documents mentioned, can be found on the Sveriges
Riksbank's website.
* * *
Ten years ago - on 1 January 1999 - a number of legislative changes came into force that
made the Riksbank more independent. As a result, the monetary policy decisions since then
have been made by an Executive Board consisting of six members who are not permitted to
seek or take instructions in the course of their work. These legislative changes can be seen
as part of an international trend that has now been underway for a couple of decades - a
trend towards an institutional framework for monetary policy with the aim of making monetary
policy more effective and more able to promote monetary stability and stability in the real
economy.
This institutional framework for monetary policy rests on three pillars:
1. A government or parliamentary mandate for monetary policy, which is normally to
maintain price stability.
2. Independence for the central bank to conduct monetary policy and fulfil the
mandate.
3. Mechanisms for accountability and supervision.
The central bank's independence gives its governors a lot of power. In a democratic society,
it is natural that the activities of the central bank are monitored and evaluated and that its
independent management can be called to account. Transparency and regular evaluations
are necessary to maintain the legitimacy of the central bank's operations. They also give the
central bank stronger incentives to fulfil its mandate and motivate the central bank to develop
the monetary policy analysis in the best possible way.
It also appears that detailed evaluations of monetary policy are becoming increasingly
common. In Sweden, the parliamentary Committee on Finance conducts an annual
evaluation of the monetary policy conducted over the last three years. An external and more
comprehensive evaluation of the monetary policy conducted in Sweden between 1995 and
2005 was carried out on behalf of the Committee on Finance by professors Francesco
Giavazzi and Frederic Mishkin. The Committee on Finance has decided that Swedish
monetary policy should be evaluated by external experts every fourth year. Norges Bank (the
central bank of Norway) includes an evaluation of its monetary policy in its Annual Report.
On behalf of the Norwegian Ministry of Finance, an annual evaluation is also carried out by
Norges Bank Watch, which consists of a group of independent experts. I myself have taken
part in two evaluations - an evaluation of ten years of monetary policy in New Zealand on
behalf of the country's Minister of Finance in 2001 and an evaluation of monetary policy in
Norway as Chairman of Norges Bank Watch in 2002.1
Evaluations of monetary policy are thus important and appear to be becoming common
practice. One must therefore ensure that reasonable principles and appropriate methods for
evaluations are developed and applied. This is what I intend to talk about today. What should
we consider when we evaluate monetary policy? What are the principles for a good
evaluation of monetary policy and what is practically possible? How can principles and
practice be developed compared with how evaluations are carried out today? Here I present
a number of preliminary proposals for such development.
Why not just examine whether inflation deviates from the target?
An increasing number of central banks focus their monetary policy on achieving an inflation
target. My discussion today will therefore be about evaluations of monetary policy with an
inflation target.
Given this, you may ask why an evaluation of monetary policy should be so complicated.
When there is an inflation target, is it not simply enough to compare the actual outcome for
inflation with the inflation target; that is to evaluate monetary policy after the fact? There are
at least two circumstances that make such an evaluation inadequate.
Unforeseen shocks affect outcomes
First, monetary policy does not provide complete control over inflation. A central bank is
therefore unable to ensure that inflation will be exactly on target at every point in time. In
actual fact, monetary policy is conducted under conditions of great uncertainty. Our
knowledge of the economic situation is not complete, and neither is our knowledge of how
monetary policy affects inflation and the real economy, the so-called transmission
mechanism. There is a considerable time lag before monetary policy measures have an
impact on inflation, and the duration of this time lag also varies depending on the
circumstances. This impact is normally gradual and becomes apparent over the course of a
couple of years.
As monetary policy works with a time lag, it is most effective if it is based on forecasts. In
order to achieve an inflation rate of two per cent, it is therefore best to set the policy rate so
that the inflation forecast a couple of years ahead corresponds to the inflation target.
However, during the time it takes for changes in the interest rate to have a full impact on
inflation the economy will be affected by new and unexpected shocks. The inflation outcome
a couple of years ahead will therefore have been affected by events that could not be
predicted when the monetary policy decisions were made.
A direct comparison of outcomes and targets for inflation may therefore lead to the wrong
conclusions. The inflation outcome may be in line with the target even if the monetary policy
decisions were incorrect because the central bank was lucky and unexpected shocks
nevertheless resulted in the right inflation outcome. Alternatively, the inflation outcome may
deviate from the target even if the monetary policy decisions were correct because the
central bank was unlucky and unexpected disruptions resulted in the wrong inflation
outcome.
Inflation targeting is flexible
A second circumstance that means that a simple comparison of outcomes and targets for
inflation is inadequate is that the Riksbank and all the other central banks that have inflation
targets conduct flexible inflation targeting rather than strict inflation targeting. Under a flexible
inflation targeting regime, the central bank strives not only to stabilise inflation around the
inflation target but also to stabilise the real economy. Under a strict inflation targeting regime,
the central bank strives only to stabilise inflation without concerning itself about the
consequences for the real economy.
In many situations, a conflict of objectives may arise between stabilising inflation and
stabilising the real economy. Let us assume that a shock, such as a sharp rise in the oil
price, has driven up inflation at the same time as production has slackened significantly. If, in
such a case, the aim of monetary policy is to quickly bring inflation back to the target, a
significant interest rate increase may be required which will dampen production even further.
By quickly stabilising inflation - which would be the case with strict inflation targeting - the
central bank would destabilise the real economy. By not aiming to bring inflation back to the
target as quickly as possible, the central bank would help to stabilise the real economy. How
long it should take to return inflation to the target depends, among other things, on the type,
magnitude and duration of the shock that has occurred and the importance that the central
bank attaches to real economic stability.
A difference between the outcome and the target for inflation may thus be deliberate. It may
be part of an appropriate compromise between stabilising inflation and stabilising the real
economy. It is, therefore, simply not good enough to just compare outcomes and targets for
inflation in an evaluation of monetary policy.
What does flexible inflation targeting entail?
Before I begin to discuss what evaluations of monetary policy should focus on, let me go into
a little more detail about what characterises flexible inflation targeting.
As I have already said, flexible inflation targeting entails the central bank striving to stabilise
inflation around the inflation target and at the same time to stabilise the real economy.
Stabilising the real economy may be more precisely described as stabilising resource
utilisation at a normal level.
There is an asymmetry between the impact of monetary policy on inflation and its impact on
the real economy that it is very important to understand. Monetary policy can affect both the
average level and the variability of inflation. Monetary policy can not, on the other hand,
affect the average level of real quantities such as production, employment and resource
utilisation. Historically, attempts to use monetary policy to affect the average level of real
variables such as employment and unemployment have led to serious mistakes and high
inflation. In the case of the real economy, monetary policy can only affect, and to a certain
extent dampen, fluctuations in real variables around their average levels. For monetary
policy, it is thus meaningful to select a certain target for average inflation, but it is not
meaningful and in fact counterproductive to select a certain target for average production or
employment, other than the normal level that is determined by the workings of the economy
and factors other than monetary policy.
With a flexible inflation targeting regime, the central bank then has to set a policy rate and
choose a policy rate path that make the forecast for inflation and resource utilisation together
"look good". By a forecast that looks good I mean a forecast in which inflation is on target
and resource utilisation is normal, or in which inflation is approaching the target and resource
utilisation is approaching a normal level at an appropriate pace. To be more precise, it
means a forecast for inflation and resource utilisation that as effectively as possible stabilises
inflation around the inflation target and resource utilisation around its normal level and, in the
event of conflicting objectives, achieves a reasonable compromise between inflation stability
and resource utilisation. Different central banks express this in slightly different ways. The
Riksbank has often used the term "a well-balanced monetary policy".
We can formalise and specify this reasoning somewhat by saying that it is a case of selecting
a policy rate path that minimises a forecast loss function, written as the following common
quadratic form:
Evaluation ex ante - given the information available at the time of the decision
So, given that a central bank conducts flexible inflation targeting, how should we evaluate the
monetary policy the bank conducts? When evaluating monetary policy, as when evaluating
decisions in general, we may adopt one of two different starting points. The first option is to
base our evaluation on the information that is currently at hand; that is also including the
information that has become available since the decision was made. This is, in other words,
an evaluation after the fact. We can call this an ex post evaluation. The second option is to
put ourselves in the position of the decision makers at the time the decision was made and to
try to evaluate the decision given the information that was then available. We can call this an
ex ante evaluation.
When evaluating monetary policy, the most interesting question is not whether the decision
could have been better after the fact. As I said earlier, monetary policy is conducted under
conditions of great uncertainty - there is a time lag before monetary policy has an impact on
inflation and the real economy and the economy is constantly subject to new shocks. It is
almost self-evident that monetary policy could have been better if the decision-makers had
been aware that these shocks would happen when they made their decision. It is often
equally self-evident that it was not possible to foresee the shocks at the time the decision
was made. The relevant question is therefore primarily whether monetary policy could have
been better given the information on the state of the economy and other factors that the
central bank had access to when the decisions were made.
How then should an ex ante evaluation of monetary policy be carried out? I believe that we
must take flexible inflation targeting seriously. For a central bank that conducts flexible
inflation targeting it is important, as I mentioned earlier, to choose a policy rate path so that
the forecasts for inflation and resource utilisation "look good" in terms of stabilising both
inflation and the real economy and, in the event of conflicting objectives, entail a reasonable
balance between stabilising inflation and stabilising the real economy. An ex ante evaluation
should then aim to assess whether the central bank has succeeded in doing this.
Before making such an assessment of the central bank's forecasts it is natural to first
examine the general quality of the forecasts. Any assessment of the quality of the forecasts
obviously entails an ex post analysis with the help of historical forecast errors. If the
assessment is that the forecasts are of a reasonable quality it then becomes a question of
assessing the monetary policy deliberations held on the basis of the forecasts the central
bank makes. This primarily entails an ex ante analysis.
Were the forecasts good enough?
The first question we should ask is thus whether the central bank's forecasts are normally
good enough. It would of course be going too far to demand that the central bank's forecasts
should be perfect. As I have said, the economy is constantly subject to unexpected shocks,
which means that the forecasts are always likely to be incorrect to some extent. Analysing
the accuracy of a forecast in an individual year thus provides limited information. A significant
forecasting error may indicate that the forecast was poor, but it may also be due to the fact
that a shock occurred that could not have been predicted.3
Do the forecasts systematically over- or underestimate the actual outcomes?
A reasonable demand is that the forecasts for inflation, resource utilisation and so on do not
systematically over- or underestimate the actual outcomes. In other words, the forecasts
should not have any bias. If, for example, the forecasts for inflation over a long period of time
on average over- or underestimate the actual inflation outcome then this is a sign that there
is information that the central bank is missing and that could be used to improve the
forecasts. It is, however difficult to determine whether the forecasts are unbiased on the
basis of a small number of outcomes, especially if we take into account the uncertainty in the
functioning of the economy.
How do the central bank's forecasts compare with other forecasts?
Another reasonable demand is that the central bank's forecasts are on average not poorer
than those of other forecasters. If the central bank's forecasts are systemically poorer than
those of other forecasters, then this is obviously an indication that it would have been
possible to make better assessments than those made by the central bank. This also means
that better information was available which the central bank would have been able to use as
a basis for its decision-making.
It is important to remember, however, that the preconditions governing forecasting at a
central bank differ somewhat compared to those for many other forecasters. For the central
bank it is not enough that the forecasts are accurate. This is one very important quality, but
there are others. It must be possible, for example, to understand the driving forces behind
the forecasts - why does the forecast look the way it does? How is the forecast affected by
changes in assumptions regarding, for instance, demand abroad or the oil price? The central
bank also needs a forecasting apparatus that can effectively investigate the consequences of
an alternative monetary policy. For the Riksbank, which publishes it own interest rate path,
this is extremely important. To put it more technically, the central bank must use so-called
structural models to assess the consequences of various policy rate paths, while other
forecasts can mainly be produced with the help of statistical models alone. Normally, there
are good reasons for believing that structural models provide poorer forecasts than statistical
models. However, the Riksbank's main structural model, Ramses, has very good forecasting
properties.4
When making historical comparisons between the forecasts of central banks and those of
other analysts, we also face a number of other difficulties. The first is that it is not certain that
the central bank's forecasts are based on what the bank really believes is the best forecast of
the policy rate and other variables. For a long time, the Riksbank, for example, based its
forecasts on the assumption that the policy rate, that is the repo rate, would remain
unchanged during the forecast period. Some central banks base their forecasts on both a
constant interest rate and a constant exchange rate, which are often completely unrealistic
assumptions. In such cases, the forecasts for inflation and the real economy are of course
not the best forecasts. For a while, the Riksbank instead used the assumption that the repo
rate would follow the market's expectations regarding the development of the repo rate.
Although this is a more realistic assumption, it is not necessarily the same as the Riksbank's
best forecast. These difficulties no longer apply to the Riksbank's forecasts as, since
February 2007, the forecasts are based on the Riksbank's best forecast for the repo rate.
In order to arrive at a fair comparison of the forecasts of various analysts, we should also
take into account the fact that the forecasts are made at different times and are therefore
based on different quantities of information. A forecaster that, for example, always publishes
its forecasts later than other analysts has generally speaking more information on the
economic situation when the forecasts are made. A comparison of the accuracy of the
forecasts should therefore make adjustments for the forecasts being made at different times.
Such a forecast comparison is included in the material for the evaluation of monetary policy
in the period 2006-2008 that the Riksbank published a month ago.5
Figures 1 and 2 show a comparison of the accuracy of the forecasts in the period 1999-2008
for CPI inflation and GDP growth for a number of analysts. The red bars show the absolute
mean error adjusted for differences in publication dates. The blue bars show the mean error
with positive or negative sign. The shorter the blue bars, the smaller the systematic over- or
underestimation has been - and the less bias the forecasts have had. If the bar is above the
zero line the mean error has been positive and the forecasts have on average been too low,
and vice versa. In the case of both inflation and GDP growth, the Riksbank's accuracy has
been relatively good and its bias relatively small compared to other forecasters.
Was monetary policy well-balanced ex ante?
Given that it has been determined that the central bank's forecasts are normally satisfactory,
the next step is to analyse, ex ante, the monetary policy deliberations the central bank has
conducted on the basis of the forecasts.
The first question to answer is whether the monetary policy conducted has been efficient.
Given the information available at the time the decision was made, would it have been
possible, by selecting a different interest rate path, to have stabilised inflation or the real
economy better without stabilising the other less well? Would it even have been possible to
achieve a better stabilisation of both?
The idea behind an efficient monetary policy can be illustrated using a modified Taylor curve,
named after the economist John Taylor.6 As I have said, under a flexible inflation targeting
regime the central bank aims to stabilise inflation and resource utilisation; that is to minimise
the deviations from the inflation target and the normal level for resource utilisation. In Figure
3, the sum of squares for the inflation-gap forecast is measured along the horizontal axis and
the sum of squares for the output-gap forecasts along the vertical axis. The curve through
points A, B and C is the modified Taylor curve, that is, all the efficient combinations of
forecasts for inflation and resource utilisation respectively that it is possible to achieve in a
certain decision-making situation with the help of different interest rate paths. Points to the
left and below the curve cannot be reached due to the initial state of the economy and the
transmission mechanism between inflation, resource utilisation and the repo rate. Points to
the right and above the curve are inefficient in the sense that it is possible for monetary
policy to achieve a smaller sum of squares for the inflation-gap forecast for a given sum of
squares for the output-gap forecasts, or vice versa.
In the Monetary Policy Reports, the Riksbank usually presents alternative scenarios with a
different repo rate path in addition to the main scenario. These generate other paths for
inflation and the output gap. Figures 4 and 5 show examples from February 2008 and
February 2009. The sums of squares that can be calculated from these paths are shown in
Figure 6. Two observations can be made here. The first observation is that the sums of
squares in the main scenario are smaller than in the alternative scenarios - the main
scenario is closer to origin in Figure 6. The main scenario is thus more successful in terms of
stabilising both inflation and resource utilisation. The alternative scenarios are clearly
inefficient compared with the main scenario. This does not necessarily mean, however, that
monetary policy is efficient in the sense that the main scenario lies on the modified Taylor
curve. There may be a repo rate path that would stabilise inflation and resource utilisation
even better. Excluding this possibility requires a comparison with many more alternative
scenarios.
The second observation is that the position of the modified Taylor curve may vary
considerably depending on the initial state of the economy. The situation in February 2008
was much more favourable than in February 2009. The point in Figure 6 that corresponds to
February 2008 is not far from origin, while the point that corresponds to February 2009 is a
long way from origin and represents substantial sum of squares for the inflation-gap and
output-gap forecasts.
Assessing whether monetary policy has been efficient thus entails attempting to determine
whether monetary policy has been on the modified Taylor curve or not. The analysis is
therefore ex ante, which means that the starting point is the central bank's forecasts for
inflation and resource utilisation rather than the actual outcomes. In practice it is of course
difficult to perform a more precise analysis, it becomes rather a question of determining to
what extent monetary policy has been clearly inefficient in the sense that it is easy to find
another repo rate path that would stabilise inflation more without stabilising resource
utilisation less, or that would even stabilise both more. A factor that can make the analysis
even more complicated is if the central bank, apart from inflation and a measure of resource
utilisation, also includes other targets or limitations in its monetary policy deliberations. On
such conceivable factor is so-called interest-rate smoothing, in other words that the central
bank also chooses to even out the changes in the policy rate and ensure that they are made
in relatively small and regular steps, for example by 0.25 percentage points at a time. With
such a restriction, monetary policy will be located at a point above and to the right of the
modified Taylor curve. Alternatively, it can be said that an additional axis and thus an
additional dimension are required that correspond to the sum of squared changes in the
interest rate, so that the modified Taylor curve becomes a three-dimensional, bowl-shaped
Taylor surface.
A major difficulty in this analysis is that it may be unclear what is meant by stabilising the real
economy. From a monetary policy perspective it is the stabilisation of resource utilisation
rather than GDP growth that is relevant. This means stabilising resource utilisation around a
normal level. The problem is that resource utilisation can be measured in several ways. A
reasonable and commonly used measure of resource utilisation is the so-called output gap;
that is the difference between actual production and potential production. However, potential
production is not a magnitude that can be observed directly - it must be estimated. There is
considerable uncertainty, both theoretically and empirically, about the best way to estimate
and forecast potential production. The output gap shown in the figures is the so-called HP
gap, which has significant weaknesses.7 It is important and desirable from several points of
view that the Riksbank and other central banks develop better and clearer measures of
resource utilisation and potential production and that they publish their measurements and
forecasts. Such work is underway at the Riksbank.
Assuming, however, that we nevertheless conclude that monetary policy has not been clearly
inefficient in the sense that I described earlier, the next step is to focus on what combination
of the stabilisation of inflation and the real economy the central bank actually selected or, in
other words, which of all the possible points on the Taylor curve the central bank selected.
There are many different efficient monetary policy alternatives to choose between every time
a monetary policy decision is made. But did the central bank make a good choice? In the
event of a conflict between stabilising inflation and stabilising the real economy, did the
combination chosen by the central bank represent a reasonable balance between the two?
Did the central bank attach reasonable importance to the stabilisation of the real economy in
relation to the stabilisation of inflation?
In the literature, as in the case of the forecast loss function I presented earlier, the constant
that is the relative weight that the central bank gives to the stabilisation of the real economy
in relation to the stabilisation of inflation is usually denoted by the Greek letter lambda, λ. In
Figure 3, we can show the forecast loss function with the help of iso-loss curves for
combinations of sums of squared inflation-gap and output-gap forecasts that generate
equally large losses. Such iso-loss curves are in this case downward-sloping, straight lines
with a slope of 1/λ, the reciprocal of lambda. Iso-loss lines closer to origin correspond to
lower losses. The ideal, but normally unattainable, situation would be an iso-loss line at the
origin, which represents a loss of zero and means that the forecast for inflation is exactly on
target and that the forecast for resource utilisation is exactly equal to the normal level.
However, the modified Taylor curve shows the minimum sums of squares that are possible
on each decision-making occasion. The best monetary policy therefore entails selecting a
point on the Taylor curve so that the iso-loss line for the loss function is as close to origin as
possible. This is the point at which the iso-loss line is a tangent to the Taylor curve. Figure 3
shows an iso-loss line that is a tangent to the Taylor curve at point B. For the given lambda,
which determines the slope of the iso-loss lines, this point thus represents a well-balanced
monetary policy.
As I have emphasised, the position of the Taylor curve depends on the initial state of the
economy at the time the decision is made. In a situation in which it is more difficult to
stabilise resource utilisation, the Taylor curve will be closer to the vertical axis than to the
horizontal axis, for example like the dashed curve above point A. The point of tangency for
an iso-loss line with the same slope, which represents a well-balanced monetary policy, will
then be at D. In a situation in which it is more difficult to stabilise inflation, the Taylor curve
will instead be closer to the horizontal axis than to the vertical axis, for example like the
dashed curve to the right of point C. The point of tangency for an iso-loss line with the same
slope will then be at E.
A central bank that has a low numerical value for lambda, that is a lower weight placed on
the stability of the real economy, has steeper iso-loss lines. For a central bank with such a
lambda, the point of contact with a given Taylor curve for a given decision-making situation
will be higher up to the left in the figure, for example at point A, where the Taylor curve is
steeper. The inflation forecast will then be closer to the target, but the forecast for resource
utilisation will deviate more from the normal level. A central bank with a high numerical value
for lambda, that is a high weight placed on the stability of the real economy, has iso-loss
lines that are flatter. In the case of such a lambda, the point of contact will be lower down to
the left, for example at point C, where the Taylor curve is flatter. The inflation forecast will
then be further from the target while the forecast for resource utilisation will be closer to a
normal level.
It is of course not so easy to determine what a reasonably large lambda should be for a
central bank. Neither the Riksbank nor other central banks have yet announced whether they
apply a specific lambda and if so what this lambda is. In those cases where the decisions are
made by a committee made up of several members, as at the Riksbank, it is also possible
that different members attach different degrees of importance to the stability of the real
economy. A question that then arises is how these different views should be aggregated. Nor
is it entirely certain that monetary policy decision-makers are even clearly aware of the
importance they attach to stability compared with the stability of inflation.
If the central bank's forecasts are only based on a known model, like the Riksbank's main
model Ramses, it is possible to determine the position of the modified Taylor curve and
which point the Riksbank's main scenario in each Monetary Policy report corresponds to.8 In
practice, the Riksbank's forecasts and those of other central banks are based on several
different models and a great deal of judgement. This means that it is not quite as easy to
determine the position of the modified Taylor curve. It is still possible, however, to enter the
position of various forecast alternatives on the graph and at least assess whether the
forecast is extreme in any respect with regard to the deviation of inflation from the target and
the deviation of resource utilisation from the normal level. As yet, it is in practice mainly a
question of whether the choice of interest rate path was extreme in either direction in the
sense that the central bank gave considerable or very little relative weight to the stability of
the real economy. Another interesting aspect to investigate is whether the weight attached to
the stability of the real economy actually has been constant over time. If it has not been
constant then the central bank's loss function has not been consistent over time, or it is more
complicated than the quadratic loss function that I have discussed here and that is regarded
as normal and reasonable in the literature on monetary policy.
Figures 4 and 6 can be studied more closely in the light of this discussion. In Figure 4 we can
see that the main scenario for February 2008 leads to resource utilisation being stabilised
rather well, but that the CPI forecast is a little high in relation to the target of 2 per cent. In
Figure 6, we can also see that the sum of squares for the output-gap forecast is much
smaller than for the inflation-gap forecast. A higher repo rate path should have been able to
push down both the inflation forecast and the forecast for resource utilisation. This would
have reduced the sum of squares for the inflation-gap forecast but not increased the sum of
squares for the output-gap forecast by very much. This would move the point for February
2008 in Figure 6 to the left and somewhat upwards. Unless the iso-loss line for the forecast
loss function is very flat, that is unless the weight given to the stabilisation of resource
utilisation is very high, this should give rise