sou · GQB3118d4 · 2003-01-01 00:00:00 — sou 2002 118 d4 REVISIONS OF SWEDISH NATIONAL ACCOUNTS 1980–1998 and an International Comparison REVISIONS OF SWEDISH NATIONAL ACCOUNTS 1980–1998 and an International Comparison A revision generally augments a preliminary growth rate. The revision distributions are skew, often with fat tails of outliers. For some Swedish variables, including GDP, revisions are corre- lated with the business cycle. This is also true of most European GDP revisions. Growth rates are revised upwards in upturns and downwards in downturns, and this also results in a tendency to- ward bimodality in the frequency distributions. We identify where in the accounts the greatest benefits from increased reliability may be achieved. In the international comparison has the small- est revisions. This study is part of an investigation of the quality of economic statistics ordered by the Swedish Commission on the Review of Economic Statistics. The study analyzes revisions of the () during the period 1980–1998, and its major components according to the expenditure approach. The Swedish data are compared to GDP revisions in 11 countries that have agreed to take part in the investigation. 1.1Small revisions desirable, but not an aim per se 1.2Revisions as a source of unreliability 1.4Preliminary figures vs. 1.5Outline of this study 2.1Revisions of National Accounts in 3.Revisions of Swedish Annual Accounts 3.1.1Frequency distributions of revisions 3.1.3Have revisions decreased? 3.1.4Conclusions so far Tables 3.2.1Comments on diagrams and tables Tables 3.3.1Comments on diagrams and tables Tables 3.4Discussion of the results 4.Revisions of Swedish Quarterly Accounts in 4.1Frequency distributions of revisions 4.3Have revisions decreased? 4.4Discussion of the results as compared to annual data Diagrams and tables 5.International Comparison of Revisions of Annual GDP Figures 5.1Frequency distributions of revisions 5.3Have revisions decreased? 5.4Comments from national statistical offices Diagrams and tables 6.1Information that should accompany preliminary fig- ures and some suggestions Appendix A: Time series of preliminary and final figures Appendix B: Publication Dates National Accounts are the most popular device in macroeconomic analysis. This is because they are designed to offer an internally consistent picture of the entire economy. Their weak point is inac- curacy. It is no trivial task to classify the data collected from soci- ety into the precise framework of the accounts, and often the fig- ures are approximations. A major cause of unreliability stems from the need to revise already published figures. National Ac- counts started in many countries half a century ago, and their inac- curacy has been the main topic of critical voices ever since. Forecasters, analysts and planners, whether using econometric or judgemental methods, but whose activity is forward-looking, need early figures. figures, although they may be more accu- rate, may have lost their relevance when a decision has to be made. However, if the final figure gives an essentially different picture of the economy, the decision may be seriously sub-optimal. Unreli- ability of base data may be incorporated in the decision, but then, too, it would carry an extra cost, leading to a welfare loss, as com- pared to the ideal situation of exact early data. Quoting (1969, p.3):(the revisions) may be considered a measure of the price, in terms of accuracy, of up-to-date GNP statistics. On the other hand, not revising figures that are known to be wrong is even more deceptive and requires even larger in decisions, thus carrying an even higher social cost. European GDP forecast errors are studied in and Barot (2000). For the root mean squared error of one-year-ahead GDP forecasts for the period 1971–1997 is 1.6 percentage points, but the same measure for revisions is 0.7 pct. A forecaster can be expected to be able to forecast revisions, which means that the revision figure forms a lower limit for how accurate a forecast can be made. Since all errors are never detected it is an exaggeration to say that half of the forecast error is due to inaccurate data. Inaccurate and inconsistent data is an equally serious problem for analysts, and especially for econometricians. Our study only focuses on the disadvantage of working with data of different vin- tages, the disadvantage being expressed by revisions. 1.1Small revisions desirable but not an aim per se In order to achieve timeliness and punctuality early statistical fig- ures are published as preliminary information that is eventually re- vised when more information becomes available. The statistical characteristics of these revisions are measures, albeit poor, of the reliability of preliminary figures. This is the aspect of quality that we are investigating in this study on revisions, measured as the dif- between final and preliminary growth rates. It is important not to mix the concept of general statistical quality and small revi- sions. One can never be sure that a revised figure really is more relevant and accurate than the preliminary one. In fact, there is plenty of evidence of revisions that introduce more error than pre- sent in the preliminary figure. Furthermore, (1995) warns for equating size of revisions and quality: an improvement in the current estimates results in a permanent decrease in revision size, while an improvement in the latest available estimates results in a permanent increase in revision size. Furthermore, revisions are to some extent an ethical matter. Large revisions can be a sign of diligent statisticians who do their utmost to find errors in the data and who are brave enough to admit that an early figure was wrong, and vice versa for the case of small revisions. A study of revisions can provide measures of the nuisance a consumer of Na- tional Accounts experiences when the figure, trusted to be correct, changes. The consumer is then at least informed of the revision to be expected. Another aim of the present study is to expose the shortcomings of the statistical production process and, if possible, to point out at least the most urgent needs of improvement, which ultimately would increase the reliability of the statistics. 1.2Revisions as a source of unreliability Preliminary figures often have to be based on sample estimates that are revised when total annual account data become available. In this case quality measures like standard errors can easily be calculated. However, a much larger source of error is of another type. In Bark- lem (2000) the following sources are given1: 1 See (1953), which contains an interesting and still up-to-date discussion of error sources in National Accounts. Model assumption errors. All of these afflict final figures. For none of them can we easily produce some numerical measure of reliability. For different reasons not all statistics are revised. The Consump- tion Price Index (CPI) is used as a base for commercial contracts and revisions would cause serious damage in the market where many contracts would have to be renegotiated. According to a gov- ernment decision CPI cannot be revised. Business tendency sur- veys provide snapshot pictures of the economy using ordinal data that are never revised. Also, there are variables that do not need any revision because the first figure published is accurate. Stock market data is an example. Deals on the stock market are registered as they occur and both price and quantity are correct. Furthermore, such data are generally delivered on-line. The high quality of some fi- nancial data has made them popular in econometric research. Pro- ducers of National Accounts must accept the challenge and look for ways to increase the timeliness and reliability of statistics on the real economy. In order to be able to work in scale-free metrics we decided to use annual percentage (pct.) change instead of levels. A revision is de- fined as the difference between a final and a preliminary growth fig- ure. For Swedish annual data for year t, preliminary means t +12, which is the time of publication of the complete quarterly ac- counts for year t. Final is defined as of t + 2. This is the time when the first revised annual accounts are published. Thus revisions after t + 2 are ignored in this study, although both meth- odological and general revisions are made much later. By choosing t + 2 we try to avoid, as much as possible, revisions that are due to changes in definitions or methods. Leaving the final figure open- ended would further obstruct the meaning and comparability of 2 Swedish preliminary quarterly data have been published 70–85 days after the quarter has expired, during the period studied here. The publication delay varies between quarters and has become shorter in the 1990s. the measures to be used. All variables are related (in pct.) to GDP in tables and diagrams in order to indicate their relative magnitude. But it would be important to know something about these later revisions. According to (1992) they are not ignorable, see Table 1.1. Table 1.1 Effect of general revisions (%) 1981:1 – 1989:4 Mean (Bias) m(ABS)1) rm(ABS)2) 1)Mean absolute value of revisions 2)r = relative Source: (1992), Table 5.1 In the 1980s general revisions were still performed once in a dec- ade, but the frequency seems to increase; a general revision is under way when this is being written, the previous one was done as re- cently as 1998. General revisions should be of interest to economic historians, analysts of structure and econometricians. The international data, analyzed in Chapter 5 were supposed to follow this publication scheme as closely as possible, but some de- viations had to be accepted, because publishing schemes differ be- tween countries. In some countries t + 3 comes closest to what we here mean by final, and some studies, to which will be referred in the text, use the last available estimate. In the Swedish study, we are mimicking the situation when a user of National Accounts receives a preliminary figure. Here, revi- sions of price data are directly copied from published an- nual growth rates of non-seasonally and working day adjusted data. In the case of current price data growth rates are calculated from data in levels, as known at the time, and deflator revisions are then obtained implicitly. The detailed revision scheme of quarterly data can best be shown in a table: SOU 2002:118 Bilaga 4 Table 1.2 Publication and revision chronology for quarterly data of year t Quarters in t : The entries in the row Preliminary show when that quarters pre- liminary figure is published. The next three rows present the dates when these preliminary figures are revised. The preliminary figure for the first quarter is revised three times, that of the second quar- ter twice, the third quarter figure once and the preliminary figure for the fourth quarter is never revised in the quarterly accounts. It will normally be revised only when annual accounts are published, as are again all the other quarterly figures, but then, as a rule, the intra-year pattern will not be changed. Our final figure is here, as for annual figures, from the annual accounts of t + 2. Revisions 1–3 have not been studied here. A fast and convenient way of conveying to the reader an impres- sion of how revisions behave is to present them as histograms, i.e. If historical revisions have any relationship to future ones, a histogram can help a user of statistics to assess the reliability of a preliminary figure. A histogram, in a concentrated form, says much about location, spread, non-normality and out- liers. For a deeper understanding this visual overview is here supple- mented by a number of statistical characteristics. They have been collected into tables that have been made almost identical for each set of data studied so as to simplify comparisons. Like in other studies in this field we are interested in knowing if there is a systematic tendency for final figures to increase or de- crease as compared to preliminary values. This is called bias. There are two statistical measures for bias here: the mean and the median (Med)3 of revisions. Both are necessary because the histograms show that the distributions may be skew, in which case the mean is not a good measure of bias. 3 The figure that divides the observations into two equal parts. Another feature of revisions that could seriously hamper the usefulness of preliminary data is high dispersion, which translates into uncertainty. A practical measure that takes into account both bias and spread is the mean of the absolute value of revisions m(ABS), i.e. the mean of revisions, neglecting their sign4. The standard deviation (S)5 measures only dispersion (not bias), as does the distribution-free range (R), which is the largest () minus the smallest (Min) value. If the bias is positive we have added bias to dispersion into one figure (Mean+S6 and Med+R), and furthermore m(ABS) has been added to these two to form one measure of bias and dispersion. This has been done so as to help the reader to form an overall opinion of the reliability of prelimi- nary figures of different variables. The next four characteristics in the first table of every set of variables studied are: (i)R(F,F-P), which is the coefficient of correlation between revision (F-P) and (final) growth (F). (ii)The row marked Sign shows the number of times the sign of the preliminary and final figure differs. (iii)Analogous to (ii) is a figure on how many times the two time series have disagreed on acceleration or deceleration (Ac/Dc)7. (iv)Impact is the product of the mean absolute revision, m(ABS), and the share of the variable of the total, GDP. The characteristics (i) – (iii) are important for those who monitor the business cycle. Positive correlation with the business cycle (growth rate) tells the user that a bias in a preliminary estimate may be larger than average in a period of strong growth, and smaller, or even negative in a sharp decline. Sign and Ac/Dc show if you can rely on the direction of the preliminary growth signal staying the same after a revision, or if the preliminary and final figures seem to indicate different stages of a business cycle. The impact on GDP (iv) indicates to the producer of the statistics where improved measurement of preliminary data would help the most to reduce revisions of GDP. 4 m(ABS) = (1/N) | final – preliminary | , where N is the number of observations. 5 As all parametric characteristics, the standard deviation is not a good measure of dispersion for skew distributions. 6 It can be shown that for positive mean and standard deviation: Mean + S < Root Revision 1.25m(ABS). 7 Acceleration = change in change when positive, deceleration when negative. 1.4Preliminary figures vs. Following McNees (1989) one can say that: the process of esti- mating GNP starts with forecasts made many years before a quar- ter has begun and continues for years after it has ended as prelimi- nary estimates are repeatedly revised. A preliminary figure of a variable can be considered as an estimate, or more generally, since we will never know the true figure of GDP, all published figures of GDP are estimates, be they produced ex ante or ex post8. the years a standardized technique has been developed for assessing the accuracy of forecasts; see e.g. Holden and Peel (1982) and and Barot (2000). In our study these standards have been followed only to a limited extent. We use the mean absolute value of revisions, instead of root mean square forecast error, common in forecast studies. We do not test for rationality, efficiency, etc. The distributions presented in the following chapters do not look like symmetric normal distributions, and this further complicates test- ing. Forecasting studies assume that the eventual outcome is an exact figure that the forecast is trying to hit. But in National Accounts there is no true outcome and any measure of accuracy based on the closeness of a preliminary figure to an outcome would only measure the distance between two estimates. In fact this applies not only to preliminary figures, but to GDP forecasts alike. 1.5Outline of this study In Chapter 2 we start by reviewing some previous studies of revi- sions of National Accounts in and elsewhere. In Chapter 3 Swedish annual accounts are studied both in and current prices. The implicit deflator revisions are also analyzed. The GDP components to be studied are: Private Consumption, Government Consumption (Central and Local), Gross Fixed Capital Formation (henceforth for short: Investments), Change in Inventories (hence- forth: Inventories), Exports (goods and services), Imports (goods and services), and the total, (). In Chapter 4 the revisions of these same variables, now in quarterly observations are analyzed, and only in prices. In Chapter 5 the Swedish revisions are compared to GDP revisions of annual data from the following countries: , , Den- 8 (1969) finds similarities between preliminary figures and extrapolative forecasts. mark, , , , , , New Zea- land, UK and . This chapter is written so that it can be read separately from the two preceding chapters on Swedish data. And finally, in Chapter 6 the results of this study are discussed and some suggestions are made for how to follow-up our study. In all chapters the data cover the period 1980–1998. Appendix A presents the preliminary and final figures from which the revisions and their histograms as well as their statistical characteristics were calculated. 2.1Revisions of National Accounts in We have found two studies that have relevance here: (1992) and (1992). Both reports concern quarterly National Accounts, and annual growth rates. is a separate report, while Eklöfs is a chapter in a Ph. Neither study mentions any previous investigation of Accounts revi- sions, which we have understood as an implicit statement that there are no earlier studies published on this subject9. (1992) starts by studying the statistical discrepancies that arise because data compiled from expenditure and production sources do not initially result in the same estimate. He finds that, in the harmonization process, the discrepancies resulted in an up- ward revision for GDP of in the 1970s and 1980s. This is an important measure of unreliability that we have not had access to in this study, because these have not been saved in a consistent and retrievable form in the 1990s. We return to this matter in Section 6.1. We have not studied the production side of the accounts. In Table 2.1 some results from (1992) are presented. It provides further information on revisions that is not available from our study. Some years from the 1970s are included and three revi- sions are Table 2.1 Bias and m(ABS) in GDP growth revisions in the 1970 and the 1980s m(ABS) m(ABS) m(ABS) m(ABS) Source: (1992, Table 5.1) 9 In most statistical annual and in some quarterly publications from the 1970s and 1980s the reasons to, the problems with and the effects of revisions are commented on. studied, see Table 1.2. and Section 1.3 for the revision dates. Comparing the two decades in Table 2.1 we see that according to all measures revisions became smaller in the 1980s. The biases were sizable in the 1970s. One would expect that the need to revise the same figure would become smaller over time. This would imply that the figures decrease from the left to the right. This is true for the bias in the 1970s, but not in the 1980s where the revision of a figure already revised once after 6 months, is larger than for the preliminary figure. Also, both m(ABS) and relative rm(ABS) in- crease from one revision to the next in the 1970s. Note that biases are positive for both decades, meaning that preliminary growth fig- ures have been too low. (1992) also contains tables with the same measures as in Table 2.1 for the Production Accounts and an international comparison of GDP revisions. However, in the international study the variables being revised are quarter on quarter changes of sea- sonally adjusted data, so that the results cannot be compared to the results of our study. Biases are positive (underestimation). The eight countries studied were: , , , , Fran- ce, Germany10, UK and . Ranked according to smallest bias and m(ABS), receives the highest rank, followed by and , in that order. According to relative measures and get the same ranking, followed by and , also with the same ranking. reports that the National Accounts are built up from pieces that are produced by different departments of Statistics Sweden11. Some of the pieces, he continues, are notoriously unreli- able as preliminary statistics and have to be corrected judgmentally by the National Accounts department. Since the main task for that unit, he argues, should be to do the compilation of the data into the current SNA framework, and this being a huge work, correct- ing mistakes by others should be discontinued, and the responsibil- ity transferred to those who compile the bits and pieces from which the accounts are formed. also calls for more open explanations to why revisions are made. (1992) considers revisions of quarterly data consisting of both expenditures and of incomes, the former in , and the latter in current prices. The periods studied were 1974–1977 and 1984–1987. Five separate revisions are studied. The main question asked is whether the preliminary figure is a better forecast than 10Federal Republic of . 11In (1995) the graphic expression built up as a mosaic is used. some naïve or model-based artifacts? The answer is in the affirma- tive12. Both for the domestic data and for some international fig- ures concludes that preliminary data are more accurate than the automatic extrapolations he generates. He also draws attention to some large errors found in the data, some of them typos. The errors are never commented on in revised issues of the tables. It is easy to agree with him that statisticians should always honestly re- port mistakes and tell the consumer what went wrong. He also finds that some GDP revisions correlate positively with growth and that m(ABS) for some variables is intolerably high, particularly in a relative sense, i.e. as compared to the same measure for the variable being revised. One of the first to criticize National Accounts for inconsistency and poor empirical applicability, was (1950). since researchers have encountered these problems that have not been solved yet. They have looked at inconsistencies of GDP data based on expenditures, production and income, on revisions due to changes in definitions, base years of price and other revisions. But all efforts boil down to the same issue. We can com- pare different estimates, such as the difference between preliminary and final figures. But comparing two estimates says or noth- ing about the closeness of these estimates to the underlying, true figure. In this section we comment on national studies on revisions in , , UK and . Some academic articles will also be highlighted. We have been informed that studies of revi- sions have been conducted in some countries not mentioned here, but we have not been able to find any published reports, neither were any such reports mentioned by the contact persons who de- livered the international data analyzed in Chapter 5. When reading the next paragraphs the reader should be aware of the differences in measuring revisions in different countries. 12 (1969) arrives at the same conclusion for U.S. Three studies by the Australian () should be mentioned here. ABS(1997) covers the three measures of GDP: expenditures, production and income revisions for the period 1984–1993. The data are quarterly and measure quarterly change, not annual as in the present study. Initial estimates of Private Con- sumption were found to be close to final figures.