sou · GQB3118d4 · 2003-01-01 00:00:00 — They turn up sporadically and most frequently in 3–4 variables in the annual data in Table 3.1.3. In quarterly data large revisions are at least as com- mon as smaller ones. Some variables have very few black numbers; Exports of Services have 9 and Imports of Services only 3. In the 1990s Central Government Consumption had only 6 and Invest- ments 7 black numbers. The variables with the fewest large revi- sions were the heavy ones: Private Consumption with 12 large revi- sions and GDP with 21. For GDP, 21 is a high frequency of revisions of at least one per- centage point, considering that the average annual growth rate of the Swedish economy is close to . Table 4.3 again shows how difficult it must have been for an ana- lyst to assess the state of the economy, especially GDP, at the turning point (down) in the beginning of 1981. The preliminary figure for 1980:4 was 1.6 pct. points too low, 1981: was 1.1 pct. points too high and 1981:2 was 2.7 (!) points too low. Al- though the general picture of the 1990s crisis that preliminary fig- ures provided is not misleading, in single points a forecaster could have gone wrong. The upturn 1994–1995 was systematically under- stated in preliminary figures, which in the uncertain situation in the beginning of the upturn might have postponed the growth signals, and after that understated the boom, see also Figure 4.14. Fortu- nately, this time messy data did not cause bad forecasts by the Na- tional . In this chapter we have added a new table, Table 4.4 showing re- vision characteristics for each quarter separately. As mentioned in Section 4.1 the number of times each quarterly figure is revised var- ies and it was thought that this might show up as different charac- teristics for different quarters. Not very much of the kind was found. The bias seems to be larger in the beginning of the year, ex- cept for GDP for which the bias and m(ABS) are largest for the 3rd quarter. The period was split into the 1980s and the 1990s (not shown in the table). In the 1980s GDP had the largest m(ABS) in the 1st quarter, while in the 90s the 4th quarter showed the largest m(ABS). This can have to do with a change in accounting routines. The GDP revisions are strongly correlated with growth only in the 1st and 4th quarters. The reason to this is not clear. The high values of bias, m(ABS) and R(F,F-P) for Investments in the 2nd quarter can be traced to lack of other than expectation (plan) data, as dis- cussed in Section 4.1. 5International Comparison of Revisions of Annual GDP Figures The respective national statistical offices have kindly delivered all data studied here22. The countries whose GDP revisions are studied are: (AUS), (CAN), (DEN), (FIN), (FRA), (GER), (NL), Nor- way (), (NZ), (SWE), the (UK), and the of (). Here we will use the devices of the previous chapters to show the character- istic features of revisions in the countries participating in this stu- dy. Please keep in mind what was said in the introduction. ranking that can be derived from the frequency distributions and tables is not to be equated to a ranking of quality. The statistical description is simply intended to show how reliable preliminary figures are, assuming the final ones are more accurate. As compared to the previous chapters there is an additional complication here. Preliminary, and especially final figures are not published on the same dates in all cases. For some countries these dates have varied over the period studied. For some, we do not know the dates. Still, by and large the figures should be comparable. A scheme of publi- cation dates can be found in Appendix B. 5.1Frequency distributions of revisions The histograms have all the features one has become familiar with in the previous chapters. Note that the horizontal scales are differ- ent here, because the spread in the international GDP data is much smaller than for the Swedish disaggregated variables in Chapters 3– 4. The revisions generally have underestimation bias, the distribu- tions tend to have skewness, kurtosis and bimodality. The law of large numbers seems to be working here, too. The distributions of small countries (, , , ) have slightly larger spread than that of larger economies. Possibly, there are two exceptions: The rather large spread of GDP revisions in that has a resemblance to smaller countries, and 22 The preliminary and final series used in this study can be found in Table A5 in Appendix A. distribution that may be better thought of as belonging to the group of larger countries. Outliers are common. Even some coun- tries with generally small revisions, notably and the UK, have outliers. In Table 5.13 all revisions have been pooled as if the statistics had been made by the same agency. We see that with 238 data points the distribution comes closer to the Gaussian distribution, but some skewness still remains. The characteristics shown in Table 5.1 resemble those of Swedish price annual data, see Table 3.1.1. Only and Ger- many have means (bias) of zero, but if bias is measured by the more appropriate median, only the Canadian preliminary GDP data are unbiased. The average bias is between 0.2 and 0.3 percentage points, depending on which measure is used. For Australian, Fin- nish, Dutch and Swedish preliminary figures, one would be better off adding 0.4 pct. points to the preliminary figures. The standard deviation does not vary much around half a pct. point, whereas measured by the outlier-sensitive range the spread varies between 1.3 pct. points for and 3 pct. The last row shows the coefficient of correlation R(F,F-P) between revi- sions and final growth figures. Considerable correlation is found in seven European countries: , , , , , and the UK. But for , , , and there is only low correlation or none at all. The divide goes between Europe and the rest of the countries in this sample, the only exception being . This may indicate a difference in compilation methods. The figures in the shaded area of Table 5.1 give overall measures of the revisions, meaning that they comprise both bias and spread. The first is the Mean Absolute Value of Revisions (m(ABS)), a measure often employed in comparisons of this kind. All countries are quite close to the average of half a pct. The smallest val- ues are achieved by and (0.3). The other two statis- tics that measure both bias, which here is non-negative, and spread are the sum of the mean and the standard deviation and the sum of the median and the range. According to the first measure, again gets the smallest value and according to the second crite- rion, . If one adds all three figures in the shaded area there is a clear divide between a group of four countries: , , and the with small bias and spread, and the rest of the countries. In the last two rows the business cycle description of prelimi- nary figures is compared to that of final figures. The row marked Sign shows how many times preliminary and final figures have had different signs. For most countries this has not happened during the two decades studied here, but for some small countries differ- ent signs have occurred once (and ) or twice (Den- mark and ). The next row (Ac/Dc) shows the number of times that preliminary figures indicate faster growth, but final figures say lower growth, or vice versa. In most countries this has happened once, but never in , or and twice in . For the years where these discrepancies occurred the reader is referred to Table A5 in Appendix A. 5.3Have revisions decreased? In rows two and three of Table 5.2 the mean absolute value of revi- sions is calculated separately for two periods: the 1980s (m(ABS)80) and the 1990s (m(ABS)90), in order to investigate if there has been any change in revisions during these two decades. No tests have been performed because of the small sample and the irregular distributions, but a decrease in m(ABS) of size 0.2 or more, has been marked in bold figures. The and Nor- way have been able to lower the average absolute amount of revi- sion, still not achieving the level of and , whereas UK has moved from a group of countries with the largest m(ABS) in the 1980s to the lowest value in the 1990s. Note that in some countries the spread of revisions has increased in magnitude. Preliminary figures of GDP growth may have to be revised more if the reference variable varies a lot, or when an economy grows very fast. In the latter case there will be more structural changes in the economy than in a case of a low-growth, or even a stagnant economy. For that reason we have also calculated the rela- tive mean (rMean) and relative standard deviation (rS) in Table 5.2. The ratio of the mean of revisions and the average growth of GDP gives the relative bias of GDP revisions. For , the value 0.1 means that preliminary growth figures are on average too low, so that a growth figure of will on average be revised to . The smallest value of relative standard deviation, 0.2 pct. points, is achieved by , , , , , UK and . The standard deviation S measure the un- certainty associated with a preliminary figure and rS relates that dispersion measures to the same uncertainty measure of the vari- able being revised (growth). Note that has low revision spread in Table 5.1, but in relation to growth variations, the uncer- tainty of preliminary figures is rather high, while the opposite is true for . The relative mean absolute value of revisions (rm(ABS)) also measure relative uncertainty. still has the lowest figure 0.1 (0.3 for m(ABS)). For some countries like and the uncertainty of preliminary figures is much lower when related to the uncertainty associated with the (volatile) growth rate in the respective country. The decrease in m(ABS) in the 1990s as compared to the 1980s for the and is also reflected in the relative un- certainty measure rm(ABS). With just one exception m(ABS) and rm(ABS) decreased between the two decades. For no de- crease was registered, but comparing our estimate rm(ABS) = 0.2 (Table 5.3) to the one reported by Morgenstern (1963, p. 270) for the period 1947–1958, rm(ABS) = 0.4, the decrease is substantial. These relative figures should be interpreted with caution. One doesnt know if large or increased variance in final growth figures is due to the phenomenon itself being or becoming more volatile or if it is a sign of low or deteriorating precision in final figures. For completeness, in Table 5.3 we give the time series of revi- sions in this study. These will show when the outliers (marked in bold) occurred that can be seen in the diagrams. Table 5.3 also raises another question. For some countries there are distinct runs of revisions with the same sign. The same was observed in some Swedish data, although this feature is more distinct here. A nega- tive revision is more likely to be followed by another negative revi- sion, a positive by another positive, a large by another large, etc. This is, like bias, a systematic error. 5.4Comments from national statistical offices A report on the results in this chapter was distributed to all partici- pating offices. The respondents were asked to answer the following questions (abbreviated here): 1.Dates (year and month) when the preliminary and the first final figures are released? 2.Any comments on revision outliers? 3.Do you know why the countries form two groups according to correlation between revision and final growth? 4.Why do we see runs of positive and negative revisions? 5.Is it pure coincidence that some revisions have decreased in the 90s as compared to the 80s? If not, what has been done to achieve this? 6.Generally, are the figures and procedures correct? Can you suggest some other procedures? Do you think there may be institutional and/or organisational explanations to some of the results? Three offices, those of , and replied and their answers were (abbreviated, not citation): 1.The first final figures were published in t+2, except for 1994, 1997 and 1998 when they were published in De- cember t+2. 2.Outliers in 1982, 1991 and 1993. In 1982, preliminary fig- ures on construction and industrial production proved un- reliable. As a result of the critique a monthly indicator of GDP was launched. was hit by an outright depres- sion in 1991–1993 and GDP decreased by a total of . At first the depth of the crisis was not realised. The wave of bankruptcies resulted in too high adjustments of sample es- timates to the population level and vice versa at the end of the crisis. 4.The runs of positive and negative revisions may be related to cyclical turning points. 5.The primary data may have improved. The amount of data has increased. 6.The analysis is well done. Naturally, practices vary from country to country. 1.Preliminary figures were released in in the period 1980–1984 and in April for the rest of the data. 2.Final figures for year t are released in t+3 in NL. 4.There are only three negative revisions, two of them in a row. 5.All three outliers occurred in the 1980s and this may cause the smaller m(ABS) in the 1990s. delivered the most comprehen- sive reply. It is remarked that any comments on the sturdy are hampered by the framework of this study being different from U.S. studies of revisions in many respects: (a)U.S. studies mostly concern quarterly, not annual data, (b)The final figure that is used in this study is from t + 2, which is less interesting, because it is in the middle of a three year stream of annual frequency data, and (c)uses a different strategy to substitute for missing data. Referring to the report Fixler and Grimm (2002), we are ad- vised to be careful in using words like error and reliability when talking about revisions. Another general comment concerns the introduction of chain price indexes. They have been found to remove a major source of systematic error, see Fixler and Grimm (2002), p.22. Countries that used fixed base year indexes for most of, or for the whole period can be expected to have biased revi- sions. has seen evidence of runs of positive/negative in revi- sions of quarterly U.S. GDP data have been found to be smoother in the 1990s than in the 1980s, but this could change when a new input/output matrix for the year 1997 will be available late in 2002. This could explain why the relative mean absolute value of revisions rm(ABS) is the same for the 1990s and the 1980s, although m(ABS) de- creased. has discontinued to use relative measures in revision reports. Among the final comments is one concerning missing births and deaths of firms, see the comments from . Despite a reminder, no further replies were received. A complete table of the exact publication dates of the countries involved might have been valuable to some readers. In Appendix B the dates are presented as known. Even more interesting would have been re- ports on revisions in other countries. Studies of revisions, at least for internal use, probably exist in many more countries. 6 Conclusions and Suggestions 6.1Information that should accompany preliminary figures and some suggestions Ten years have passed since a comprehensive study on revisions of National Accounts was published in ((1992)). Not many countries make any revision studies at all. in has hitherto published no less than 14 such reports. The Office for National Statistics in UK has also published some reports, as has , see and (1997). In the U.S. and the U.K. academic researchers have found an interest in this area and have made some studies. Various inter- national studies were highlighted in Section 2.2. A public with high demands on statistical quality requires regular reports on revisions. It is the hope of the authors of this report that it could serve as a starting point for regular revision studies of National Accounts and the underlying statistics in all countries. Note, however, that we have studied only the expenditure side of the accounts. A complete study should include the production and income sides as well. It would be an advantage for international comparisons if some standard procedures would be adopted internationally. We have seen in this study that measures differ so much between countries that meaningful comparisons are sometimes impossible. By show- ing long time series of preliminary and final figures on the website students would be attracted to make their own studies, thus help- ing both the producers and the consumers of statistics. Some basic information on how much a preliminary figure is ex- pected to change, and if biased, in what direction and how much, should be attached to the very time series data when published. A minimum would be a histogram, and/or the mean or median and m(ABS) of revisions, all these being continuously updated. Also, it would be informative to the user of National Accounts if the sta- tistical office would publish the discrepancy that emerges as the difference between the Expenditure and Production Accounts es- timates of GDP. In fact, this would be real quality information on reliability, not just comparing two estimates, as we have done in this study. The magnitude of this item measures something no- body knows where it belongs, if anywhere. Furthermore, the user would like to know what has happened to this discrepancy. Has it been added to (or subtracted from) the change in Inventories? What procedures have been followed in preliminary, quarterly ac- counts, vs. annual National Accounts? Ideally, revision studies would result in better statistics. How- ever, it is important to realize that the target is not to reduce revi- sions as such. They can readily be set to zero just by discontinuing all revisions, leaving all errors and ambiguities in the data. One might also mechanically improve the preliminary figures by util- izing the systematic features, bias and autocorrelation. In fact, many academic studies have shown that preliminary data could be brought closer to final figures through modeling; see e.g. et al. (1981) and Howrey (1984). Modeling is a second best solution, the first best being trying to find the source of large and especially systematic discrepancies be- tween preliminary and final figures. attention should be paid to making the best use of modern data transferring techniques so as to get more timely and accurate figures. Quoting (1995): Electronic data collection and transfer methods are beginning to improve data collection and editing of data. One of the most im- portant improvements can make to address the problems in this paper is the reengineering of the information technology sys- tem. Especially there seems to be a need to co-ordinate account- ing practices in companies and government agencies with the needs of National Accounts. The study also suggests where the greatest benefits are to be at- tained in the form of smaller revisions. Still, we know about the cost to implement them. A complete cost-benefit analysis of reducing uncertainty of preliminary estimates is still to be done. We stated in the introduction that those who forecast could be required to guess on the revisions that Statistics will make, and hence the spread of revisions form a lower limit to the accuracy of macroeconomic forecasts. The chain can be contin- ued two more steps. Accounts cannot be more accu- rate than the data that are fed into the accounts and that have been produced outside the National Accounts Unit. We like to repeat what was said in (1992) that those who produce the input data should be better aware of the use of that data in the National Accounts, and should carry the full responsibility for the accuracy of the figures delivered. Also, revision studies should be extended to these data sources. One more step could be added. The primary data in many cases are more or less black boxes to both producers and users of statis- tics. Wrong primary data can be corrected at the Statistics only in cases where the error is obvious and fairly easily detected, and even then at a considerable cost. Where necessary, accounting rules in companies, organizations and government agencies should be made to comply with the need of Statistics so that all necessary data are easily accessible. When this has been achieved the next step is to have the data automatically transferred online; (1996 and 2002) are examples of such attempts. Thus two benefits can be achieved from revision studies: better information to the users and improved statistics. Undocumented judgment that is so common in forecasting should be avoided in official statistics. Revisions were found to be positively biased (preliminary figures underestimate) and their distributions were skew, often with thick tails. This is particularly the case for Swedish data, but also for most of the international GDP revisions. In Swedish data, some variables were revised so much and so erratically that the use of their preliminary figures can seriously be questioned. Exports and Imports of Services in the 1980s and Central Government Con- sumptions in the 1990s are cases in point. By and large the prelimi- nary GDP figures convey the same message of the business cycle as final data, but in some cases, notably in critical turning points the preliminary and the final statistics havent quite agreed, obstructing the task of the analyst and the forecaster. All these features can also be found in reports from other countries. A related phenomenon is the positive correlation between revi- sions and the business cycle. This could be due to old structures being projected forward in time, either by imputation where pre- liminary data are unavailable, or through a database that does not stay up to date in major turning points. An example is the birth of new firms in upturns and the deaths that occur in downturns, both being registered only when annual accounts are available. Another explanation is given by Mork (1987): Suppose for example that the available observations and other information indicate growth. By releasing an estimate of, say , the BEA23 can both signal strong growth and hedge against a potential embarrassment should the current signals turn out to be misleading. 23 , the producer of Accounts. The positive bias in price data can to some extent be explained by a tendency to underestimate price changes. The intro- duction of improved price will probably reduce this bias. According to (1992) the revisions have decreased in the 1980s as compared to the 1970s. In the international studies there is also some evidence of decreasing revisions from the early times of National Accounts, but again evidence exists for a continuation of that tendency in later decades. (1992) called for a database on revisions that is regu- larly updated. This study started by that database. All re- searchers are welcome to continue and build on the work reported here. Those who are looking for macroeconomic statistics of all vintages, Croushore and Stark (2001) is the reference for the U.S. and Egginton et al. (2002) for the U.K. As stated in the introduction, ours is a simple descriptive study. Few sophisticated statistical techniques have been applied. We have refrained from using tests that assume normal distributions for re- visions because some evidence speaks against that assumption. However, nonparametric tests could be applied. Also, the study has been entirely univariate in the sense that one variable has been studied at a time. It would also be interesting to try to compare the overall (multivariate) economic picture that preliminary data paint of the recent past as compared to the final picture of the Swedish economy, and to check the consistency between different vintages, as in Siklos (1996) and and Heravi (2002) for US data, and (2002) for U.K. We want to thank for initiating this project and for presenting constructive comments during the work.