(*) This is the unabridged version of a letter which appeared in the Sunday Times of 11 July 2004.
Well-directed and informed criticism is always something to be welcomed. That, after all, is the way institutions and individuals learn, and the way in which quality is improved and standards raised.
Over the past two weeks, Hogarth has levelled a number of criticisms at Statistics South Africa in general, and South Africa's statistician-general in particular (June 27 and July 4). Reflection on these criticisms, however, indicates that they are based on either misinformation or misunderstanding, and hence do not advance discussion on Stats SA's ongoing initiatives to improve the quality of data.
Hogarth's first comment ('What's a few billion between friends') raises three separate issues: the definition of unemployment; the calculation of the consumer price index (CPI); and measurement of sales in wholesale, retail and motor trade. In each area of discussion, Hogarth is mistaken: a cursory glance at Stats SA's labour force survey shows that unemployment is measured and reported on according to two definitions: the official definition, used by the International Labour Organisation for purposes of comparison between countries, which excludes what might be called 'discouraged job seekers'; and the expanded definition, which includes them. To suggest that Stats SA has somehow excluded 'discouraged job seekers' from its reporting on unemployment misunderstands the two definitions of unemployment, and misreads Stats SA's statistical releases on the labour market.
Hogarth then goes on to suggest that Stats SA 'brought about a fall in inflation' by discarding 'faulty data'. This comment is based on a misunderstanding of the dependency of CPI calculation on various data sources. When deficiencies in one of the data sources became apparent (rentals), a better source of information was sourced and substituted. This is common not only in statistical practice: after all, methodologies governing data gathering must always be subject to review, change and improvement. Without this, science easily descends to dogma.
Hogarth comments critically on Stats SA's introduction of an improved and expanded business register, used for drawing samples in economic surveys. Continuous improvement and expansion of a sampling frame, both to improve coverage and to take account of new developments in a dynamic economy, is a clear indication of dedication to continuous improvement in data quality. A logical consequence of this, which Hogarth appears to find mysterious, is more-accurate measurement of economic activity.
Hogarth's second comment ('And one for the road') seems to be based on a misunderstanding of the way in which national accounts are estimated, and gross domestic product (GDP) calculated. Quoting from the statistician-general's column in a rival newspaper, Hogarth seems to have missed the central points of that column: that quarterly GDP estimates are always preliminary, and are routinely revised on the basis of additional evidence that has become available by the time the subsequent quarter's estimates are released; and that benchmarking and rebasing are undertaken periodically as a matter of course, and are in line with international best practice. To suggest that there is something 'strange' in this is to seek conspiracy in what is both ordinary and transparent.
Finally, in a third comment ('Wrong again'), Hogarth completely misses the point in Stats SA's criticism of erroneous use of data in a South African Reserve Bank publication. Hogarth claims that, according to a report in a sister newspaper (Business Day), the statistician-general criticised employment figures published by the Bank as being 'too negative'. A reading of that report, however, reveals no reference to figures being 'too negative'. The report does set out Stats SA's critique of the way the Reserve Bank used certain data, particularly its attempt to compare data over 20 years which methodologically defies comparison. (A simple example: 1980 data on employment excluded the 'homeland' areas; data 20 years on included those areas. A simple comparison between 1980 and 2000 is therefore likely to mislead rather than inform).
The Business Day report referred to by Hogarth concludes by quoting the author of the Reserve Bank publication acknowledging that there were limitations in the data used, and 'that it would be wrong to conclude that employment in the formal sector had fallen over the 20 year period'. Hogarth has missed the point: this was precisely the basis of the statistician-general's criticism of the Reserve Bank publication - a criticism acknowledged by the author of the publication!
Perhaps Hogarth did not read as far as the final paragraph of the Business Day report, which renders the 'Wrong again' headline introducing Hogath's comments unusually pertinent.