Quality in short-term statistics : the Euro-SICS database of Eurostat.
Date: 2003-12-23
Institutions: Eurostat
Subjects: EU statistics, Eurostat, database, economic analysis, economic statistics, euro area, qualitative analysis, statistical method

N
O

I
T
I

D
E

3
0
0
2

Quality in Short-term
statistics: The Euro-SICS
database of Eurostat

E U R O P E A N
C O M M I S S I O N

 
Europe Direct is a service to help you find answers to your questions about the European Union

New freephone number:
00 800 6 7 8 9 10 11

A great deal of additional information on the European Union is available on the Internet.
It can be accessed through the Europa server (http://europa.eu.int).

Luxembourg: Office for Official Publications of the European Communities, 2003

ISBN 92-894-5388-5
ISSN 1725-4825

© European Communities, 2003

TABLE OF CONTENTS

1.

Introduction .......................................................................................................... 9

2. The Euro-SICS database ..................................................................................... 10

2.1.  General description of the Euro-SICS database.......................................... 10

2.2. Euro-SICS Metadata .................................................................................... 11

3. The necessity of a quality measure .......................................................................11

4. Relevance and completeness................................................................................ 12

4.1. Relevance and consultation ......................................................................... 13

4.2. Coverage and regular updating.................................................................... 13

4.3. Frequency .................................................................................................... 14

4.4. Length .......................................................................................................... 14

4.5. Missing values ............................................................................................. 15

5. Timeliness ............................................................................................................ 16

5.1. Concepts and definitions.............................................................................. 16

5.2. The choice thresholds .................................................................................. 17

5.3. Timeliness in the Euro-SICS database......................................................... 18

6. Accuracy .............................................................................................................. 18

6.1.  Irregular values and outliers........................................................................ 19

6.2. Revisions...................................................................................................... 20

Bibliography......................................................................................................... 21

(cid:52)(cid:88)(cid:68)(cid:79)(cid:76)(cid:87)(cid:92)(cid:3)(cid:76)(cid:81)(cid:3)(cid:54)(cid:75)(cid:82)(cid:85)(cid:87)(cid:16)(cid:87)(cid:72)(cid:85)(cid:80)(cid:3)(cid:86)(cid:87)(cid:68)(cid:87)(cid:76)(cid:86)(cid:87)(cid:76)(cid:70)(cid:86)(cid:29)
(cid:55)(cid:75)(cid:72)(cid:3)(cid:40)(cid:88)(cid:85)(cid:82)(cid:16)(cid:54)(cid:44)(cid:38)(cid:54)(cid:3)(cid:71)(cid:68)(cid:87)(cid:68)(cid:69)(cid:68)(cid:86)(cid:72)(cid:3)(cid:82)(cid:73)(cid:3)(cid:40)(cid:88)(cid:85)(cid:82)(cid:86)(cid:87)(cid:68)(cid:87)

Dominique LADIRAY, Fabio SARTORI
(cid:40)(cid:56)(cid:53)(cid:50)(cid:54)(cid:55)(cid:36)(cid:55)(cid:15)(cid:3)(cid:56)(cid:81)(cid:76)(cid:87)(cid:3)(cid:36)(cid:25)
(cid:54)(cid:87)(cid:68)(cid:87)(cid:76)(cid:86)(cid:87)(cid:76)(cid:70)(cid:68)(cid:79)(cid:3)(cid:44)(cid:81)(cid:71)(cid:76)(cid:70)(cid:68)(cid:87)(cid:82)(cid:85)(cid:86)(cid:3)(cid:73)(cid:82)(cid:85)(cid:3)(cid:40)(cid:88)(cid:85)(cid:82)(cid:16)(cid:93)(cid:82)(cid:81)(cid:72)(cid:3)(cid:37)(cid:88)(cid:86)(cid:76)(cid:81)(cid:72)(cid:86)(cid:86)(cid:3)(cid:38)(cid:92)(cid:70)(cid:79)(cid:72)(cid:3)(cid:36)(cid:81)(cid:68)(cid:79)(cid:92)(cid:86)(cid:76)(cid:86)
(cid:45)(cid:72)(cid:68)(cid:81)(cid:3)(cid:48)(cid:82)(cid:81)(cid:81)(cid:72)(cid:87)(cid:3)(cid:37)(cid:88)(cid:76)(cid:79)(cid:71)(cid:76)(cid:81)(cid:74)(cid:15)(cid:3)(cid:47)(cid:16)(cid:21)(cid:28)(cid:21)(cid:19)(cid:3)(cid:47)(cid:88)(cid:91)(cid:72)(cid:80)(cid:69)(cid:82)(cid:88)(cid:85)(cid:74)
(cid:72)(cid:16)(cid:80)(cid:68)(cid:76)(cid:79)(cid:29)(cid:3)(cid:71)(cid:82)(cid:80)(cid:76)(cid:81)(cid:76)(cid:84)(cid:88)(cid:72)(cid:17)(cid:79)(cid:68)(cid:71)(cid:76)(cid:85)(cid:68)(cid:92)(cid:35)(cid:70)(cid:72)(cid:70)(cid:17)(cid:72)(cid:88)(cid:17)(cid:76)(cid:81)(cid:87)(cid:15)(cid:3)(cid:73)(cid:68)(cid:69)(cid:76)(cid:82)(cid:17)(cid:86)(cid:68)(cid:85)(cid:87)(cid:82)(cid:85)(cid:76)(cid:3)(cid:35)(cid:70)(cid:72)(cid:70)(cid:17)(cid:72)(cid:88)(cid:17)(cid:76)(cid:81)(cid:87)

$EVWUDFW(cid:29)(cid:3) The  Euro-SICS  database  of  Eurostat  presents  a  collection  of  monthly  and
quarterly business cycle indicators for the Euro-zone, the European Union and the EU
Member  States.  The  database  is  evaluated  monthly  through  the  Eurostat  Qualistat
approach. Quality indicators, by country and statistical domain, have been produced to
measure the relevance, timeliness, completeness, accuracy and clarity of the time series

.H\ZRUGV(cid:29) Times series, Quality indicators, Database.

(cid:20)(cid:17)(cid:3)

(cid:44)(cid:81)(cid:87)(cid:85)(cid:82)(cid:71)(cid:88)(cid:70)(cid:87)(cid:76)(cid:82)(cid:81)

Unlike most of the literature about quality in official statistics, which mainly focus on
surveys, this paper raises a new issue concerning the measurement of data quality in time
series  statistics.  Our  study  describes  the  main  features  of  the  Euro-SICS1  database  of
Eurostat, which presents a collection of monthly and quarterly business cycle indicators for
the Euro-zone, the European Union and the EU Member States.
The database was created to provide data for business cycle analysis and satisfy the needs
of  the  European  Central  Bank  for  timely  and  long  time  series  helpful  for  devising  an
appropriate monetary policy for the Euro-zone.
The state of affairs of the database is evaluated monthly through the Eurostat Qualistat
approach. Quality indicators, by country and statistical domain, have been produced to
measure the relevance, timeliness, completeness, accuracy and clarity of the time series.
As there is a demand for long time series, the relevance of the statistics provided by the
database has been measured in terms of time series length. Moreover, short-term data are
relevant  if  they  are  timely.  Hence  the  key  quality  aspect  is  timeliness.  Data  clarity  is
another important quality aspect and supposed to be achieved through the provision of
metadata  describing  the  features  of  the  EU  aggregates  according  to  the  Special  Data

1 Euro-6,&6 stands for Euro 6tatistical ,ndicators &ommon 6ite. Euro-SICS is thus a database of the European
Statistical System as a whole to which National Statistical Institutes (NSIs) and National Central Banks (NCBs)
are contributing.

                                                          
’RPLQLTXH(cid:3)/DGLUD\(cid:15)(cid:3))DELR(cid:3)6DUWRUL

Dissemination Standard of the IMF.
A special issue to deal with while measuring the quality of time series is the establishment
of quality objectives for time series that not only concern different statistical domains, but
whose indicators also have different frequency. Another issue that should be stressed is the
accuracy measurement. Analysis of revisions can be carried out to assess the accuracy of
estimations. This implies that all the statistical domains and countries concerned have to
apply comparable revision policies and all the Euro-SICS domains should adopt the same
flagging system so that the different revisions of data can be tracked.
Finally, the main point to be borne in mind is that quality is a dynamic concept, so what
today is deemed to be an appropriate measurement of a phenomenon, may not be suitable
for  future  changes  in  user’s  needs  as  well  as  in  the  configuration  of  the  database.  A
continuous monitoring of the former and periodic checks of the latter will let us fulfil our
purposes.
The  outline  of  the  paper  is  as  follows:  Section  2  presents  the  general  features  of  the
Euro-SICS  database,  focussing  mainly  on  its  structure,  its  users  and  the  provision  of
metadata. Section 3 discusses some topics regarding quality of official statistics and its
evaluation. Section 4 presents some results concerning some indicators of relevance and
completeness, such as the length of the series, the number of missing values, etc. Sections
5 and 6 focus, respectively on the evaluation of timeliness and accuracy.

(cid:21)(cid:17)(cid:3) (cid:55)(cid:75)(cid:72)(cid:3)(cid:40)(cid:88)(cid:85)(cid:82)(cid:16)(cid:54)(cid:44)(cid:38)(cid:54)(cid:3)(cid:71)(cid:68)(cid:87)(cid:68)(cid:69)(cid:68)(cid:86)(cid:72)

Euro-SICS supplies institutional users with a wide range of short-term indicators for the
Euro-zone, the European Union and EU Member States. It is an extension of the Euro-
indicators template accessible through Eurostat's web site. The project was launched at the
beginning of 1999 and the beta version of Euro-SICS was presented in July 1999. The
Euro-SICS  database  was  opened  to  privileged  users  at  the  end  of  1999  and  a  revised
version will be accessible to the public at large in June 2001.

(cid:21)(cid:17)(cid:20)(cid:3) (cid:42)(cid:72)(cid:81)(cid:72)(cid:85)(cid:68)(cid:79)(cid:3)(cid:71)(cid:72)(cid:86)(cid:70)(cid:85)(cid:76)(cid:83)(cid:87)(cid:76)(cid:82)(cid:81)(cid:3)(cid:82)(cid:73)(cid:3)(cid:87)(cid:75)(cid:72)(cid:3)(cid:40)(cid:88)(cid:85)(cid:82)(cid:16)(cid:54)(cid:44)(cid:38)(cid:54)(cid:3)(cid:71)(cid:68)(cid:87)(cid:68)(cid:69)(cid:68)(cid:86)(cid:72)

The Euro-SICS series were selected on the basis of economic and statistical criteria. They
have  to  give  an  encompassing  and  fairly  accurate  picture  of  the  short-term  economic
situation.  Some  series  were  chosen  because  of  their  cyclical  features  (leading-lagging
structure), others because of their relevance in the construction of other macroeconomic
aggregates. As quarterly national accounts are at the centre of this exercise, mainly those
short-term  series  were  retained  whose  evolution  is  closely  related  to  the  evolution  of
quarterly account aggregates.
The Euro-SICS series can be classified into three main categories: fully harmonised series;
not fully harmonised national series available in all Member States; and Member State
specific series. Eurostat is in charge of series from the first two categories while Member
States  for  series  from  the  last  category.  The  Euro-SICS  database  contains  56  main
indicators. With the different breakdowns (by sector, by activity, etc.) the total number
increases to approximately 450 indicators. The indicators are organised in 15 domains (see
Table (cid:20))2.

2 The classification plan of Euro-SICS has some weaknesses: lack of domain homogeneity and a certain degree of

10

                                                          
4XDOLW\(cid:3)LQ(cid:3)6KRUW(cid:16)WHUP(cid:3)VWDWLVWLFV(cid:29)(cid:3)7KH(cid:3)(XUR(cid:16)6,&6(cid:3)GDWDEDVH(cid:3)RI(cid:3)(XURVWDW

To assure its usefulness for short-term analysts and policy-makers several principles have
to be respected: real-time updating (a delay of one working day between national release
and  the  update  of  the  database);  series  of  sufficient  length;  and  full  documentation.
Moreover, in view of ever changing needs the database has to be assessed and improved
continuously.  A  working  group  composed  of  contributors  and  key  users  has  been
established to monitor the content of the database and the user needs. After the opening of
the site to the general public, periodic surveys will be conducted to monitor the satisfaction
of users. Eurostat issues a “Monitoring Report” to inform all contributors and main users
about  the  evolution  of  the  content  of  the  database.  This  report  is  supplemented  by  a
“Global  Quality  Index”  meant  to  summaries  and  condense  the  quality  features  of  the
database in a single synthetic indicator.

(cid:21)(cid:17)(cid:21)(cid:3)

(cid:40)(cid:88)(cid:85)(cid:82)(cid:16)(cid:54)(cid:44)(cid:38)(cid:54)(cid:3)(cid:48)(cid:72)(cid:87)(cid:68)(cid:71)(cid:68)(cid:87)(cid:68)

Eurostat adopted in 1999 the SDDS (Special Data Dissemination Standard) for the Euro-
SICS  database  for  a  host  of  reasons.  The  heterogeneity  of  the  database  with  its  many
contributing institutions required the adoption of a standard to simplify the drafting and
updating metadata files within Eurostat and beyond. The IMF standard had the advantage
to exist already, so that Eurostat refrained from developing an own standard despite the
shortcomings of the IMF standard. Moreover, as many NSIs and NCBs are already using
SDDS to deliver metadata to the  IMF,  the  adoption  of  this  standard  made  it  easier  to
integrate metadata for national and European series. Finally the adoption of a well-known
standard was supposed to be also easier for users.
However, to facilitate SDDS usage, Eurostat has opted for a split of its metadata into three
parts: a first one containing only basic information and dissemination formats, a second one
containing a summary methodology, and a third one containing the release calendar. The
first part gives administrative details about the data (coverage, periodicity and timeliness),
about the access for the public, about the integrity (terms and conditions, internal access,
ministerial commentary, information about revision and changes in methodology), and
about  the  quality  (documentation  about  the  methodology  and  sources,  information
supporting cross-checks). A table of dissemination formats is also provided.
The second part on the summary methodology consists of six sections: (1) the analytical
framework, concepts, definitions and classifications; (2) the scope of the data (coverage
of  institutional  units,  geographic  areas,  etc.);  (3)  accounting  conventions  (time  of
recording, valuation methods); (4) the nature of the basic data; (5) compilation practices
(weighting schemes, imputation methods, etc.); and (6) other aspects  such as seasonal
adjustments, reference years, etc. The third part (Release Calendar) is directly available
from the Eurostat web-site. National release calendars will become available through links
with national web-sites.

(cid:22)(cid:17)(cid:3) (cid:55)(cid:75)(cid:72)(cid:3)(cid:81)(cid:72)(cid:70)(cid:72)(cid:86)(cid:86)(cid:76)(cid:87)(cid:92)(cid:3)(cid:82)(cid:73)(cid:3)(cid:68)(cid:3)(cid:84)(cid:88)(cid:68)(cid:79)(cid:76)(cid:87)(cid:92)(cid:3)(cid:80)(cid:72)(cid:68)(cid:86)(cid:88)(cid:85)(cid:72)

Eurostat is, of course, not really a newcomer in infra-annual economic statistics. However,
its service was not so well-known and thus not much used in the past. This had to change

redundancy in the classification structure). A new structure has been adopted for the database and will be used
for the public release.

11

                                                                                                                                            
’RPLQLTXH(cid:3)/DGLUD\(cid:15)(cid:3))DELR(cid:3)6DUWRUL

and changed also quite rapidly, once the third phase of EMU started. Therefore Eurostat
had to improve the notoriety and appreciation of its service by the public at large and, most
notably,  professional  users.  Quality  measures  for  Eurostat’s  infra-annual  economic
statistics were considered to be of help in this context. For this it was possible to draw upon
Eurostat’s  quality  initiative  launched  in  1998.  Within  the  framework  of  this  initiative
(launched also in several Member States) Eurostat developed its (user oriented) quality
concept (Eurostat 2000a and 2000b) built on the ISO 8402 norm. This norm states that:
“Quality is the totality of features and characteristics of a product or service that bear on
its ability to satisfy stated or implied needs”.
The features and characteristics retained for Eurostat’s  quality  concept  are:  relevance,
accuracy, timeliness and punctuality, accessibility and clarity, comparability, coherence and
completeness.  However,  the  definitions  seem  to  have  been  developed  mainly  in  the
framework  of  survey  statistics  and  their  extension  to  time  series  statistics  is  not  so
straightforward (at least for some of them).  Nevertheless they have served quite well as
a starting point for assessing the quality of the Euro-SICS database.
All these features and characteristics are, of course, not of equal importance and not all of
them are equally relevant in the context of time series statistics. The aspect of timeliness
and punctuality is obvious of crucial importance for the Euro-SICS database, the more so
as Eurostat has often been criticised for the lack of timeliness of its indicators. Therefore
this aspect has to be particularly elaborated in the quality assessment of Euro-SICS.
Moreover,  an  increasing  range  of  additionally  also  changing  user  needs  have  to  be
considered. Euro-SICS is for the moment accessible only for a number privileged (and
highly  professional  and  knowledgeable)  users  (NSIs,  ECB,  ministries,  etc.).  Once  the
database will be accessible by the public at large features and characteristics of minor
importance in the past (e.g. accessibility, clarity, etc.) will become much more important
and have thus to get more attention. A quality assessment such as Eurostat’s that is geared
towards user needs will have to change when the composition of users is changing.
Finally, however complex a theoretical framework for the assessment of the quality of a
database in view of ever changing user needs might be, in the end simple routines have to
be developed that can be handled easily. In the case of the Euro-SICS database a couple
of fairly straightforward data-mining routines have been set up. These routines, however
limited they are, try to cast the theoretical features into simple and still useful numbers. Of
course, they are only the starting point, more sophisticated routines have to be developed,
most notably also when the underlying database becomes more sophisticated.

(cid:23)(cid:17)(cid:3) (cid:53)(cid:72)(cid:79)(cid:72)(cid:89)(cid:68)(cid:81)(cid:70)(cid:72)(cid:3)(cid:68)(cid:81)(cid:71)(cid:3)(cid:70)(cid:82)(cid:80)(cid:83)(cid:79)(cid:72)(cid:87)(cid:72)(cid:81)(cid:72)(cid:86)(cid:86)

In Eurostat 2000a, the aspect of relevance and completeness is defined as follows:
Relevance of statistical concepts: The statistical concepts used for statistics are relevant if
they  meet  user  needs.  The  identification  of  users  and  their  expectations  is  therefore
necessary.
Completeness: Domains for which statistics are available should reflect  the  needs  and
priorities expressed by the users of the European Statistical System.
Such a definition of quality with respect to relevance and completeness implies a scrutiny
of the database content by users. Therefore a survey has been conducted recently to know
more about the needs of Euro-SICS users (a very limited group of NCBs, NSIs and some
other institutions such as DG ECFIN, ECB, etc.). Eurostat intends to repeat such surveys

12

4XDOLW\(cid:3)LQ(cid:3)6KRUW(cid:16)WHUP(cid:3)VWDWLVWLFV(cid:29)(cid:3)7KH(cid:3)(XUR(cid:16)6,&6(cid:3)GDWDEDVH(cid:3)RI(cid:3)(XURVWDW

among a wider range of users (e.g. financial market operators) in the future. By the way,
the first survey (see Eurostat 2001a) led to a somewhat surprising result, as users wanted
“more indicators, but less breakdowns”. This stood in stark contrast to both the ECB and
DG  ECFIN,  because  these  institutions  came  out  in  favour  of  a  reduced  number  of
indicators (see ECB 2000). Expressed user needs can obviously be quite contradictory. In
this case the official statisticians will have to decide taking into account the resources at
their disposal and their technical possibilities.
There are, however, some aspects other than (expressed, assumed or otherwise identified)
user needs which allow to assess relevance and completeness. Firstly the length of the time
series has a strong bearing on the relevance of a statistical database, as sufficiently long
time series are needed for short-term economic analysis (at least two cycles). Moreover the
periodicity plays equally an  important  role,  because  a  short-term  analysis  can  only  be
conducted on the basis of high frequency data (monthly or quarterly). These aspects are
analysed in the following sections.

(cid:23)(cid:17)(cid:20)(cid:3) (cid:53)(cid:72)(cid:79)(cid:72)(cid:89)(cid:68)(cid:81)(cid:70)(cid:72)(cid:3)(cid:68)(cid:81)(cid:71)(cid:3)(cid:70)(cid:82)(cid:81)(cid:86)(cid:88)(cid:79)(cid:87)(cid:68)(cid:87)(cid:76)(cid:82)(cid:81)

Graph 1 provides the number of monthly extractions from the Euro-SICS database ever
since the database became available. This indicator gives an idea of how often the database
is consulted, and thus of its relevance.

(cid:25)(cid:19)(cid:19)

(cid:23)(cid:19)(cid:19)

(cid:21)(cid:19)(cid:19)

(cid:19)

Apr-99

Jul-99 Oct-99 Jan-00 Apr-00

Jul-00 Oct-00 Jan-01

(cid:42)(cid:85)(cid:68)(cid:83)(cid:75)(cid:3)(cid:20)(cid:29)(cid:3)Evolution of the number of extractions from the Euro-SICS database.

It should be noted, however, that some privileged users (ECB and its ESCB partners) can
get access to this set of infra-annual economic data not only through Euro-SICS but also
through other means: through a direct data delivery by Eurostat or through CIRCA. Both
non-standard  ways  of  accessing  data  have  not  been  taken  into  account,  so  Graph  1
underestimates the relevance of the database.

(cid:23)(cid:17)(cid:21)(cid:3) (cid:38)(cid:82)(cid:89)(cid:72)(cid:85)(cid:68)(cid:74)(cid:72)(cid:3)(cid:68)(cid:81)(cid:71)(cid:3)(cid:85)(cid:72)(cid:74)(cid:88)(cid:79)(cid:68)(cid:85)(cid:3)(cid:88)(cid:83)(cid:71)(cid:68)(cid:87)(cid:76)(cid:81)(cid:74)

The database contains only 56 indicators. However, taking into account regional, sectoral
and other break-downs the database contains a theoretical number of approximately 15,000

13

’RPLQLTXH(cid:3)/DGLUD\(cid:15)(cid:3))DELR(cid:3)6DUWRUL

time-series. For a host of reasons (confidentiality, material or logical impossibility, etc.),
not all the Member States are able to compile all indicators.
Table (cid:20) shows the distribution between active series that are regularly updated, non-active
series that have not been updated during the last twelve months, and simply missing series
that are non-compiled by Member States or Eurostat for whatever reason. The database
currently contains 8472 “active” series, while 6119 series are “missing” and 631 series are
“inactive”.

Series

Missing

Active

Inactive

Domain
Balance of Payments
Business and Consumer Surveys
Energy
External Trade
Industry
Labour Cost
Employment, Construction, M
Employment, Construction, Q
Employment, Industry, M
Employment, Industry, Q
Monetary and Financial Indicators
National Accounts
Prices
Retail sales Index
Unemployment
Total

N
630
40
.
28
409
17
.
.
26
143
84
4432
269
41
.
6119

%
57.1
6.5
.
7.4
17.6
12.5
.
.
5.0
14.5
38.9
62.5
23.6
10.0
.
40.2

N
474
572
80
332
1774
104
16
26
476
668
66
2629
753
342
160
8472

%
42.9
93.5
94.1
87.8
76.2
76.5
100.0
86.7
91.2
67.7
30.6
37.1
66.1
83.8
95.2
55.7

N

.
.
5
18
146
15
.
4
20
175
66
31
118
25
8
631

%

.
.
5.9
4.8
6.3
11.0
.
13.3
3.8
17.7
30.6
0.4
10.4
6.1
4.8
4.1

Total
N
1104
612
85
378
2329
136
16
30
522
986
216
7092
1140
408
168
15222

(cid:55)(cid:68)(cid:69)(cid:79)(cid:72)(cid:3)(cid:20)(cid:29)(cid:3)Distribution of the series by length in the Euro-SICS database (by domain as of
01/03/2001, only active series are considered)

(cid:23)(cid:17)(cid:22)(cid:3)

(cid:41)(cid:85)(cid:72)(cid:84)(cid:88)(cid:72)(cid:81)(cid:70)(cid:92)

The database consists of fifteen domains of which ten contain monthly and the remaining
five contain quarterly series. There is no domain with any annual data. In terms of series,
roughly 55% of the series are monthly and the remaining 45% quarterly,  whereby  the
domains themselves are homogeneous3.

(cid:23)(cid:17)(cid:23)(cid:3)

(cid:47)(cid:72)(cid:81)(cid:74)(cid:87)(cid:75)

The database is supposed to contain series spanning over at least 15 years. 52.3% of the
active series of the database cover already 15 years or more, and 73% cover 10 years at
least.  This  is  fairly  satisfactory  (Table  2).  But  the  situation  changes  heavily  from  one
domain to the other.
For example, as far as Balance of Payments is concerned, more than 88% of the series
cover more than 15 years: on the other hand, 76% of the series on Labour Cost cover less
than 10 years. (cid:40)(cid:85)(cid:85)(cid:82)(cid:85)(cid:4)(cid:3)(cid:53)(cid:72)(cid:73)(cid:72)(cid:85)(cid:72)(cid:81)(cid:70)(cid:72)(cid:3)(cid:86)(cid:82)(cid:88)(cid:85)(cid:70)(cid:72)(cid:3)(cid:81)(cid:82)(cid:87)(cid:3)(cid:73)(cid:82)(cid:88)(cid:81)(cid:71)(cid:17) compares the different domains.

3 This is going to change with the introduction of a new classification plan. Homogeneity of domains with respect
to frequency is not an attribute of importance. More important is a classification plan which does not give rise
to too many non-existent series.

14

                                                          
4XDOLW\(cid:3)LQ(cid:3)6KRUW(cid:16)WHUP(cid:3)VWDWLVWLFV(cid:29)(cid:3)7KH(cid:3)(XUR(cid:16)6,&6(cid:3)GDWDEDVH(cid:3)RI(cid:3)(XURVWDW

Length of the series

Domain
Balance of Payments
Business and Consumer Surveys
Energy
External Trade
Industry
Labour Cost
Employment, Construction, M
Employment, Construction, Q
Employment, Industry, M
Employment, Industry, Q
Monetary & Financial Indicators
National Accounts
Prices
Retail sales Index
Unemployment
Total

from 1 year
to less than
5 years
%

N

from 5 years
to less than
10 years
%

N

from 10 years
to less than
15 years
N

%

42
21
.
36
79
.
.
.
38
37
3
61
1
25
.
343

8.9
3.7
.
10.8
4.5
.
.
.
8.0
5.5
4.5
2.3
0.1
7.3
.
4.0

6
55
.
28
365
79
6
6
112
192
23
506
390
158
16
1941

1.3
9.6
.
8.4
20.6
76.0
37.5
23.1
23.5
28.7
34.8
19.2
51.7
46.2
10.0
22.9

6
101
23
.
525
11
4
10
94
183
20
516
95
125
40
1753

1.3
17.7
28.8
.
29.6
10.6
25.0
38.5
19.7
27.4
30.3
19.6
12.6
36.5
25.0
20.7

15 years
and
more

N

420
395
57
268
805
14
6
10
232
256
20
1546
267
34
104
4433

%
88.6
69.1
71.3
80.7
45.4
13.5
37.5
38.5
48.7
38.3
30.3
58.8
35.3
9.9
65.0
52.3

Total

N

474
572
80
332
1774
104
16
26
476
668
66
2629
753
342
160
8470

(cid:55)(cid:68)(cid:69)(cid:79)(cid:72)(cid:3)(cid:21): Distribution of the series by length in the Euro-SICS database (by domain as of
01/03/2001, only active series are considered)

For European aggregates, however, the situation is not as good. Only 43.4% of the Euro-
zone series cover more than 10 years. Table 3 reveals quite clearly the unsatisfactory length
of Euro-zone series and the same is obviously valid for EU series.

Length of the series

Domain
Balance of Payments
Business and Consumer Surveys
Energy
External Trade
Industry
Labour Cost
Employment, Construction, M
Employment, Construction, Q
Employment, Industry, M
Employment, Industry, Q
Monetary & Financial Indicators
National Accounts
Prices
Retail sales Index
Unemployment
Total

N

from 1 year
to less than
5 years
%
(cid:23)(cid:22)(cid:17)(cid:27)
(cid:17)
(cid:17)
(cid:17)
(cid:20)(cid:26)(cid:17)(cid:22)
(cid:17)
(cid:17)
(cid:17)
(cid:21)(cid:19)(cid:17)(cid:26)
(cid:21)(cid:19)(cid:17)(cid:26)
(cid:17)
(cid:17)
(cid:17)
(cid:23)(cid:26)(cid:17)(cid:23)
(cid:17)
(cid:20)(cid:24)

21
.
.
.
22
.
.
.
12
12
.
.
.
9
.
76

from 5 years
to less than
10 years
%

N

from 10 years
to less than
15 years
N

%

N

15 years
and
more

.
.
.
.
75
7
2
2
24
24
5
22
37
10
.
208

(cid:17)
(cid:17)
(cid:17)
(cid:17)
(cid:24)(cid:28)(cid:17)(cid:20)
(cid:27)(cid:26)(cid:17)(cid:24)
(cid:20)(cid:19)(cid:19)
(cid:20)(cid:19)(cid:19)
(cid:23)(cid:20)(cid:17)(cid:23)
(cid:23)(cid:20)(cid:17)(cid:23)
(cid:22)(cid:27)(cid:17)(cid:24)
(cid:20)(cid:19)(cid:19)
(cid:24)(cid:25)(cid:17)(cid:28)
(cid:24)(cid:21)(cid:17)(cid:25)
(cid:17)
(cid:23)(cid:20)(cid:17)(cid:21)

.
3
4
.
22
1
.
.
22
22
2
.
26
.
16
118

(cid:17)
(cid:27)(cid:17)(cid:25)
(cid:20)(cid:19)(cid:19)
(cid:17)
(cid:20)(cid:26)(cid:17)(cid:22)
(cid:20)(cid:21)(cid:17)(cid:24)
(cid:17)
(cid:17)
(cid:22)(cid:26)(cid:17)(cid:28)
(cid:22)(cid:26)(cid:17)(cid:28)
(cid:20)(cid:24)(cid:17)(cid:23)
(cid:17)
(cid:23)(cid:19)
(cid:17)
(cid:20)(cid:19)(cid:19)
(cid:21)(cid:22)(cid:17)(cid:23)

27
32
.
28
8
.
.
.
.
.
6
.
2
.
.
103

Total

N

48
35
4
28
127
8
2
2
58
58
13
22
65
19
16
505

%
(cid:24)(cid:25)(cid:17)(cid:22)
(cid:28)(cid:20)(cid:17)(cid:23)
(cid:17)
(cid:20)(cid:19)(cid:19)
(cid:25)(cid:17)(cid:22)
(cid:17)
(cid:17)
(cid:17)
(cid:17)
(cid:17)
(cid:23)(cid:25)(cid:17)(cid:21)
(cid:17)
(cid:22)(cid:17)(cid:20)
(cid:17)
(cid:17)
(cid:21)(cid:19)(cid:17)(cid:23)

(cid:55)(cid:68)(cid:69)(cid:79)(cid:72)(cid:3)(cid:22)(cid:29)(cid:3)Distribution of the series by length in the Euro-SICS database (by domain as of
01/03/2001, only active series referring to the Euro-zone are considered)

(cid:23)(cid:17)(cid:24)(cid:3) (cid:48)(cid:76)(cid:86)(cid:86)(cid:76)(cid:81)(cid:74)(cid:3)(cid:89)(cid:68)(cid:79)(cid:88)(cid:72)(cid:86)

15

’RPLQLTXH(cid:3)/DGLUD\(cid:15)(cid:3))DELR(cid:3)6DUWRUL

The presence of missing values in a database is obviously an indicator of bad data quality.
Series  with  several  missing  values  are  difficult  to  model,  and  hence  useless  for  most
professional  users  of  the  database.  For  the  Euro-SICS  database,  the  situation  is  very
positive, as series with missing values only account for 0.2% of the database and more than
98% of the series do not have missing value at all. Moreover, it should be born in mind that
some of the values are even missing for reasons that are quite acceptable (e.g. technical or
logical impossibility).

Number of missing values
Domain
Balance of Payments
Business and Consumer Surveys
Energy
External Trade
Industry
Labour Cost
Employment, Construction, M
Employment, Construction, Q
Employment, Industry, M
Employment, Industry, Q
Monetary & Financial Indicators
National Accounts
Prices
Retail sales Index
Unemployment
Total

None
%

N

1

N

%

From 2 to 10
N

%

10 or more
%

N

Total
N

456
572
85
349
1901
115
16
30
488
831
123
2654
862
365
160
9007

96.2
100.0
100.0
99.7
99.0
96.6
100.0
100.0
98.4
98.6
93.2
99.8
99.0
99.5
95.2
98.9

.
.
.
.
6
.
.
.
3
8
5
2
.
2
.
26

.
.
.
.
0.3
.
.
.
0.6
0.9
3.8
0.1
.
0.5
.
0.3

3
.
.
1
9
2
.
.
1
.
1
2
.
.
.
19

0.6
.
.
0.3
0.5
1.7
.
.
0.2
.
0.8
0.1
.
.
.
0.2

15
.
.
.
4
2
.
.
4
4
3
2
9
.
8
51

3.2
.
.
.
0.2
1.7
.
.
0.8
0.5
2.3
0.1
1.0
.
4.8
0.6

474
572
85
350
1920
119
16
30
496
843
132
2660
871
367
168
9103

(cid:55)(cid:68)(cid:69)(cid:79)(cid:72)(cid:3)(cid:23)(cid:29)(cid:3)Distribution of the series by number of missing values in the Euro-SICS database
(by domain as of 01/03/2001)

(cid:24)(cid:17)(cid:3) (cid:55)(cid:76)(cid:80)(cid:72)(cid:79)(cid:76)(cid:81)(cid:72)(cid:86)(cid:86)

In Eurostat (2000a), the aspect of timeliness, once again from a user perspective, is defined
as follows:
(cid:55)(cid:76)(cid:80)(cid:72)(cid:79)(cid:76)(cid:81)(cid:72)(cid:86)(cid:86)(cid:3) (cid:68)(cid:81)(cid:71)(cid:3) (cid:83)(cid:88)(cid:81)(cid:70)(cid:87)(cid:88)(cid:68)(cid:79)(cid:76)(cid:87)(cid:92)(cid:29)(cid:3) (cid:48)(cid:82)(cid:86)(cid:87)(cid:3) (cid:88)(cid:86)(cid:72)(cid:85)(cid:86)(cid:3) (cid:90)(cid:68)(cid:81)(cid:87)(cid:3) (cid:88)(cid:83)(cid:16)(cid:87)(cid:82)(cid:16)(cid:71)(cid:68)(cid:87)(cid:72)(cid:3) (cid:73)(cid:76)(cid:74)(cid:88)(cid:85)(cid:72)(cid:86)(cid:15)(cid:3) (cid:90)(cid:75)(cid:76)(cid:70)(cid:75)(cid:3) (cid:68)(cid:85)(cid:72)(cid:3) (cid:83)(cid:88)(cid:69)(cid:79)(cid:76)(cid:86)(cid:75)(cid:72)(cid:71)
(cid:73)(cid:85)(cid:72)(cid:84)(cid:88)(cid:72)(cid:81)(cid:87)(cid:79)(cid:92)(cid:3)(cid:68)(cid:81)(cid:71)(cid:3)(cid:82)(cid:81)(cid:3)(cid:87)(cid:76)(cid:80)(cid:72)(cid:3)(cid:68)(cid:87)(cid:3)(cid:83)(cid:85)(cid:72)(cid:16)(cid:72)(cid:86)(cid:87)(cid:68)(cid:69)(cid:79)(cid:76)(cid:86)(cid:75)(cid:72)(cid:71)(cid:3)(cid:71)(cid:68)(cid:87)(cid:72)(cid:86)(cid:17)
Timeliness is, of course, a crucial quality feature of short term statistics. An appropriate
evaluation of this aspect for the Euro-SICS database is therefore of utmost importance.
However, due to the fairly large number of series contained in this database and above all
their  heterogeneity  (compilation  methods  applied  by  Member  States,  revision  policy
followed by Member States, estimation of Euro-zone and EU aggregates, etc.) it is quite
difficult to get a sufficiently uniform grip on this aspect.
The following section tries to clear the conceptual ground around timeliness, taking into
account the time series context. Thereafter the timeliness measurements applied for the
Euro-SICS database  are  presented.  Finally  the  future  development  in  the  field  will  be
sketched.

(cid:24)(cid:17)(cid:20)(cid:3) (cid:38)(cid:82)(cid:81)(cid:70)(cid:72)(cid:83)(cid:87)(cid:86)(cid:3)(cid:68)(cid:81)(cid:71)(cid:3)(cid:71)(cid:72)(cid:73)(cid:76)(cid:81)(cid:76)(cid:87)(cid:76)(cid:82)(cid:81)(cid:86)

Different indicators can be considered for measure this particular quality aspect:

16

4XDOLW\(cid:3)LQ(cid:3)6KRUW(cid:16)WHUP(cid:3)VWDWLVWLFV(cid:29)(cid:3)7KH(cid:3)(XUR(cid:16)6,&6(cid:3)GDWDEDVH(cid:3)RI(cid:3)(XURVWDW

the time lag between the reference time point (or the end of the reference

the correspondence between the planned / promised date and factual date

the  delay  between  the  date  the  user  consults  the  database  and  the

 (cid:55)(cid:76)(cid:80)(cid:72)(cid:79)(cid:76)(cid:81)(cid:72)(cid:86)(cid:86):
period) and the time for publication of the statistics.
(cid:51)(cid:88)(cid:81)(cid:70)(cid:87)(cid:88)(cid:68)(cid:79)(cid:76)(cid:87)(cid:92):
for dissemination of the statistics.
(cid:41)(cid:85)(cid:72)(cid:86)(cid:75)(cid:81)(cid:72)(cid:86)(cid:86):
reference period of the most up-to-date data.
Indicators a) and b) are suggested in Eurostat (1999a) while indicator c) has been proposed
by  Grünewald  (2000).  Timeliness  and  punctuality  are  defined  from  a  producer’s  and
freshness from a user’s the point of view.
Conceptually  at  least  seven  dates  can  be  associated  with  each  national  figure  in  the
database:
(cid:140) 

the end of the reference period (i.e.: if the reference period is November 99, then
d1 = 30 Nov 99);
the announced date for its publication (d2 = 23 Dec 99);
the date it has been published (d3 = 23 Dec 99);
the date it arrives to Eurostat (d4 = 24 Dec 99);
the date it appears in Euro-SICS (d5 = 7 Jan 00);
the date of the last update of the domain the series belongs to (d6 = 9 Jan 00);
the date of the database quality control (d7 = 15 Jan 00).

(cid:140) 
(cid:140) 
(cid:140) 
(cid:140) 
(cid:140) 
(cid:140) 
For the moment a simple delay indicator is compiled which corresponds to the different
d6 -
  d1 for the last available data element of the series. It has been defined in this way
since  the  two  dates  are  easily  available  from  Euro-SICS.  However,  to  compile  more
meaningful timeliness indicators, it is, of course, necessary to monitor the evolution of the
Euro-SICS database in conjunction with the release calendars of Member State. Therefore
some improvements are planned to obtain a deeper evaluation of all the different aspects
concerning timeliness in Euro-SICS.
a)
Timeliness in a narrow sense could be defined as d5 -
  d1. However, this indicator is not
easy to compute because Euro-SICS can be updated every day. So a daily check of the
database would be necessary to detect new values as soon as they are inserted.
b)
Punctuality refers directly to a release calendar and pre-defined dissemination dates. If the
different release calendars can be linked to the database, it would possible to compute:
(cid:140)  Publication Punctuality = d3 -
(cid:140)  Euro-SICS Punctuality = d5 -
(cid:140) 
(cid:140) 
c)
This is the indicator suggested by Grünewald: Freshness = d7 -
 d1 . There is obviously no
problem to compute it, but it depends equally on the timing of the quality control and is
thus not a feature related entirely to the data, but partly to the control management.

a Member State component = d4 -
a Eurostat component = d5 -
 d4 .

 d2 .
 d2 . This delay can be divided in:

(cid:51)(cid:88)(cid:81)(cid:70)(cid:87)(cid:88)(cid:68)(cid:79)(cid:76)(cid:87)(cid:92)

(cid:55)(cid:76)(cid:80)(cid:72)(cid:79)(cid:76)(cid:81)(cid:72)(cid:86)(cid:86)

(cid:41)(cid:85)(cid:72)(cid:86)(cid:75)(cid:81)(cid:72)(cid:86)(cid:86)

 d2 ;

(cid:24)(cid:17)(cid:21)(cid:3)

(cid:55)(cid:75)(cid:72)(cid:3)(cid:70)(cid:75)(cid:82)(cid:76)(cid:70)(cid:72)(cid:3)(cid:82)(cid:73)(cid:3)(cid:87)(cid:75)(cid:85)(cid:72)(cid:86)(cid:75)(cid:82)(cid:79)(cid:71)(cid:86)

These  different  aspects  are  essential  to  assess  the  quality  of  the  database.  They  must
therefore  be  part  of  a  global  quality  index.  However,  to  compile  such  an  index  in  a
meaningful way, it is necessary to define adequately a scale and set a standard for the

17

’RPLQLTXH(cid:3)/DGLUD\(cid:15)(cid:3))DELR(cid:3)6DUWRUL

various aspects.
As far as punctuality is concerned, adequate standards can be set easily:
there should, of course, be no delay between the date promised in the release calendar and
the actual dissemination date, therefore the “Publication Punctuality” standard is 0;
the Member States are supposed to send the data to Eurostat as soon as possible and a
realistic  objective  is  that  Eurostat  receives  data  the  very  same  day.  Eurostat  domain
managers should also be able to update the database immediately. Their and the standard
should also be 0.
For timeliness and freshness, the choice of an appropriate threshold is much more difficult.
A threshold should in principle reflect first of all a compromise between the requirements
of users and the resources at the disposal of producers. However, other aspects should
equally be taken into account. What can realistically be expected by users is, of course, at
least what producers elsewhere providing. Thus standards could be set on the basis of a
comparison between the EU and the United States4. The current performance of the US
statisticians could be used as the standard to be matched by EU statisticians. Finally, the
statistical domains are quite different (e.g. speed and mode of data collection). Therefore
different thresholds should be established for different domains and in some cases even for
different series in the same domain.

(cid:24)(cid:17)(cid:22)(cid:3)

(cid:55)(cid:76)(cid:80)(cid:72)(cid:79)(cid:76)(cid:81)(cid:72)(cid:86)(cid:86)(cid:3)(cid:76)(cid:81)(cid:3)(cid:87)(cid:75)(cid:72)(cid:3)(cid:40)(cid:88)(cid:85)(cid:82)(cid:16)(cid:54)(cid:44)(cid:38)(cid:54)(cid:3)(cid:71)(cid:68)(cid:87)(cid:68)(cid:69)(cid:68)(cid:86)(cid:72)

Among the various indicators introduced so far (timeliness, punctuality and freshness),
timeliness is for the moment quite difficult to compute since an automatic tracing of the
daily modifications of the database has not yet been developed. But even punctuality is
difficult  to  compute,  because  the  database  is  not  yet  linked  to  Member  States  release
calendars. Improvements, however, are planned in near future, so that a more complete
picture of the different aspects of timeliness will be available for a more elaborate quality
evaluation.
For the moment we can only evaluate the freshness of the database, that is timeliness from
the users' point of view. The results of this analysis are presented in Table 5. The results
vary  considerably  across  domains.  This  means  that  improvement  are  not  required
everywhere, but also that different thresholds will probably have to be adopted for different
domains (or indicators).

(cid:25)(cid:17)(cid:3) (cid:36)(cid:70)(cid:70)(cid:88)(cid:85)(cid:68)(cid:70)(cid:92)

In Eurostat 2000a, the following definition is suggested for accuracy:
(cid:36)(cid:70)(cid:70)(cid:88)(cid:85)(cid:68)(cid:70)(cid:92)(cid:3)(cid:82)(cid:73)(cid:3)(cid:72)(cid:86)(cid:87)(cid:76)(cid:80)(cid:68)(cid:87)(cid:72)(cid:86)(cid:29)(cid:3)(cid:36)(cid:70)(cid:70)(cid:88)(cid:85)(cid:68)(cid:70)(cid:92)(cid:3)(cid:76)(cid:86)(cid:3)(cid:71)(cid:72)(cid:73)(cid:76)(cid:81)(cid:72)(cid:71)(cid:3)(cid:68)(cid:86)(cid:3)(cid:87)(cid:75)(cid:72)(cid:3)(cid:70)(cid:79)(cid:82)(cid:86)(cid:72)(cid:81)(cid:72)(cid:86)(cid:86)(cid:3)(cid:69)(cid:72)(cid:87)(cid:90)(cid:72)(cid:72)(cid:81)(cid:3)(cid:87)(cid:75)(cid:72)(cid:3)(cid:72)(cid:86)(cid:87)(cid:76)(cid:80)(cid:68)(cid:87)(cid:72)(cid:71)(cid:3)(cid:89)(cid:68)(cid:79)(cid:88)(cid:72)
(cid:68)(cid:81)(cid:71)(cid:3)(cid:87)(cid:75)(cid:72)(cid:3)(cid:11)(cid:88)(cid:81)(cid:78)(cid:81)(cid:82)(cid:90)(cid:81)(cid:12)(cid:3)(cid:87)(cid:85)(cid:88)(cid:72)(cid:3)(cid:83)(cid:82)(cid:83)(cid:88)(cid:79)(cid:68)(cid:87)(cid:76)(cid:82)(cid:81)(cid:3)(cid:89)(cid:68)(cid:79)(cid:88)(cid:72)(cid:17)(cid:3)(cid:36)(cid:86)(cid:86)(cid:72)(cid:86)(cid:86)(cid:76)(cid:81)(cid:74)(cid:3)(cid:87)(cid:75)(cid:72)(cid:3)(cid:68)(cid:70)(cid:70)(cid:88)(cid:85)(cid:68)(cid:70)(cid:92)(cid:3)(cid:82)(cid:73)(cid:3)(cid:68)(cid:81)(cid:3)(cid:72)(cid:86)(cid:87)(cid:76)(cid:80)(cid:68)(cid:87)(cid:72)(cid:3)(cid:76)(cid:81)(cid:89)(cid:82)(cid:79)(cid:89)(cid:72)(cid:86)
(cid:68)(cid:81)(cid:68)(cid:79)(cid:92)(cid:86)(cid:76)(cid:81)(cid:74)(cid:3)(cid:87)(cid:75)(cid:72)(cid:3)(cid:87)(cid:82)(cid:87)(cid:68)(cid:79)(cid:3)(cid:72)(cid:85)(cid:85)(cid:82)(cid:85)(cid:3)(cid:68)(cid:86)(cid:86)(cid:82)(cid:70)(cid:76)(cid:68)(cid:87)(cid:72)(cid:71)(cid:3)(cid:90)(cid:76)(cid:87)(cid:75)(cid:3)(cid:87)(cid:75)(cid:72)(cid:3)(cid:72)(cid:86)(cid:87)(cid:76)(cid:80)(cid:68)(cid:87)(cid:72)(cid:17)
Obviously this definition has been developed and is thus well suited for survey data, but
not so much for time series. For time series, the availability of high-quality metadata is
probably a very important factor for an accuracy evaluation (see Section 2.2). Nevertheless,

4 Recently this issue raised a deep debate inside the European Statistical System and lead to the establishment of
a Task Force for EU-US comparison, involving Eurostat, the European Central Bank, the OECD and several
Member States.

18

                                                          
4XDOLW\(cid:3)LQ(cid:3)6KRUW(cid:16)WHUP(cid:3)VWDWLVWLFV(cid:29)(cid:3)7KH(cid:3)(XUR(cid:16)6,&6(cid:3)GDWDEDVH(cid:3)RI(cid:3)(XURVWDW

there are also simple ways to measure the accuracy of a time series database such as Euro-
SICS. This could be done e.g. by the number of missing values in the series, the number
of irregular or "strange" values in the series (outliers), and an analysis  of  the  revision
patterns.

Delay of the series

Domain
Balance of Payments
Bus. and Cons. Surveys
Energy
External Trade
Industry
Labour Cost
Employment, Const, M
Employment, Const, Q
Employment, Ind., M
Employment, Ind., Q
Mon. & Fin. Indicators
National Accounts
Prices
Retail sales Index
Unemployment
Total

Under
30 days

N

%

From 30
 Days to less
than 60 days
N

%

From 60 days
to less than
90 days
N

%

From 90 days
to less than
180 days
N

.
572
.
.
.
.
.
.
.
.
30
.
.
.
112
714

.
100
.
.
.
.
.
.
.
.
22.7
.
.
.
66.7
7.8

.
.
.
186
34
.
.
.
.
218
.
.
459
16
32
945

.
.
.
53.1
1.8
.
.
.
.
25.9
.
.
52.7
4.4
19.0
10.4

.
.
16
146
955
19
.
.
202
.
36
776
278
208
8
2644

.
.
18.8
41.7
49.7
16.0
.
.
40.7
.
27.3
29.2
31.9
56.7
4.8
29.0

346
.
34
.
713
78
16
16
268
418
.
1752
5
110
8
3764

Over
180 days

N

128
.
35
18
218
22
.
14
26
207
66
132
129
33
8
1036

%
27.0
.
41.2
5.1
11.4
18.5
.
46.7
5.2
24.6
50.0
5.0
14.8
9.0
4.8
11.4

Total

N

474
572
85
350
1920
119
16
30
496
843
132
2660
871
367
168
9103

%
73.0
.
40.0
.
37.1
65.5
100
53.3
54.0
49.6
.
65.9
0.6
30.0
4.8
41.3

(cid:55)(cid:68)(cid:69)(cid:79)(cid:72)(cid:3)(cid:24)(cid:29)(cid:3)Distribution of the delay between the reference period and the date of the quality
control, that is 01/03/2001 (by domain)

The first approach has already been dealt with above and the third approach cannot be
pursued at least for the time being due to the limits imposed by the technical characteristics
of the Euro-SICS database. Therefore only the second aspect can be dealt with here.

(cid:25)(cid:17)(cid:20)(cid:3)

(cid:44)(cid:85)(cid:85)(cid:72)(cid:74)(cid:88)(cid:79)(cid:68)(cid:85)(cid:3)(cid:89)(cid:68)(cid:79)(cid:88)(cid:72)(cid:86)(cid:3)(cid:68)(cid:81)(cid:71)(cid:3)(cid:82)(cid:88)(cid:87)(cid:79)(cid:76)(cid:72)(cid:85)(cid:86)

The presence of irregularities in a series is often a good indicator of bad quality. Series
containing  "strange"  values  or  outliers  are  generally  difficult  to  model  and  thus  fairly
useless. This aspect has been stressed also by database users (see ECB(2000b)). Of course,
such an indicator is not supposed to have any "objective" value. It is cannot be anything
else  than  a  monitoring  indicator  that  can  draw  the  attention  to  quite  some  series  and
therefore detect possible errors.
To obtain an irregular or "strange" values indicator the database has been screened with
the TRAMO software to detect the number of irregular values (outliers) in each series.
TRAMO  can  detect  different  kinds  of  "irregularities"  in  time  series,  such  as  additive
outliers,  level  shifts  and  transitory  changes.  However,  only  a  (relevant)  subset  of  the
database has been analysed for obvious reasons. Inactive series and series with missing
values were excluded and so were series with missing values as well as some series that
displayed such an irregular pattern that they could not be properly analysed by the TRAMO
software. Table 6 shows the results for the different domains of the database. They vary a
lot from one domain to another, and they are, at least for some domains, not so satisfactory.

19

’RPLQLTXH(cid:3)/DGLUD\(cid:15)(cid:3))DELR(cid:3)6DUWRUL

However, as the TRAMO software has been used with the default settings, an (cid:68)(cid:71)(cid:3)(cid:75)(cid:82)(cid:70)
tuning of the parameters may be needed to compute a somewhat better quality indicator.

Number of irregular values
Domain
Balance of Payments
Business and Consumer Surveys
Energy
External Trade
Industry
Labour Cost
Employment, Construction, M
Employment, Construction, Q
Employment, Industry, M
Employment, Industry, Q
Monetary and Financial Indicators
National Accounts
Prices
Retail sales Index
Unemployment
Total

2
%

1
None
%
%
31.1 27.4 19.9
28.4 23.1 19.1
22.5 35.0 13.8
18.3 16.3 21.4
24.1 20.3 17.8
58.4 23.8 13.9
. 31.3 12.5
9.1
7.0 11.7
38.6 24.9 15.2
31.7 20.0 18.3
47.2 23.8 14.2
18.9 15.8 14.5
41.8 24.1 12.3
36.3 22.5 11.3
33.0 21.5 15.7

27.3 40.9
8.0

3 and more
%

4
%
9.2
4.6
5.6
7.3
7.5
10.0
14.6 11.5
8.1
12.7
2.0
2.0
6.3
6.3
9.1
4.5
7.7
10.3
6.6
10.2
1.7
11.7
4.0
6.9
10.0 10.3
5.4
5.0
6.5

8.9
8.1
9.2

5 or more Total

%

N
412
7.8
550
16.5
80
11.3
18.0
295
17.0 1645
101
.
16
43.8
22
9.1
426
55.2
591
4.6
16.7
60
4.0 2543
739
30.4
316
7.6
16.9
160
14.1 7956

(cid:55)(cid:68)(cid:69)(cid:79)(cid:72)(cid:3)(cid:25)(cid:29)(cid:3)Distribution of the series by number of irregular values in the Euro-SICS database
(by domain as of 01/03/2001)

(cid:25)(cid:17)(cid:21)(cid:3) (cid:53)(cid:72)(cid:89)(cid:76)(cid:86)(cid:76)(cid:82)(cid:81)(cid:86)

absolute errors, used to evaluate the accuracy of preliminary growth rates.

relative errors, giving information on the accuracy of preliminary estimates of

Time series of revisions must be tested for any systematic behaviour. As Eurostat considers
this to be important a strategy for the analysis of revisions has been described in Eurostat
(1999b) for the National Accounts domain. This strategy can, of course, be generalised to
other kinds of series beyond the National Accounts domain. The proposed approach is
based on the studies of Patterson (1992), Di Fonzo, Pisani and Savio (1994) and many
others. Summary statistics can be used to compare between preliminary and final estimates.
Two types of errors are distinguished in Eurostat (1999b):
a)
levels;
b)
For each error type (level and growth rate) a series of accuracy indices for the preliminary
estimate can  be  compiled  based  on  (1)  the  mean  relative  error,  (2)  the  mean  absolute
relative error, (3) the standard deviation of the relative error, (4) the square root of the
mean quadratic relative error, and (5) the bias component of the mean quadratic relative
error.  Clearly,  all  these  measures  cannot  be  considered  as  reliability  measures  of
provisional estimates. However, they allow to find out if revisions have always (or almost
always) the same sign. If this is the case, a correction of preliminary estimates would be
needed. The compilation of such revision indicators requires a daily checking of the entire
database to capture the complete history of each series. Thus it seems difficult to compile
such indicators for all database series . Therefore some further tests are required to select
the a group of the series for which this in-depth revision analysis will be performed.

20

4XDOLW\(cid:3)LQ(cid:3)6KRUW(cid:16)WHUP(cid:3)VWDWLVWLFV(cid:29)(cid:3)7KH(cid:3)(XUR(cid:16)6,&6(cid:3)GDWDEDVH(cid:3)RI(cid:3)(XURVWDW

(cid:37)(cid:76)(cid:69)(cid:79)(cid:76)(cid:82)(cid:74)(cid:85)(cid:68)(cid:83)(cid:75)(cid:92)

[1] 

[2] 

[3] 
[4] 
[5] 

[6] 

[7] 

[8] 

[9] 

[10] 

[11] 

European Central Bank (2000a)(cid:15)(cid:3)(XUR(cid:16)6,&6(cid:29)(cid:3)SURSRVHG(cid:3)FRYHUDJH(cid:3)DQG(cid:3)IHDWXUHV(cid:3)IRU(cid:3)LPSOHPHQWDWLRQ(cid:3)E\
HQG(cid:16)(cid:21)(cid:19)(cid:19)(cid:19), 8 September 2000, note ST/GEFS/WGGES/GES/SICS1
European  Central  Bank  (2000b)(cid:15)(cid:3) (XUR(cid:16)6,&6(cid:29)(cid:3) FRPPHQWV(cid:3) RQ(cid:3) (XURVWDW¶V(cid:3) 4XDOLW\(cid:3) 5HSRUW,
7 December 2000
Eurostat (1999a), +RZ(cid:3)WR(cid:3)ILOO(cid:3)LQ(cid:3)WKH(cid:3)4XDOLW\(cid:3)5HSRUW, Unit A4.
Eurostat (1999b), +DQGERRN(cid:3)RI(cid:3)4XDUWHUO\(cid:3)1DWLRQDO(cid:3)$FFRXQWV, Unit A4.
Eurostat  (2000a),  ’HILQLWLRQ(cid:3) RI(cid:3) 4XDOLW\(cid:3) LQ(cid:3) 6WDWLVWLFV,  Working  Party  Assessment  of  the  Quality  in
Statistics, 4-5 April 2000, Unit A4.
Eurostat (2000b), 6WDQGDUG(cid:3)4XDOLW\(cid:3)5HSRUW, Working Party Assessment of the Quality in Statistics, 4-5
April 2000, Unit A4.
Eurostat (2001a), $QDO\VLV(cid:3)RI(cid:3)0DLQ(cid:3)5HVXOWV(cid:3)RI(cid:3)WKH(cid:3)(XUR(cid:16)6,&6(cid:3)XVHUV(cid:3)VXUYH\, Euro-SICS Working Party,
15-16 January 2001, Document 047/01, Unit A6.
Eurostat (2001b), ([WHQVLRQ(cid:3)RI(cid:3)WKH(cid:3)/LVW(cid:3)RI(cid:3)&RXQWU\(cid:3)6SHFLILF(cid:3),QGLFDWRUV, Euro-SICS Working Party, 15-16
January 2001, Document 051/01, Unit A6.
Eurostat (2001c), (XUR(cid:16)6,&6(cid:3)GHWDLOHG(cid:3)$QDO\VLV(cid:3)DV(cid:3)DW(cid:3)(cid:21)(cid:26)(cid:3)1RYHPEHU(cid:3)(cid:21)(cid:19)(cid:19)(cid:19), Euro-SICS Working Party,
15-16 January 2001, Document 051/01, Unit A6.
Eurostat (2001d), $(cid:3)0RQLWRULQJ(cid:3)5HSRUW(cid:3)IRU(cid:3)0HPEHU(cid:3)6WDWH(cid:3)5HSUHVHQWDWLYHV, Euro-SICS Working Party,
15-16 January 2001, Document 053/01, Unit A6.
Grünewald, W., (2000), )UHVKQHVV(cid:3)RI(cid:3)1(:&52126(cid:3)GDWD(cid:29)(cid:3)$QDO\VLV(cid:3)RI(cid:3)WKH(cid:3)VLWXDWLRQ(cid:3)DV(cid:3)RI(cid:3)$SULO(cid:3)(cid:21)(cid:19)(cid:19)(cid:19),
internal note, Eurostat.

21