Quality report on European statistics on research and development : 2024 edition.
Date: 2024-07-22
Institutions: Eurostat
Subjects: EU research policy, EU statistics, Eurostat, database, dissemination of EU information, innovation, metadata, qualitative analysis, report, research and development, research expenditure (EU), research staff

P

u

b

l

i

c

a

t

i

o

n

t

i

t

l

e

Quality report on  
European statistics on 
research and development

2024 edition

S TAT I S T I C A L   
R E P O R T S

2023 edition 
Quality report on  
European statistics on 
research and development 

2024 edition 

 
 
 
 
 
Manuscript completed in July 2024 

The Commission is not liable for any consequence stemming from the reuse of this publication. 

Luxembourg: Publications Office of the European Union, 2024 

© European Union, 2024 

The reuse policy of European Commission documents is implemented based on Commission Decision 
2011/833/EU of 12 December 2011 on the reuse of Commission documents (OJ L 330, 14.12.2011, p. 
39). Except otherwise noted, the reuse of this document is authorised under a Creative Commons 
Attribution 4.0 International (CC-BY 4.0) licence (https://creativecommons.org/licenses/by/4.0/). This 
means that reuse is allowed provided appropriate credit is given and any changes are indicated. 

For any use or reproduction of elements that are not owned by the European Union, permission may 
need to be sought directly from the respective rightholders. The European Union does not own the 
copyright in relation to the following elements: 

Copyright for the photographs: Cover © Adobe stock/choknitiy 

Theme: Science, technology, and digital society 
Collection: Statistical reports 

PDF: 

ISBN 978-92-68-19018-0  ISSN 2529-3222   doi: 10.2785/948814   KS-FT-24-005-EN-N 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Foreword 

Research and experimental development (R&D) comprise creative and systematic work 
undertaken in order to increase the stock of knowledge – including knowledge of humankind, 
culture and society – and to devise new applications of available knowledge. R&D statistics play 
a central role in measuring the expenditure and the employment in R&D activities. The availability 
and quality of the R&D statistics are key to benchmarking the relation between R&D expenditure 
and innovation.   

This is the first edition of the quality report for R&D statistics. It aims to provide users with 
relevant information on the quality of the R&D data disseminated in Eurostat’s online database 
under the Science and technology domain.  

The indicators included in the report are the quality indicators used within the European business 
statistics (EBS) quality framework. They relate to the 2021 reference year, which was the first 
reference year of implementation of the EBS Regulation (Regulation (EU) 2019/2152 on 
European Business Statistics1). The report contains core information both for the producers and 
users of R&D statistics, serving as a practical reference document not only for national statistical 
authorities but also for a wide array of users as a guide to R&D data. 

The indicators were calculated by Eurostat based on the data reported by the European Union 
(EU) Member States, EFTA countries and EU enlargement countries. Unit G4 Innovation and 
digitalisation would like to take the opportunity to thank the national experts in R&D statistics for 
providing the data and filling in the national metadata reports and for their valuable comments on 
this document. Their work and efforts ensured the successful preparation of this report. 

1 Regulation (EU) 2019/2152 of the European Parliament and of the Council of 27 November 2019 on European business statistics, 
repealing 10 legal acts in the field of business statistics Regulation - 2019/2152 - EN - EUR-Lex (europa.eu) 

Quality report on European statistics on research and development, 2024 Edition 

3 

   
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Keywords  

R&D expenditure, R&D employment, EBS quality framework, national metadata (quality 
assessment)  

Acknowledgements   

Authors 

Alvaro Diez Soto, Daniela Enache, Stefania Panaitescu, Sorina-Carmen Vâju, Bertrand Brou 

Editors:  

Annika Johansson and Catherine Coyette 

Copy editing and layout  

Daniela D’Acunto  

Contact details  

Eurostat 

Statistical Office of the European Union 

Joseph Bech Building 

5, rue Alphonse Weicker 

2721 Luxembourg 

E-mail: estat-user-support@ec.europa.eu 

4                               Quality report on European statistics on research and development, 2024 Edition 

  
 
 
 
 
 
 
 
Table of contents 

1.Introduction ................................................................................................................................................. 8 

2.Coverage ................................................................................................................................................... 10 

3.Legal basis and guidelines ...................................................................................................................... 11 

4.Relevance and completeness ................................................................................................................. 12 

4.1. Relevance ...................................................................................................................................... 12 
4.2. Completeness – Data completeness rate and Overview ............................................................... 16 
4.3. Conclusions on Relevance and Completeness ............................................................................. 22 

5.Accuracy .................................................................................................................................................... 23 

5.1. Assessment of the accuracy with regards to the main indicators .................................................. 23 
5.2. Compilation methods ..................................................................................................................... 24 
5.3. Statistical errors ............................................................................................................................. 26 
5.4. Conclusions on Accuracy ............................................................................................................... 31 

6.Coherence and comparability ................................................................................................................. 32 

6.1. Comparability over time ................................................................................................................. 34 
6.2. Coherence ..................................................................................................................................... 38 
6.3. Conclusions on Coherence and Comparability ............................................................................. 45 

7.Timeless and punctuality ......................................................................................................................... 46 

7.1. Timeliness ...................................................................................................................................... 46 
7.2. Punctuality ..................................................................................................................................... 48 
7.3. Conclusions on Timeliness and Punctuality .................................................................................. 49 

8. Accessibility and Clarity ......................................................................................................................... 50 

8.1. Accessibility .................................................................................................................................... 50 
8.2. Clarity ............................................................................................................................................. 52 
8.3. Conclusions on Accessibility and Clarity........................................................................................ 52 

9. Conclusions on the Summary Quality Report ...................................................................................... 54

Quality report on European statistics on research and development, 2024 Edition 

5 

   
 
 
 
 
 
 
 
 
 
 
Glossary 

BES 
BESSI 
Section 72   

Candidate countries  

CIS 
CV 
EBS GIA 
ECB 
EFTA 
ESS 
EU 
Eurobase 
FM2015 
FORD 
FTE 
GERD data  
GOV 
GOVSI 
HC 
HERD data 
HES 
HESSI 
IFATS 
ILO 
IM_RND_EXPN 
IMF 
MH 
NACE 
NUTS 
NSIs 
OECD 
PNP 
PNPSI 

Business enterprise sector 
SIMS reference metadata files for the Business enterprise sector 
NACE Code 72 - Scientific research and development 
Albania, Bosnia and Herzegovina, Montenegro, North Macedonia, Serbia and 
Türkiye 
Community Innovation Survey 
Coefficient of variation  
Commission Implementing Regulation (EU) No 2020/1197 
European Central Bank 
European Free Trade Agreement countries: Iceland, Norway, and Switzerland 
European Statistical System 
European Union  
Eurostat’s online database  
Frascati Manual 2015  
Fields of research and development  
Full time equivalent 
Expenditure data for GOV  
Government sector 
SIMS reference metadata files for government sector 
head count 
Expenditure data for HES  
Higher education sector 
SIMS reference metadata files for the higher education sector 
Inward Foreign Affiliates Statistics  
International Labour Organization 
Intramural R&D expenditures  
International Monetary Fund 
Eurostat’s Metadata Handler 
Statistical Classification of Economic Activities in the European Community 
Nomenclature of Territorial Units for Statistics  
National Statistical Institutes  
The Organisation for Economic Co-operation and Development 
Private non-profit sector 
SIMS reference metadata files for private non-profit sector 

6                               Quality report on European statistics on research and development, 2024 Edition 

  
 
 
 
 
 
 
 
 
 
 
 
 
R&D 
RND_PER 
SIMS 
UN 
USS 

Research and development 
R&D personnel  
Single integrated metadata structure  
United Nations 
User Satisfaction Survey  

Quality report on European statistics on research and development, 2024 Edition 

7 

 
   
 
 
1 

 Introduction 

1 

Introduction 

1. Introduction 

The present Quality report on European statistics on research and development, 2024 edition, 
looks at the quality of the R&D data reported by the Member States, EFTA countries and 
candidate and potential candidate countries of the European Union. The report is meant to be a 
useful tool for data producers and users alike.   

The quality concept applied in this report is in conformity with the common quality framework of the 
European Statistical System2. According to this framework, the quality of a statistical output 
includes the following components: relevance and completeness, accuracy and reliability, 
timeliness and punctuality, comparability and coherence, accessibility, and clarity. Each quality 
component consists of several sub-components and is shortly explained at the start of the 
respective section in this report3. 

R&D statistics provide an overview of the research and development expenditure spent within 
the European Union (EU) across various economic sectors, as well as the number of R&D 
personnel and researchers engaged on the EU territory. The national R&D data is collected 
annually by Eurostat from the National Statistical Institutes (NSIs) and published on Eurostat's 
website together with EU aggregated figures.  

Together with the data, the NSIs provide to Eurostat the national reference metadata/quality 

2 ‘The common quality framework of the European Statistical System is composed of the European Statistics Code of Practice, the 
Quality Assurance Framework of the European Statistical System and the general quality management principles (such as continuous 
interaction with users, commitment of leadership, partnership, staff satisfaction, continuous improvement, integration and harmonisa-
tion).’  Available at: https://ec.europa.eu/eurostat/web/products-catalogues/-/KS-02-18-142 
3 Most of the introductory texts shortly explaining each quality component are taken from the ‘ESS handbook for Quality Reports’, 
available at: European Statistical System (ESS) handbook for quality and metadata reports — 2020 edition - Products Manuals and 
Guidelines - Eurostat (europa.eu) 

8                             Quality report on European statistics on research and development, 2024 Edition 

  
 
 
 
 
 
 
 
 
 
 
 
 
 
Introduction    1 

reports, in Eurostat’s Metadata Handler (MH) which is the repository of the metadata/quality 
reports. These reference metadata files present the methods used to collect and process the R&D 
data until their dissemination. This is an instrument used to assure the users that the R&D data 
disseminated on Eurostat's website are comparable across countries and are of high quality.  

The present report is structured as follows: Introduction, Coverage, Legal basis, and a quality 
assessment based on the following output-related concepts of quality: Relevance and 
Completeness, Accuracy and Reliability, Timeliness and Punctuality, Comparability and 
Coherence, Accessibility and Clarity.  

The individual country metadata/quality reports for R&D statistics for reference year 2021 were 
submitted to Eurostat by the NSIs at the end of October 2023. They are the main source for the 
present report. The most recent ESS reference metadata format is the single integrated metadata 
structure (SIMS) and this applied for the reference year 2021.  

The metadata/quality reports were published in Eurostat’s online database on 8 March 2024, 
attached to the R&D European metadata file:  

as annexes to the file:  

The present metadata/quality report closely follows the standard quality report format developed 
by Eurostat. The goal of this standard format is to report homogeneous production processes 
within each country. However, information from the individual countries is, in some cases, not 
sufficient to provide a comprehensive summary.  

Eurostat wishes to thank all experts in the countries for their participation, for providing the data 
and metadata, the descriptions as well as for their essential support in compiling this report. 

Quality report on European statistics on research and development, 2024 Edition 

9 

 
   
 
 
 
 
  
 
 
 
Introduction    1 

2 

Coverage 

2. Coverage  
This report covers the thirty-six participating countries that provided Eurostat with R&D statistics in 
2021: the Member States of the European Union (EU), three EFTA countries (Iceland, Norway, 
and Switzerland) and six candidate countries (Albania, Bosnia and Herzegovina, Montenegro, 
North Macedonia, Serbia and Türkiye). R&D expenditure and personnel data are collected by 
national statistical authorities. The data are collected through sample or census surveys, from 
administrative registers or through a combination of sources. R&D statistics are compiled for four 
institutional sectors of performance: business enterprise (BES); government (GOV); higher 
education (HES) and private non-profit (PNP). These sectors are defined based on the System of 
National Account (SNA), with the difference that higher education has been established as a 
separate sector because of its policy relevance, and households have, by convention, been 
merged with the private non-profit (PNP) sector. The report does not cover the publicly funded 
R&D.  

10                             Quality report on European statistics on research and development, 2024 Edition 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Introduction    1 

3 

Legal basis and guidelines  

3. Legal basis and guidelines  
The collection of R&D statistics is based on Commission Implementing Regulation (EU) No 
2020/1197 of 30 July 2020 (EBS GIA). The Regulation sets the framework for the collection of 
R&D statistics and specifies the main variables of interest and their breakdowns at predefined level 
of detail. 

R&D data are compiled in accordance to the guidelines laid down by OECD (2015), Frascati 
Manual 2015: Guidelines for Collecting and Reporting Data on Research and Experimental 
Development, The Measurement of Scientific, Technological and Innovation Activities and 
Eurostat’s  European Business Statistics Methodological Manual for R&D Statistics.  

 Quality report on European statistics on research and development, 2024 Edition 

11 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
4 

Relevance and completeness 

4 

Relevance and completeness 

4.   Relevance and completeness  

4.1. Relevance 

Relevance is the degree to which statistics meet current and potential users’ needs. It includes the 
production of all needed statistics and the extent to which concepts used (definitions, 
classifications etc.) reflect user needs. The aim is to describe the extent to which the statistics are 
useful to, and used by, the broadest array of users. For this purpose, statisticians need to compile 
information, firstly about their users (who they are, how many they are, how important is each one 
of them), secondly on their needs, and finally to assess how far these needs are met. 

4.1.2. Users’ Needs  
When assessing the users of the R&D data and the users’ needs, the following type of users are 
identified in the SIMS metadata/quality report for the different sectors of R&D statistics:   

Mainly institutions at: 

•  European level: European Commission (Directorates Generals, Secretariat General), 
European Council, European Parliament, European Central Bank, other European 
agencies etc.,   

•  Member States level, at the national or regional level: ministries of economy or finance, 
other ministries (for sectoral comparisons), NSI and other statistical agencies (norms, 
training, etc.), and 

• 

International organisations: OECD, UN, IMF, ILO, etc.  

Then, the second main users are social actors: employers’ associations, trade unions, 
lobbyists, among others, at the European, national, or regional level. 

Followed by the media: International or regional media – specialized or for the public – interested 
both in figures and analyses or comments. The media are the main channels of statistics to the 
public.  

12                             Quality report on European statistics on research and development, 2024 Edition 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Relevance and completenes   4 

On the 4th position are researchers and students (Researchers and students need statistics, 
analyses, ad hoc services, access to specific data).   

Enterprises or businesses are on 5th position (Either for their own market analysis, their 
marketing strategy (large enterprises) or because they offer consultancy services) 

On the 6th position are other users (a user class defined for national purposes, different from 
the previous classes).  

When looking at the information reported by Member States, regardless of the reporting country, 
the users who make the most use of R&D data are:  

• 

Institutions – with Eurostat and other European Commission’ s Directorates General 
disputing the first place with national/federal authorities. 

•  Social actors - a wide variety of actors such as trade unions, lobbyist, banks, business 

association.  

For eight EU Member States the second most important users were the media (national and 
regional). For six Member States the second most important user were researchers and 
students/PhD students - Universities.  For the Higher education sector one country listed as the 
most important users the researchers, while the great majority of countries listed as most important 
users the Institutions.   

4.1.2.1. EU Users’ needs and satisfaction 
To evaluate if users' needs have been satisfied, the best way is to conduct user satisfaction 
surveys. 

EUROSTAT USERS AND THEIR NEEDS - REPORT ON EUROSTAT USER SATISFACTION SURVEY  

The most recent Eurostat User Satisfaction Survey (USS) dates from 2022. The USS did not 
include direct questions about the R&D data. There were 1 486 replies - almost 50% more than in 
the previous USS in 2019 - and many aspects of data dissemination received the highest 
satisfaction rating since the beginning of user satisfaction surveys at Eurostat. 

The survey was designed to increase knowledge about users, their needs and their satisfaction 
with Eurostat's services. The survey covered four main aspects: 

• 

information on types of users and uses of European statistics; 

•  quality; 

• 

trust in European statistics; 

•  dissemination of statistics. 

Almost half of the respondents were students, academic and private users, followed by users 
working for a government (22%) and for EU and international organisations (14%). Looking at user 
types, most of the respondents identified themselves as intermediate users (38%), followed by 
advanced users (35%) and light users (27%). 

72% of all users indicated that European statistics were either 'essential' or 'important' for their 
work. About a third of the users stated they used European statistics in their daily or weekly 
activities, 29% use them on a monthly basis and the remaining 36% at other intervals. 

User assessment of the quality and user friendliness of Eurostat's products was generally positive, 
too, with shares of 'very good/good' replies at 60% or above. The 'Eurostat euro indicator releases' 
and 'Eurostat database' were rated highest for quality, followed by 'Statistics Explained' articles. 

 Quality report on European statistics on research and development, 2024 Edition 

13 

 
 
4 

Relevance and completeness 

Users continue to trust European statistics: 95% of users said they trusted European statistics 
greatly or tended to trust them. 

Recently Eurostat has launched the 2024 USS4 which was accessible to all users for a delimited 
period. Eurostat encourages users to participate in general in USS. The USS surveys are an 
excellent tool for improving the quality of data and services with a focus on the data quality 
indicators such as timeliness, comparability and completeness and accessibility of the data, 
doubled by a focus on specific users’ needs.  

Furthermore, Eurostat has also launched a short survey on the metadata files and a recruitment 
form of metadata users, with information sessions taking place in May and June 2024. The form to 
participate were accessed via the metadata files or at a specified link5.  

Eurostat has put in place designated services6 which receive inquiries from users via the 
Eurostat's User Support Team. Thus, Eurostat has first-hand information on how users use 
Eurostat data for analyses, news articles, reports, etc. For the reference year 2021 the User 
Support Team has received inquiries from, among other countries, users in France, Italy, UK, 
Poland, Portugal, Luxembourg, and the United States.  

Another service offered by Eurostat is a tool7 to offer support to representatives of the media and 
social actors and to other European Institutions.  

Lastly, Eurostat Media Support Team is dedicated to the mass media users of Eurostat data. All 
these channels are also used to evaluate the users’ satisfaction with Eurostat data. In 2021 
Eurostat received 33 inquiries into R&D data.  

4.1.2.2. Country level Users’ needs and satisfaction 
User satisfaction with the national R&D statistics is reported as high. In most countries, this 
assessment is based not on specific user satisfaction survey but on ad-hoc feedback received 
from users. 

Although the great majority of countries do not conduct dedicated national USS for R&D data, the 
NSIs make all efforts to meet the needs of their users in an ongoing process via several actions 
especially regular meetings with the stakeholders. The NSIs are aware of the users’ needs and 
they monitor that the users are satisfied with the published R&D data. This information results 
either from general USS (most recently online ones) carried out by NSIs (9 NSIs), or from annual 
workshops or seminars and other regular consultations the NSIs have with representatives from 
business, science, and politics (five NSIs).  

One NSI conducted between October 2022 and January 2023 a user satisfaction survey on all its 
statistical products among 381 experts. Included were four questions on the topics ‘Research, 
Innovation, Digitalisation’ with the following results: percentage of users assessing the following 
dimensions with ‘very good’ or ‘good’: Timeliness: 78%; Accuracy: 75%; Comparability: 78%; 
Quality: 75%.  

Another NSI conducted a USS that covered several statistical areas, where products of R&D 

4 Have your say with the 2024 user satisfaction survey - Eurostat (europa.eu) 
5 EUSurvey - Survey (europa.eu) 
6 Contact us - Eurostat (europa.eu) 
7 Press services - Eurostat (europa.eu) 

14                             Quality report on European statistics on research and development, 2024 Edition 

 
 
 
 
 
Relevance and completenes   4 

statistics were included together with innovation, energy, and environment statistics in one 
category and the average rate of user satisfaction with products of these statistics was 68,9 %.  

A third NSI conducted a USS specific for R&D data and collected information on how interested 
users were in current R&D figures and time series. The survey concluded that commercial 
associations were interested in R&D data of their own NACE8 class and that users were interested 
in the methodology of the R&D survey. 

Even if dedicated and regular R&D users’ satisfaction surveys are not carried out, the statistical 
program is announced every year by the NSIs and is given for consultation to ministries, 
universities, and scientists, who can put forward suggestions which are taken into consideration 
and the statistical plan may be changed accordingly.  

Another source of information from the users are the discussions with the representatives of 
ministries and other political actors (four NSIs). In one Member State, as a result of the workshops, 
some additional breakdowns have been added to the R&D questionnaires. In another Member 
State the data was produced at a more detailed level of the fields of research and development 
(FORD) because of user needs.  

In the case of another NSI there exists a specific users’ council for R&D statistics that advises the 
NSI. For another NSI the R&D statistics fall within the scope of the Statistical Advisory Committee 
on Research and Development Activities and Technologies (however last meeting of the 
Committee was in 2021). Results showed that R&D indicators are used at least once every three 
months by institutional users, and they are considered very important. 

The specific needs of users are also considered when revising the survey design (two NSIs), in 
order to adapt the content of the survey to their specific requirements increasing the level of 
satisfaction. The results of the surveys and workshops are applied to provide better services for 
users as well as the improvement of statistical products.  

In addition to this, each year, in two NSIs, the statistical units surveyed receive a link to the NSIs 
website with a summary of the main results of the survey.  

Moreover, in the R&D survey the surveyed units can give feedback in the additional field that is 
provided in the survey (two NSIs).  

More recent developments to help measure users’ satisfaction were mentioned by an NSI and also 
by Eurostat: the official statistics portal traffic was monitored, website visitor opinion polls, general 
opinion polls on the products and services, target user groups’ opinion polls and other IT-based 
uses measurements are conducted.  

When it comes to specific sectoral user satisfaction, for the Higher education sector (HES) 
statistics one NSI organised a survey on data use and customer satisfaction in 2023 but they did 
not have the results yet. Still for this sector, other NSIs have mentioned in their HESSI reference 
metadata files that no additional demands on R&D statistics on HES were addressed to them and 
the comments they received from key users (ministries, universities) proved that users were 
satisfied with the available information on R&D-related variables. Another NSI reported they 
reduced the burden of data delivery by using more administrative sources of data.  

NSIs reported that in general users are satisfied with the R&D statistics for HES. One NSI reported 
having published in 2023 the expenditure data for HES (HERD data) earlier than in previous years 

8 NACE (Statistical classification of economic activities in the European Community) 

 Quality report on European statistics on research and development, 2024 Edition 

15 

 
 
 
 
 
 
4 

Relevance and completeness 

and launching several new statistics on an annual basis, e.g., on diversity in research personnel 
and a monitoring system for researcher recruitment. 

The NSIs report in their metadata reports, the complex balancing they do between satisfying new 
users’ requests and maintaining a correct reporting burden for the statistical units.   

Occasionally some users want timelier data or more detailed breakdowns but accept the difficulties 
in reducing the duration of the production process (feedback received by three NSIs). In such a 
case one NSI conducted additional surveys to the ordinary R&D surveys and combined the data, 
but this NSI reported the conflict between user needs and response burden.  

We can conclude that, irrespective of the formula the NSIs use for monitoring the users’ 
satisfaction with the R&D statistics, the NSIs do make efforts to assess the users ‘needs and the 
users’ satisfaction, and adapt, wherever possible, their statistical products accordingly. Both the 
NSIs and Eurostat are reassured that their main users, when asked to comment on the overall 
quality of the R&D data published, the users are satisfied that the data were useful, on time and in 
sufficient detail.   

4.2. Completeness – Data completeness rate and 
Overview  

Completeness is assessed via the comparison of the data delivered against the requirements of 
Commission Implementing Regulation (EU) No 2020/1197. Data completeness of both preliminary 
and final mandatory data are very good. Optional variables in the BES, GOV and HES are less 
complete, as a few countries do not collect part or all of these data in their national surveys. 

Most of the NSIs reported 100% completeness rate for R&D data for 2021 reference year.  

The completeness rate for Intramural R&D expenditure was calculated based on the data 
requirements listed for the breakdowns on Annex I, Part B. Elements of data requirements, Table 
18, and Table 31 of the Regulation (EC) 2020/1197, respectively.  

-  TABLE 1: Completeness rate overview for Intramural R&D expenditure, by country, 2021 

-  TABLE 3: Completeness rate overview for Regional intramural R&D expenditure, by 

country, 2021 

The completeness rate for R&D personnel and researchers was calculated based on the data 
requirements listed for the breakdowns on Annex I, Part B. Elements of data requirements, Table 
19 and at Table 32 of the Regulation (EC) 2020/1197.  

-  TABLE 2: Completeness rate overview for R&D personnel and researchers, by country, 

2021 

-  TABLE 4: Completeness rate overview for Regional R & D personnel and Researchers by 

country, 2021

16                             Quality report on European statistics on research and development, 2024 Edition 

 
 
 
 
Relevance and completenes   4 

4.2.1 Overview of Completeness  
Most of the NSIs scored 100% completeness rate when reporting the R&D expenditure data with 
the breakdown by sector of performance. In all, 9 countries scored a completeness rate of 80.00% 
for the sectors of performance indicators - please refer to TABLE 1: Completeness rate overview 
for Intramural R&D expenditure, by country, 2021.  

Regarding reporting for the source of funds, a majority of NSIs scored 100% completeness rate. 
Denmark scored the lowest with only 9.62%, seven Member States scored 80.77% and four 
scored over 50%.

 Quality report on European statistics on research and development, 2024 Edition 

17 

 
 
4 

Relevance and completeness 

TABLE 1:  
Completeness rate overview for Intramural R&D expenditure, by country, 2021   

R&D expenditure breakdowns9 by 

-
b
o

i

c
m
o
n
o
c
e
o
c
o
s

i

)
l
a
n
o
i
t
p
o
(

e
v
i
t
c
e

j

Legend:   
Very good 
Good 
Satisfactorily 
Unsatisfactorily 
Optional: fully reported 
Optional: partially reported 

  Missing data 

l

s
s
a
c
e
z
s

i

d
n
a

f
o

l

d
e
i
f

j

r
o
a
m

D
&
R

n
o
i
t
a
t
n
e
i
r
o

y
r
t
s
u
d
n

i

)
l
a
n
o
i
t
p
o
(

l

s
s
a
c
e
z
s

i

d
n
u
f

f
o

e
c
r
u
o
s

Country 

f
o

r
o
t
c
e
s

e
c
n
a
m
r
o
f
r
e
p

s
d
n
u
f

f
o

e
c
r
u
o
s

D
&
R

f
o

e
p
y
t

t
s
o
c

f
o

e
p
y
t

y
t
i
v
i
t
c
a

BE 
BG 
CZ 
DK 
DE 
EE 
IE 
EL 
ES 
FR 
HR 
IT 
CY 
LV 
LT 
LU 
HU 
MT 
NL 
AT 
PL 
PT 
RO 
SI 
SK 
FI 
SE 

CH 
NO 
IS 

BA 
ME 
MK 
RS 
TR 

 Source: Eurostat’s online database (Eurobase), (R&D) (rd) 

9 Country-level business statistics on intramural R&D expenditure, Annex I, Part B. Elements of data requirements, Table 18 of the 
Regulation (EC) 2020/1197. 

18                             Quality report on European statistics on research and development, 2024 Edition 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Relevance and completenes   4 

TABLE 2:   
Completeness rate overview for R&D personnel and researchers, by country, 2021  

R & D personnel and researchers breakdowns 10 by 

l

s
s
a
c
e
z
s

i

e
c
n
a
m
r
o
f
r
e
p
f
o

r
o
t
c
e
s

r
e
d
n
e
g

d
n
a

d
n
a

f
r
e
p
f
o

r
o
t
c
e
s

)
l
a
n
o
i
t
p
o
(

s
n
o
i
t
a
c
i
f
i
l

a
u
q

_
e
c
n
a
m
o
r
f
r
e
p
f
o

r
o
t
c
e
s

_
s
n
o
i
t
a
c
i
f
i
l

a
u
q

)
l
a
n
o
i
t
p
o
(

r
e
d
n
e
g

_
e
c
n
a
m
r
o
f
r
e
p
f
o

r
o
t
c
e
s

p
u
o
r
g

e
g
a

)
l
a
n
o
i
t
p
o
(

r
e
d
n
e
g
_

_
e
c
n
a
m
r
o
f
r
e
p
f
o

r
o
t
c
e
s

i

_
p
h
s
n
e
z
t
i
c

)
l
a
n
o
i
t
p
o
(

r
e
d
n
e
g

 Legend: 
 Very good 
 Good 
 Satisfactorily 
 Unsatisfactorily 
 Optional: fully reported 
Optional: partially reported 
Missing data 

    Country 

e
c
n
a
m
r
o
f
r
e
p
f
o

r
o
t
c
e
s

e
c
n
a
m
r
o
f
r
e
p
f
o

r
o
t
c
e
s

n
o
i
t
a
p
u
c
c
o

d
n
a

p
u
c
c
o
_
f
r
e
p
f
o

r
o
t
c
e
s

r
e
d
n
e
g
_

y
t
i
v
i
t
c
a

d
n
a
D
&
R

f
o

l

d
e
i
f

j

r
o
a
m

r
e
d
n
e
g

r
e
d
n
e
g
_
y
t
i
v
i
t
c
a

BE 
BG 
CZ 
DK 
DE 
EE 
IE 
EL 
ES 
FR 
HR 
IT 
CY 
LV 
LT 
LU 
HU 
MT 
NL 
AT 
PL 
PT 
RO 
SI 
SK 
FI 
SE 

CH 
NO 
IS 

AL 
BA 
ME 
MK 
RS 
TR 

Source: Eurostat’s online database (Eurobase), (R&D) (rd) 

10 Country level business statistics on employment in R&D, Annex I, Part B. Elements of data requirements, Table 19 of the Regulation 
(EC) 2020/1197. 

 Quality report on European statistics on research and development, 2024 Edition 

19 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
4 

Relevance and completeness 

The picture of completeness for R&D personnel is more complex than the one for R&D 
expenditure. In this case the majority of the NSIs scored 100% completeness rate when reporting 
the R&D personnel data with the breakdown by sector of performance. In all, eleven countries 
scored a completeness rate of 80.00% for the sectors of performance indicators - please confer 
TABLE 2: Completeness rate overview for R&D personnel and researchers, by country, 2021. 

TABLE 3:   
Completeness rate overview for Regional intramural R&D expenditure, by country, 2021  

Country 

Regional Intramural R&D expenditure by 
breakdowns 11  

  Legend: 
  Very good 
  Good 
  Satisfactorily 
  Unsatisfactorily 

BE 
BG 
CZ 
DK 
DE 
EE 
IE 
EL 
ES 
FR 
HR 
IT 
CY 
LV 
LT 
LU 
HU 
MT 
NL 
AT 
PL 
PT 
RO 
SI 
SK 
FI 
SE 

CH 
NO 
IS 

AL 
BA 
ME 
MK 
RS 
TR 

   Source: Eurostat’s online database (Eurobase), (R&D) (rd) 

In the case of the Regional intramural R&D expenditure we note that the great majority of countries 
(30) scored very good and good for the completeness rate. 

11 Regional business statistics on R&D expenditure, Annex I, Part B. Elements of data requirements, Table 31 of the Regulation (EC) 
2020/1197. 

20                             Quality report on European statistics on research and development, 2024 Edition 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Relevance and completenes   4 

TABLE 4:   
Completeness rate overview for Regional R&D Personnel and Researchers by country, 2021 

Regional R & D personnel and researchers breakdowns12 by 

Country 

region and sector of  
performance 

sector of performance 
and gender 

Legend: 
  Very good 
  Good 
  Satisfactorily 
  Unsatisfactorily 

Optional: partially  
reported 
  Missing data 

BE 
BG 
CZ 
DK 
DE 
EE 
IE 
EL 

ES 

FR 
HR 
IT 
CY 
LV 
LT 
LU 
HU 
MT 
NL 
AT 
PL 
PT 
RO 
SI 
SK 
FI 
SE 

CH 
NO 
IS 

AL 
BA 
ME 
MK 
RS 
TR 

     Source: Eurostat’s online database (Eurobase), (R&D) (rd) 

12 Regional business statistics on employment in R&D, Annex I, Part B. Elements of data requirements, Table 19 of the Regulation 
(EC) 2020/1197. 

 Quality report on European statistics on research and development, 2024 Edition 

21 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
  
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
  
 
  
 
  
 
  
 
 
 
 
 
 
 
 
4 

Relevance and completeness 

In the case of the Regional R&D personnel and researchers we note that the great majority of 
countries (28) scored very good and good for the completeness rate. 

4.3. Conclusions on Relevance and Completeness  

The quality indicator relevance has been treated comprehensively both referring to the relevance 
of R&D data at the national level and at the EU level. One clear conclusion is that the wide array 
of users of R&D data (institutions, researchers, business, mass media, etc.) are satisfied with the 
availability, breakdowns and quality of the R&D data and they provide useful feed-back to the 
data producers.  

As regards completeness, the four overview tables on R&D data completeness for the reference 
year 2021 draw a telling picture: Member States’ completeness is very good or good in almost all 
the cases except for DK which had issues on sending the data for the 2021 reference year.  

22                             Quality report on European statistics on research and development, 2024 Edition 

 
 
 
Relevance and completenes   4 

5 

Accuracy  

5.  Accuracy  
Accuracy in the statistical sense denotes the closeness of computations or estimates to the exact 
or true values. Statistics are not equal with the true values because of variability (the statistics 
change from implementation to implementation of the survey due to random effects) and bias (the 
average of the possible values of the statistics from implementation to implementation is not 
equal to the true value due to systematic effects).  

5.1. Assessment of the accuracy with regards to the 
main indicators 

For the three main R&D sectors of performance: BES, GOV, and HES the following table 
presents the reporting countries’ self-assessment of accuracy regarding the main R&D indicators. 
The reference metadata reports for PNP sector lack this analysis because the sector is so small. 
The overview of the accuracy self-assessment presented below for the main R&D sectors is 
based on the information filled by the countries in the national reference metadata reports: 

• 

• 

• 

for the business sector (BES), the great majority of countries (19) reported ‘very good’ and 
‘good’ accuracy for all three main indicators; only one Member State reported ‘satisfactory’ 
and ‘poor’ for all three indicators.  

for the government sector (GOV) 20 countries reported ‘very good’ accuracy for all three 
indicators; there was no ‘poor’ accuracy reported.  

for the higher education sector (HES) 19 countries reported ‘very good’ accuracy for all three 
main indicators; there was no ‘poor’ accuracy reported.  

 Quality report on European statistics on research and development, 2024 Edition 

23 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
5 

Accuracy 

TABLE 5:   
Overview of the accuracy self-assessment13 for the main R&D indicators by country, for BES, 
GOV, and HES, 2021  

Total R&D personnel in 
FTE 
GOV 

HES 

BES 

Total researchers in FTE 

BES 

GOV 

HES 

Legend: 
Very good 
Good 
Satisfactory  
Poor 
Missing 

Total intramural R&D 
expenditure 
GOV 

HES 

BES 

Country 

AT 
BE 
BG 
CY 
CZ 
DE 
DK 
EE 
EL 
ES 
FI 
FR 
HR 
HU 
IE 
IT 
LT 
LV 
LU 
MT 
NL 
PL 
PT 
RO 
SE 
SI 
SK 

CH 
NO 
IS 

BA 
ME 
MK 
RS 
TR 

Source: Eurostat’s Metadata Handler (MH) 

5.2. Compilation methods 

Apart from collecting the data directly through a statistical survey, R&D data is also collected 
through administrative data sources or a combination of the two.  

Administrative data may include both financial data from revenue agencies as well as other types 
of administrative sources such as company records. Frascati Manual 2015 cautions that 

13 Sub-concept 13.1.2. Assessment of the accuracy with regard to the main indicators, at BESSI, GOVSI and HESSI national 
reference metadata reports 

24                             Quality report on European statistics on research and development, 2024 Edition 

 
 
 
 
 
 
 
 
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
 
  
  
  
  
  
  
  
  
  
 
  
  
  
  
  
  
  
  
  
 
  
  
  
  
  
  
  
  
  
 
  
  
  
  
  
  
  
  
  
 
  
  
  
  
  
  
  
  
  
 
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
 
 
 
 
  
 
 
  
 
 
  
  
  
  
  
  
  
  
  
 
 
 
 
 
 
 
 
 
  
  
  
  
  
  
  
  
  
 
 
 
 
  
 
 
  
 
 
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
‘Particular attention should be paid to the use of administrative data in the identification of R&D 
performing and funding institutions’14. Frascati Manual underlines the need that the administrative 
data uses the same concepts, definitions and coverage as defined in the Manual and cautions 
against discrepancies between the concepts used by the administrative data sources and the 
Manual. The uses of administrative data vary across countries based on their availability and 
quality.  

Accuracy     5 

TABLE 6:   
Overview of the data sources for R&D sectors, by country 

Country 

BE 

BG 

CZ 
DK 
DE 
EE 
IE 

EL 

ES 

FR 

HR 
IT 

CY 

LV 

LT 

LU 

HU 

MT 

NL 

AT 
PL 
PT 

RO 

SI 
SK 

FI 

SE 

BES 

census and sample 
survey 

census 

census 
sample survey 
census 
census 
census 
census and sample 
survey 
census 
census and sample 
survey 
census 
census 

census 

threshold survey 
census and sample 
survey 
sample survey 

R&D data sources – general information15  

GOV 

census 

census 

HES 

census 

census 

PNP 

census 

census 

census 
census 
census 
census 
sample survey 

census 
census 
administrative data 
census 
census 

census 
census 
PNP included in GOV 
census 
no PNP data 

census 

census 

census 
census and admin-
istrative sources 
census 
census 

census 
census and adminis-
trative data 
census 
administrative data 

census 

census 

sample survey 

census 

census 

census 

census 

survey 

census 

census 

census 

census 
census 
survey, no sampling is 
used 
no PNP data 

PNP included in BES 

PNP included in 
GOV&HES 
no PNP data 

census 

census 

census 
census and sample 
survey 
census 
census 
census 
census and sample 
survey 
census 
census 

survey 
census and sample 
survey 
census 
census 
census 

census 

census 
census 

census/panel and sam-
ple survey 

census, panel of 
known or supposed 
R&D performers 

administrative data 

census and sample survey 

census 
census 
census 

census 

census 
census 
census, administra-
tive sources and time-
use survey of univer-
sity personnel 

census 
census 
census 

census 
census 

census 

census and sample 
survey 

census 

census 

model based extrapolation 

14 Frascati Manual 2015 edition, pages from 184 to 188. 

15 Concept 18.1.1. Data source – general information, BESSI national reference metadata reports 

 Quality report on European statistics on research and development, 2024 Edition 

25 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
5 

Accuracy 

Country 

R&D data sources – general information15  

BES 

survey 

census and sample 
survey 
census 

GOV 

census 

census 

census 

sample survey 

sample survey 

census 
census 

census 
census 

CH 

NO 

IS 

BA 
ME 
MK 
RS 
TR 

HES 

PNP 

administrative data/ 
precompiled statistics 

census 

census 

census 
census 

no PNP data 

census 
census 

Source: Eurostat’s Metadata Handler (MH) 
Legend: Empty cell = missing 

5.3. Statistical errors 

Several types of statistical errors occur during the survey process. The following typology of 
errors has been adopted: 

1. Sampling errors. These only affect sample surveys. They are due to the fact that only a 
subset of the population, usually randomly selected, is enumerated. 

2. Non-sampling errors. Non-sampling errors affect sample surveys and complete enumerations 
alike and comprise: 

a)  Coverage errors, 

b)  Measurement errors, 

c)  Non-response errors and 

d)  Processing errors. 

5.3.1. Sampling errors   
The sampling error is part of the difference between a population value and an estimate thereof, 
derived from a random sample, which is due to the fact that only a subset of the population is 
enumerated. 

The main indicator used to measure sampling errors is the coefficient of variation (CV). 

CV= (Square root of the estimate of the sampling variance) / (Estimated value). 

In the table below are listed the coefficient of variations for R&D indicators (R&D expenditure and 
R&D personnel (FTE)) and total NACE sectors or Total Size Classes) filled by the counties in the 
BESSI national reference metadata reports, at sub-concepts ‘13.2.1.2. Coefficient of variation for 
key variables by NACE’ and ‘13.2.1.3. Coefficient of variation for key variables by Size Class’. 

26                             Quality report on European statistics on research and development, 2024 Edition 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
TABLE 7:   
Overview of the status of the Coefficient of variations, by country, BES, 2021 

Country 

CVs for R&D expenditure and R&D personnel (FTE) indicators,  
by NACE and by size class16 

Accuracy     5 

BE 

BG 
CZ 
DK 
DE 
EE 
IE 
EL 
ES 
FR 
HR 
IT 
CY 
LV 
LT 
LU 
HU 
MT 
NL 
AT 
PL 
PT 
RO 
SI 
SK 
FI 
SE 

CH 
NO 
IS 

AL 
BA 
ME 
MK 
RS 
TR 

R&D personnel (FTE) => 2.9 

R&D personnel (FTE) => 3.98 

R&D personnel (FTE) => 2.65 

not relevant (as 97.5% of R&D expenditures come from the census share of surveyed set of enter-
prises) 
not applicable, census 
not applicable, census 
R&D expenditure => 1.9 
 not relevant 
not applicable, census 
not applicable, census 
R&D expenditure => 1.49 
not applicable, census 
R&D expenditure => 5.71 
not applicable, census 
not applicable, census 
not applicable, census 
Since the year 2021 the CV is no longer calculated. Threshold sampling and no changes in strata. 
R&D expenditure => 1.43 
R&D expenditure => 0.01 
not applicable, census 
not applicable, census 
not applicable 
not applicable, census 
not applicable, census  
not applicable, census 
R&D expenditure => 0.02;  
not applicable, census 
not applicable, census 
not applicable 
R&D expenditure => 0.01 

R&D personnel (FTE) => 2.75 
R&D personnel (FTE) => 0.01 

R&D personnel (FTE) => 0.01 

R&D personnel (FTE) => 0.01 

R&D expenditure => 2.15 
R&D expenditure => 0.8 
not applicable, census 

R&D personnel (FTE) => 2.37 
R&D personnel (FTE) => 0.9 

not applicable, census 
not applicable, census 

Source: Eurostat’s Metadata Handler (MH)   
Legend: Empty cell = missing 

16 Sub-concepts 13.2.1.2. Coefficient of variation for key variables by NACE, 13.2.1.3. Coefficient of variation for key variables by Size 
Class, BESSI national reference metadata reports 

 Quality report on European statistics on research and development, 2024 Edition 

27 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
5 

Accuracy 

5.3.2. Non-sampling errors  
Non-response occurs when a survey failed to collect data on all survey variables from all the 
population units designated for data collection in a sample or complete enumeration.  

There are two elements of non-response: 

1.  Unit non-response, which occurs when no data (or so little as to be unusable) are collected 

on a designated population unit. 

2.  Item non-response, which occurs when data only on some, but not all survey variables are 

collected on a designated population unit. 

The extent of response (and accordingly of non-response) is measured with response rates. 

5.3.2.1. Unit non-response  
In assessing the unit non-response, the main interest is to judge if the response from the target 
population was satisfactory by computing the weighted and un-weighted response rate. 

Eligible are the sample units which indeed belong to the target population. Frame imperfections 
always leave the possibility that some sampled units may not belong to the target population. 
Moreover, when there is no contact with sample units and no other way to establish their 
eligibility, they are characterised as ‘unknown eligibility units’. 

The formulas used to calculate the two unit non-response rates are:  

  Un-weighted Unit Non- Response Rate = 1 - (Number of units with a response) / (Total 

number of eligible and unknown eligibility units in the survey)  

  Weighted Unit Non- Response Rate = 1 - (Total weighted responding units) / (Total 

weighted number of eligible / unknown eligibility units in the sample) 

In the table below are listed the unit non-response rates by NACE or Size Classes filled by the 
countries in the BESSI national reference metadata reports, at sub-concepts ‘13.2.1.2. Coefficient 
of variation for key variables by NACE’ and ‘13.2.1.3. Coefficient of variation for key variables by 
Size Class’. 

28                             Quality report on European statistics on research and development, 2024 Edition 

 
 
 
 
 
TABLE 8:   
Overview of the unit non-response rate, by country, BES, 2021 

Country 

un-weighted  

Unit Non-response rates  

BE 
BG 
CZ 
DK 
DE 
EE 
IE 
EL 
ES 
FR 
HR 
IT 
CY 
LV 
LT 
LU 
HU 
MT 
NL 
AT 
PL 
PT 
RO 
SI 
SK 
FI 
SE 

CH 
NO 
IS 

AL 
BA 
ME 
MK 
RS 
TR 

40% 
9.1% 
10.4% 
2.70% 
78% 
8.25% 
43.80% 
25.06% 
5.9% 
34.1% 
- 
31.1% 
0% 
4.7% 
0.8% 
2.8% 
23.1% 
28% 
12% 
5.20% 
11% 
11% 
8.4% 

17.5% 
33.40% 
21.3% 

16.30% 
0.9% 
18.1% 

37% 
0% 

Accuracy     5 

weighted 

41% 
not applicable, census 
not applicable, census 

not applicable, census 
not applicable, census 
not applicable, census 
35.65% 
0.91% 
not applicable  
not applicable, census 
not applicable, census 
not applicable, census 
0.05% 
0.99% 

not applicable, census 
not applicable, census 
24% 
not applicable, census 
not applicable, census 
not applicable, census 
0.12% 
not applicable, census 
not applicable, census 

16.30% 

18.1% 

not applicable, census 
0% 

Source: Eurostat’s Metadata Handler (MH)   
Legend: Empty cell = missing 

The imputation has been applied to treat non-response for units that did not provide 
questionnaires by the deadline. In some countries, the imputation is used only for units which 
filled R&D questionnaire in the previous years.  

The response rates for the 2021 data collection are very good with a few exceptions. 

In many reporting countries conducting surveys, the questions in the R&D surveys are perceived 
to be difficult, too specific, or too detailed. In some cases, the R&D data are reportedly not 
available in the enterprise or difficult to estimate. According to the respondents the questionnaire 
is too long and requires too many resources and too much time. Non-R&D performers or small 
enterprises often state the questionnaire is not relevant/does not apply to them.  

Therefore, to reduce the burden, no non-response surveys are conducted in majority of the 
countries. Confidentiality constraints are also sometimes raised. One Member State did conduct 
a unit non-response survey. However, the respondents did not reply even though they received 

 Quality report on European statistics on research and development, 2024 Edition 

29 

 
 
 
 
  
 
 
 
 
  
 
 
 
  
 
 
 
 
 
 
 
 
 
 
 
5 

Accuracy 

several reminders to submit the filled in questionnaire. 

Another reason for not conducting the non-response surveys is due to limited resources in the 
NSIs.  

In one Member State non-response occurs mostly among very small units (62% of all non-
responders have less than 10 persons employed). Some of them might not exist anymore; no 
R&D activity is the most likely reason for non-response. Due to the COVID-19 pandemic 
especially smaller enterprises could not be reached or have stopped economic activities. 

In another Member State conducting a combination of sample survey with census, for BES, 
weighting for the sampling part was used and imputation using previous survey data for known 
R&D performers were used to impute unit non-response for R&D performers. 

In the majority of the cases, even if a non-response survey was not conducted, the countries did 
contact all non-responding enterprises that were expected to have R&D activity.  

5.3.2.2. Item non-response  
The un-weighted Item non-Response Rate is calculated using the following formula:  

un-weighted item non-response rate (%) = 1-(Number of units with a response for the item) / 
(Total number of eligible, for the item, units in the sample) * 100 

TABLE 9:   
Overview of the item non-response rates, by country, BES, 2021  

Country 

BE 
BG 
CZ 
DK 
DE 
EE 
IE 
EL 
ES 
FR 
HR 
IT 
CY 
LV 
LT 
LU 
HU 
MT 
NL 
AT 
PL 
PT 
RO 

R&D Expenditure 
46% 
0% 
 0%  

Un-weighted item non-response rate17 
R&D Personnel (FTE) 
51% 
0% 
 0% 

Researchers (FTE) 
62% 
0% 
 0% 

 23% 
 0% 
0% 
0% 
 3.5% 
8.9% 
0% 
0% 
0% 
0% 
1% 
1% 
0% 
0.01% 
0% 
0% 
1.50% 
0% 
0% 

 34% 
 0% 
0% 
0% 
 3.5% 
33.4% 
0% 
0% 
0% 
0%  
1%  
1% 
0% 
0.01% 
0% 
if none reported => imputed 
1.40% 
0% 
2.7% 

 48% 
 0% 
0% 
0% 
 3.5% 
33.4% 
0% 
0% 
0% 
0% 
1%  
1% 
0% 
0.01% 
0% 
if none reported => imputed 
1.40% 
0% 
24% 

17 Sub-concept 13.3.3.2.1. Un-weighted item non-response rate, BESSI national reference metadata reports 

30                             Quality report on European statistics on research and development, 2024 Edition 

 
 
 
 
 
 
 
 
 
 
Accuracy     5 

Country 

SI 
SK 
FI 
SE 

CH 
NO 
IS 

AL 
BA 
ME 
MK 
RS 
TR 

R&D Expenditure 
0% 
0% 
 1.4% 
0% 

Un-weighted item non-response rate17 
R&D Personnel (FTE) 
0% 
0% 
5,6% 
0% 

Researchers (FTE) 
0% 
0% 
 11,4% 
0% 

 0.43% 
0% 
 0% 

0% 
0% 

5.46%  
0% 
6.48%  

0% 
0% 

6.55%  
0% 
 N/A 

0% 
0% 

Source: Eurostat’s Metadata Handler (MH) 
Legend: Empty cell = missing 

One country used ratio imputation (with last values) in case of item non-response. Other Member 
States also mentioned in the metadata files that they are using imputations for the item non-
response rate based on replies they received from those statistical units in the previous survey.  

5.4. Conclusions on Accuracy 

The accuracy concept in the SIMS reference metadata files is referring both to the self-assessed 
accuracy by the reporting countries as well as to the clear indicators referring to errors that could 
occur during the data collecting and processing phases. The chapter presented both elements. 
The conclusion is that the accuracy of the reporting is good but of course there are margins to 
improve it.  

From consulting the metadata/quality files, it results that the NSIs are taking all the measures 
available to diminish the unit and item non-response rates. Explanatory notes are included in the 
questionnaires throughout. Some countries also provide the statistical units with a glossary 
containing a detailed description of concepts as well as some examples according to the sector. 
The field work for data collection is conducted by experienced and well-trained staff with the R&D 
domain. Manuals and training documents are also made available.  

Intensive follow-up activities are carried out and in case of missing data, respondents are 
contacted to either correct or confirm the data. Different checking and validation routines are in 
place to detect any omissions by the respondents while filling the online questionnaires. Data are 
also compared to the respondents’ data from previous years. Reporting units are provided with 
methodological instructions and classifications needed to fill in the questionnaire.  

A large amount of time is spent re-contacting the reporting units and providing additional 
methodological support to reduce the number of errors: several phone calls and several emails 
are made to the reporting units.  

Some countries provide the opportunity for giving a feed-back on the survey methodology for the 
reporting unit to indicate if the concepts and definitions in the surveys are clear enough.    

To further diminish the unit and item non-response rates increased resources in the National 
Statistical Institutes would be necessary, and this would also increase the burden for the 
statistical units. While the former is a constant desiderate, the latter, i.e., increased burden for the 
reporting units, is to be avoided altogether.  

 Quality report on European statistics on research and development, 2024 Edition 

31 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
5 

Accuracy 

6 

Coherence and comparability 

6.  Coherence and comparability  
Comparability aims at measuring the impact of differences in applied statistical concepts and 
definitions on the comparison of statistics between geographical areas, with other statistical 
domains or over time. It is the extent to which differences between statistics are attributed to 
differences between the true values of the statistical characteristics. 

The factors that may cause two statistical figures to lose comparability are attributes of the 
surveys that produce them. These attributes may be grouped into two major categories:  

a)  concepts of the survey and  

b)  measurement / estimation methodology. 

In order to assess the comparability of data among the reporting countries, the SIMS reference 
metadata/quality report assesses the non-deviation of the definitions, concepts and methodology 
with the Frascati Manual definitions and concepts.   

Table 10 is based on the information provided by the countries in the reference metadata focused 
on two important concepts: the issues non-deviation from Frascati Manual recommendations and 
the methodological issues non-deviations from the recommendations of the Manual.  

32                             Quality report on European statistics on research and development, 2024 Edition 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Coherence and comparability    6 

TABLE 10:   

Overview of comparability of datasets among reporting countries based on the alignment 
with the Frascati Manual definitions and concepts, by country 

Country 

Survey Concept Issues  
compliance with recommenda-
tions18 

Methodological issues  
compliance with recommenda-
tions19 

  Legend: 
  Very good 
  Good 
  Satisfactorily 
  Unsatisfactorily 
  Missing 

BE 
BG 
CZ 
DK 
DE 
EE 
IE 
EL 
ES 
FR 
HR 
IT 
CY 
LV 
LT 
LU 
HU 
MT 
NL 
AT 
PL 
PT 
RO 
SI 
SK 
FI 
SE 

CH 
NO 
IS 

AL 
BA 
ME 
MK 
RS 
TR 

Source: Eurostat’s Metadata Handler (MH) 

Most of the deviations reported by countries refer to R&D personnel and come in the majority of 
the cases from the fact that no distinction is possible for internal and external personnel in the 
country.  

18 Sub-concept 15.1.3. Survey Concepts Issues, BESSI national reference metadata reports 

19 Sub-concept 15.1.4. Deviations from recommendations, BESSI national reference metadata reports 

 Quality report on European statistics on research and development, 2024 Edition 

33 

 
 
 
  
  
  
  
  
  
  
  
 
  
  
 
  
 
  
 
  
  
 
  
  
 
  
  
 
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
 
 
  
  
  
  
  
  
  
 
 
  
  
  
  
  
  
  
  
  
  
  
  
 
 
 
 
 
6   Cohérence and comparability 

For one country the concept of researcher does not resonate with respondents in the enterprise 
sector in the way that the concept is supposed to, and some more conceptual work is needed for 
adequately collecting this data. 

Another common reason is the deviation from the statistical unit enterprise, for some countries it 
is not the statistical unit enterprise but rather the legal unit. Another reason for deviation is the 
enterprise size coverage and breakdown; not all class sizes are surveyed. NACE section 72 is 
the only section where enterprises with less than 10 employees are included in the survey.  

Some issues in terms of coverage that may affect comparability have been identified and are 
currently still under investigation.  

6.1. Comparability over time 

Another important aspect of comparability refers to how comparable are data over time for the 
four sectors of performance, i.e., whether breaks in the time series have occurred. The 
information regarding breaks in time series is sent to Eurostat in both the data and the 
accompanying metadata files. The years when breaks in time series occurred are flagged with 
the ‘b’ flag when data is sent to Eurostat and the flag will appear in Eurostat’s online database 
(Eurobase). In the national metadata/quality files there are dedicated concepts under which the 
breaks in time series are reported and explained for each sector of performance. The following 
three tables below present the years when a break in the time series was notified by the reporting 
countries in the data.   

TABLE 11:   
Overview  of  comparability  over  time  for  the four  sectors  of  R&D  expenditure,  by  country 
(years of breaks in time series)20 

Country 

TOTAL 

BES 

GOV 

HES 

PNP 

1983, 1989, 1993 

1992 

1996, 1999 
1995 
2007 
1991 

1996, 1999 
1995 
2007 
1991 

1983, 1989, 1993,  
1998, 2012 
1996, 1999 
1995 
2002, 2007 
1991, 1992, 2022 

1983, 1989,  
1993, 1998 
1996, 1999 
1995 
2002, 2007 
1991, 2016 

1981, 1989, 1995, 
2008 
1992 
1981, 1997, 2000,  
2004, 2010 

1991, 1997, 2016, 
2021 

2008 

2008 

2002, 2008 
1992, 1997, 2001,  
2004, 2006 

1992, 1997,  
2000, 2010 

1981, 1983, 1989, 
1995, 2008 
1992 
1981, 1997,  
2000, 2004 

1991, 2016 

1991 

1997, 2005 

1983, 1989,  
1993, 2012 
1996, 1999 

2007 
2022 

2008 

1989, 2002 
1992 

BE 

BG 
CZ 
DK 
DE 
EE 
IE 

EL 

ES 

FR 

HR 

IT 

CY 

20 Sub-concept 15.2.2. Breaks in time series, BESSI national reference metadata reports 

34                             Quality report on European statistics on research and development, 2024 Edition 

 
  
 
 
 
 
 
 
 
 
 
 
  
 
 
 
 
 
  
 
 
 
 
 
 
 
 
 
Coherence and comparability    6 

Country 

TOTAL 

BES 

GOV 

HES 

PNP 

LV 
LT 
LU 
HU 
MT 

NL 

AT 
PL 
PT 
RO 
SI 
SK 
FI 

SE 

CH 

NO 
IS 

AL 
BA 
ME 
MK 
RS 
TR 

1996 
2012 
1994, 2004, 2018 
2004 
1982, 1990, 1994,  
1996, 1999, 2011, 
2012, 2013 

1995 
2008 
1992, 1995, 2011 
2008, 2011 
1992, 1994, 1997 
1981, 1991 
1981, 1993, 1995, 
2005 

1983, 1986, 1988, 
1989, 1996, 1998, 
2010 
1984, 1995 
2011, 2013 

2012 
1994, 2018 
2004 
1994, 1996, 2011,  
2012, 2013 

2009 
1994, 2004, 2018 

1994, 2018 

1994, 2000, 2003,  
2012, 2013 

1982, 1990,  
1999, 2013 

1991, 1994,  
2013 

1995 
2013 
1995, 2011 
2008, 2011 
1992, 1994, 1997 

1981, 1995, 2005 

1993 
1995 

1995, 2011 
2011 
1992, 1994, 1997 
1981, 1991, 1994 
1993, 2005, 2019 

1983, 1986, 1988, 
1989,1996 

1998 

2008, 2013 
1995, 2011 
2011 
1992, 1994 
1981, 1991, 1997 
1981, 1995,  
1997, 2019 

2010 

1993, 2009 

1995, 2013 
2011 
2011 

1993, 1995, 2005, 
2007, 2009, 2011 

1984, 1995 
2010, 2011, 2013 

1989, 2007 
2011, 2013 

2007 
2010, 2011, 2013 

2011, 2013 

N/A 

N/A 

N/A 

N/A 

N/A 

2014 

2014 

Source: Eurostat’s online database (Eurobase), (rd_e)  
Legend: Empty cell = not applicable; N/A = no data available 

We notice that the majority of the reporting countries have one or maximum two years of break in 
time series. In the case of eight countries there are three years of ‘b’. For five countries there are 
more than four years flagged ‘b’, with one case of eight ‘b’ flags for Total, having the last break in 
series in 2013.  

 Quality report on European statistics on research and development, 2024 Edition 

35 

 
 
 
 
 
 
 
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
  
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
6   Cohérence and comparability 

TABLE 12:  
Overview of comparability over time for the four sectors of R&D personnel in full time equiv-
alent (FTE), by country (years of breaks in time series)21 

TOTAL 
1989, 1993, 2021 

BES 
1989, 1992, 2021 

GOV 
1989, 1993,  
1998, 2012 

HES 
1989, 1993, 1998 

PNP 
1989, 1993, 
2012 

2005 
2007 
1987, 1991 

2005 
2007 
1991 

2005 
2002, 2007 
1991, 1993, 2022 

1989, 1995, 2011 

1986, 2011 

1992, 2001 
1983, 2011 

1995, 2005 
2002, 2007 
1987, 1991, 2006, 
2016 

1993, 2001 
1983, 1989, 1995, 
2011 

2005 
2007 
2022 

1998 

1989 
1981, 1997, 2000, 
2010 

2002, 2008 
1992, 1997, 2001, 
2006 

1992, 1997, 2000, 
2010 

1981, 1997, 2000 

1989, 2002 
1992, 1997 

1998, 2016 

2016 

1998, 2005 

2012 
2004, 2018 
2004 
1982, 1990, 1994,  
1996, 2003, 2011,  
2012, 2013 

2012 
2018 
2004 
1994, 1996, 2011,  
2012, 2013 

2008 
2011 
2008, 2011 
1997 
1981, 2011 
1981, 1993, 2005,  
2007, 2021 

2013 
2011 
2008, 2011 
1997 

1981, 1995, 2005,  
2007, 2021 

2004, 2018 

2018 

1994, 2000, 2003, 
2012, 2013 

1982, 1990, 1999, 
2013 

1991, 1994, 2013 

1993 

2011 
2011 
1997 
1994, 1995 
1993, 2005, 2011, 
2021 

2008, 2013 
2011 
2011 

1993 

1995, 2013 
2011 
2011 

1981, 1997, 2011 
1981, 1993, 2005, 
2021 

1993, 1995, 1997, 
2007, 2009, 2011 

1991, 1995 
2011, 2013 

1984, 1995 
2011, 2013 

1989, 2007 
2011, 2013 

1991, 2007 
2011, 2013 

N/A 

N/A 

N/A 

N/A 

2011 

N/A 

Country 

BE 

BG 
CZ 
DK 

DE 

EE 
IE 

EL 

ES 

FR 

HR 
IT 
CY 
LV 
LT 
LU 
HU 
MT 

NL 

AT 
PL 
PT 
RO 
SI 
SK 
FI 

SE 

CH 
NO 
IS 

AL 
BA 
ME 
MK 
RS 
TR 

Source: Eurostat’s online database (Eurobase), (rd_p) 
Legend: empty cell = not applicable; N/A = no data available 

Similarly, to the expenditure data, the majority of reporting countries have none or maximum two 
years of break in time series for R&D personnel. For 7 countries there are more than 4 years 
flagged ‘b’, with one case of eight ‘b’ flags for the Total, having the last break as well in 2013.  

21   Sub-concept 15.2.2. Breaks in time series, BESSI national reference metadata reports 

36                             Quality report on European statistics on research and development, 2024 Edition 

 
 
 
 
 
 
 
 
 
 
 
  
  
  
  
  
 
 
 
 
 
  
  
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
  
  
  
  
  
  
  
  
  
  
 
 
 
 
 
  
  
  
  
  
 
 
 
 
 
 
 
 
 
 
  
  
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Coherence and comparability    6 

TABLE 13:   
Overview of comparability over time for the four sectors of R&D personnel in head count 
(HC), by country (years of breaks in time series)22 

Country 

TOTAL 

BES 

GOV 

HES 

PNP 

2021 

2007 
1991 

2011 

2002, 2010 

2021 

2007 
1991 

2012 

2012 

2002, 2007 
1981, 1983, 1985, 
1991, 1992,  

2002, 2007 
1991, 1995, 2016 

2007 
1991 

2011 
2002, 2008 
2001, 2006 

2001 
2011 

2002, 2010 

2001 
2011 

2002 

2005 

2002 

2016 

2016 

2004, 2018 
2004 
2003, 2011, 2012,  
2013 

2018 
2004 
2011, 2012, 2013 

2004, 2018 

2018 

2000, 2003,  
2012, 2013 

1999, 2013 

2013 

2008 
2011 
1993, 2008, 2011 

2013 
2011 
1993, 2008, 2011 

2011 
1993, 2011 

2008, 2013 
2011 
1993, 2011 

1995, 2013 
2011 
2011 

1993, 2005, 2007,  
2009, 2021 

1995, 2005, 2007,  
2021 

1993, 2005, 2011, 
2021 

1993, 2005, 2021 

1993, 1995, 2007, 
2009, 2011 

2011, 2013 

2011, 2013 

2011, 2013 

2011, 2013 

N/A 

N/A 

N/A 

N/A 

2011 

N/A 

BE 
BG 
CZ 
DK 

DE 

EE 
IE 
EL 
ES 
FR 
HR 
IT 
CY 
LV 
LT 
LU 
HU 
MT 

NL 

AT 
PL 
PT 
RO 
SI 
SK 
FI 

SE 

CH 
NO 
IS 

AL 
BA 
ME 
MK 
RS 
TR 

Source: Eurostat’s online database (Eurobase), (rd_p)   
Legend: Empty cell = not applicable; N/A = no data available 

We notice that most of the reporting countries have none or maximum 2 years of break in time series. 
For 2 countries there are more than 4 years flagged ‘b’, with one case of 5 ‘b’ flags for the Total.   

22 Sub-concept 15.2.2. Breaks in time series, BESSI national reference metadata reports 

 Quality report on European statistics on research and development, 2024 Edition 

37 

 
 
 
  
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
  
  
  
  
  
  
  
  
 
 
 
 
 
  
  
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
  
  
  
  
 
 
 
 
 
 
 
 
 
 
  
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
  
  
 
  
  
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
6   Cohérence and comparability 

6.2. Coherence 

The coherence of statistics is their adequacy to be reliably combined in different ways and for 
various uses. The coherence of statistical information reflects the degree to which it can be 
successfully brought together with other statistical information within a broad analytic framework 
and over time. 

When originating from a single source, statistics are coherent in that elementary concepts can be 
combined reliably in more complex ways. When originating from different sources, and in 
particular from statistical surveys of different frequencies, statistics are coherent insofar as they 
are based on common definitions, classifications and methodological standards. The messages 
that statistics convey to users will then clearly relate to each other, or at least will not contradict 
each other. The coherence between statistics is orientated towards the comparison of different 
statistics, which are generally produced in different ways and for different primary uses. 

The legal basis Regulation (EC) 2020/1197 introduces elements of coherence by using the same 
variables (R&D expenditure and R&D personnel) in more business topics (e.g. IFATS - Inward 
Foreign Affiliates Statistics).  

6.2.1. Coherence between R&D and IFATS data 
Both business statistic domains are covered now by the EBS Commission Implementing 
Regulation (EU) No 2020/1197 and there have been no major changes in the legal requirements 
concerning the data collections in the two domains. Table 14 from Annex I of the EBS 
Regulation describes the ‘Country-level business statistics on enterprises by country of ultimate 
control’ (IFATS). Table 18 from the same Annex describes country-level business statistics on 
intramural R&D expenditure, and table 19 describes country level business statistics on 
employment in R&D (R&D domain). 

Among the variables reported under Table 14 as part of the IFATS data are the EBS variables 
Intramural R&D expenditures and R&D personnel. At the same time the variable R&D 
expenditure is also part of Table 18 and the R&D personnel is part of Table 19, both transmitted 
to Eurostat as part of the R&D statistics.  

In terms of timeliness the R&D statistics precede the IFATS data as the deadline for R&D 
transmission is T+18 months in comparison to T+20 months for IFATS. Another difference 
between the 2 statistical domains is the coverage due to the 1% rule23 applicable for IFATS. This 
rule, based on ‘net turnover’ or ‘number of employees and self-employed persons’ at relevant 
NACE A*38 level aggregates for NACE Sections B to F, may be applied for Table 14 of the EBS 
regulation but not for the Tables 18 and 19 referring to R&D statistics. Therefore, some 
discrepancies are expected in the aggregates of the EU total when comparing the IFATS and 
R&D outputs. Table 14 presents the percentage difference comparing the national data provided 
on the variable R&D personnel in the IFATS dataset to the data provided in the national R&D 
dataset for the reference year 2021.  

23 1 % rule as defined in Annex III.A.1 of the EBS General Implementing Act:  the 1 % rule concerns specific variables indicated in 
Annex I to this Regulation. It is not necessary to compile those variables if the contribution of the Member State for a related indicator 
(e.g. value added, turnover or number of employees and self-employed persons), or total services volume (imports plus exports of 
services) in a specific activity (NACE), EBOPS 2010 category or product (CPA) breakdown is less than 1 % of the EU total; cf.pg. 86  

38                             Quality report on European statistics on research and development, 2024 Edition 

 
 
 
 
 
Coherence and comparability    6 

TABLE 14:   
Overview of coherence between IFATS and R&D data: R&D personnel difference 
comparing IFATS and R&D data, in head count 

COUNTRY   
AT 
BE 
BG 
CY 
CZ24 
DE 
DK 
EE 
EL 
ES 
FI 
FR 
HR 
HU 
IE 
IS 
IT 
LT 
LU 
LV 
MT 
NL 
NO 
PL 
PT24 
RO 
SE 
SI 
SK 

R&D personnel (IFATS vs R&D) in head count 

NACE_B 
-0.64% 

NACE_C 
-0.46% 

NACE_D 
-0.62% 

NACE_E 
-3.49% 

NACE_F 
-3.04% 

NACE_BTF 
-0.50% 

0.00% 

0.00% 

0.00% 

0.00% 

0.00% 

0.00% 

Missing IFATS data 

-64.29% 
-27.41% 

-24.82% 
-5.04% 

-86.32% 
-88.88% 

-52.02% 
-41.62% 

-33.83% 
-40.92% 

-26.04% 
-5.60% 

1% RULE 

-100.00% 
0.00% 
0.00% 
273.38% 
0.00% 
-11.11% 
0.00% 

-95.09% 
0.00% 
-2.95% 
10.27% 
-22.06% 
-4.68% 
-7.41% 

Missing R&D data 
1% RULE 

-100.00% 
0.00% 
4.74% 
-52.90% 
-6.25% 
76.89% 
-69.05% 

-100.00% 
0.00% 
-17.16% 
-31.81% 
-69.57% 
-5.50% 
0.00% 

Missing IFATS data 

-98.06% 
-7.31% 
-0.08% 
-58.55% 
-2.67% 
-2.80% 
29.51% 

-95.93% 
-0.29% 
-2.72% 
5.64% 
-21.60% 
-3.55% 
-7.43% 

0.00% 

0.01% 

-0.14% 

1.20% 

0.13% 

0.02% 

1% RULE 
1% RULE 
1% RULE 
1% RULE 

-1.80% 
1.40% 
-1.84% 
-82.41% 
-100.00% 
-99.81% 

-10.26% 
-4.36% 
-3.47% 
-67.53% 
-17.70% 
-51.18% 

-6.32% 
0.00% 
-1.59% 
-98.01% 
-100.00% 
-96.53% 

-15.10% 
-47.00% 
-9.49% 
-87.94% 
-100.00% 
-91.57% 

-3.10% 
-9.74% 
-6.47% 
-40.79% 
-100.00% 
-91.27% 

1% RULE 

-9.89% 
-4.72% 
-3.58% 
-67.46% 
-20.49% 
-52.75% 

-33.33% 

-32.64% 

-4.84% 

-82.61% 

-77.46% 

-33.05% 

Source: Eurostat, Unit G6 - Foreign Affiliates Statistics and Unit G4 - R&D 
Note: R&D data subtracted from IFATS data 

The colour coding in the Table 14 and Table 15 is based on the filter where the cells are 
highlighted where the absolute difference between the two domains is larger than 10%. This is an 
indicative threshold that is used to assess annually the compliance of Member States with the 
data transmission requirements. 

We can notice that for 5 Member States the differences are ‘0’ or minimal for the R&D personnel 
between the two domains. Differences are important for the rest of the Member States where 
data are available.  

24  For Tables 14 and 15, the country has notified Eurostat of corrective actions that will be reflected in the next data 
transmission 

 Quality report on European statistics on research and development, 2024 Edition 

39 

 
 
 
 
 
 
 
 
 
 
 
 
6   Cohérence and comparability 

TABLE 15:  
Overview of coherence between IFATS and R&D data: Intramural R&D expenditures differ-
ence comparing IFATS and R&D data  

COUNTRY   

NACE_B 

NACE_C 

NACE_D 

NACE_E 

NACE_F 

NACE_BTF 

Intramural R&D expenditures (IFATS vs R&D) 

AT 

BE 

BG 

CY 

CZ 

DE 

DK 

EE 

EL 

ES 

FI 

FR 

HR 

HU 

IE 

IS 

IT 

LT 

LU 

LV 

MT 

NL 

NO 

PL 

PT 

RO 

SE 

SI 

SK 

0.00% 

-0.01% 

0.00% 

0.00% 

-1.23% 

-0.02% 

0.00% 

0.00% 

0.00% 

0.00% 

0.00% 

0.00% 

Missing IFATS data 

1% RULE 

0.00% 

-57.10% 

-0.54% 

17.38% 

0.00% 

-60.08% 

0.00% 

-44.07% 

-0.55% 

-42.76% 

-0.53% 

17.01% 

Missing R&D data 

1% RULE 

-100.00% 

-81.13% 

-100.00% 

0.00% 

0.00% 

545.36% 

0.00% 

0.00% 

0.00% 

0.00% 

-2.63% 

8.83% 

-37.69% 

-1.04% 

-11.30% 

0.00% 

3.32% 

-27.72% 

-8.48% 

66.33% 

-19.14% 

-99.78% 

0.00% 

-24.05% 

-28.57% 

-41.13% 

-2.52% 

0.20% 

Missing IFATS data 

-94.79% 

-12.70% 

-0.36% 

-78.80% 

0.08% 

1.60% 

111.42% 

-82.73% 

-0.28% 

-2.44% 

6.19% 

-36.71% 

-0.38% 

-11.24% 

0.00% 

0.00% 

0.00% 

0.54% 

0.07% 

0.01% 

1% RULE 

1% RULE 

1% RULE 

1% RULE 

-0.87% 

0.49% 

-0.95% 

-100.00% 

-100.00% 

-99.99% 

-8.53% 

-5.05% 

-2.56% 

-98.01% 

-3.52% 

-38.80% 

-1.84% 

-0.33% 

-2.11% 

-97.90% 

-14.72% 

c 

-15.62% 

-98.73% 

-4.27% 

-6.29% 

-5.09% 

-99.80% 

-100.00% 

-100.00% 

-100.00% 

-95.14% 

-78.76% 

-99.66% 

-8.42% 

c 

-2.70% 

-98.12% 

-9.76% 

-40.06% 

0.45% 

-0.19% 

0.18% 

0.00% 

0.00% 

-0.18% 

1% RULE 

Source: Eurostat Unit G6 - Foreign Affiliates Statistics and Unit G4 - R&D 
Note: R&D data subtracted from IFATS data 

We can notice that for 9 Member States the differences are ‘0’ or minimal for the R&D 
expenditure between the two domains. Differences are important for the rest of the EU Member 
States where data are available.  

Eurostat is further investigating the coherence between these two business topics. The BESSI 
metadata templates have been updated to include a new concept on the coherence between 
these two domains, thus increasing the awareness of experts in the NSIs that they need to 
collaborate with their colleagues on the IFATS domain to corroborate the R&D and IFATS data.    

40                             Quality report on European statistics on research and development, 2024 Edition 

 
Coherence and comparability    6 

6.2.2. Coherence between the preliminary versus the final 
R&D data  
When referring to the concept of coherence for the R&D domain another important viewpoint to 
assess the coherence of R&D data is provided by the comparison between the final R&D data 
versus the preliminary data (TABLES 16, 16a and 17, 17a).  

TABLE 16:   
Overview of the differences between the final and preliminary R&D expenditure for 2021, 
by country 

Country / 
Sector of per-
formance 

R&D expenditure (final data minus preliminary data) 

Millions of national currencies 

BE 
BG 
CZ 
DK 
DE 
EE 
IE 
EL 
ES 
FR 
HR 
IT 
CY 
LV 
LT 
LU 
HU 
MT 
NL 
AT 
PL 
PT 
RO 
SI 
SK 
FI 
SE 

CH 
NO 
IS 

AL 
BA 
ME 
MK 
RS 
TR 

BES 
700 
0 
0 
0 
572 
0 
291 
0 
0 
196 
0 
-634 
-13 
17 
1 
39 
0 
4 
16 
101 
0 
42 
0 
-4 
0 
0 
3688 

GOV 
83 
0 
0 
0 
-287 
0 
0 
11 
0 
-261 
-2 
58 
0 
0 
0 
-26 
0 
1 
-4 
20 
0 
4 
0 
0 
0 
0 
0 

HES 
321 
0 
0 
0 
48 
0 
29 
-1 
0 
159 
0 
41 
7 
0 
0 
2 
0 
0 
423 
159 
0 
-2 
0 
0 
0 
0 
0 

Final data only 
-2 
852 

Final data only 
0 
0 

Final data only 
0 
0 

PNP 
-21 
0 
0 
0 
N/A 
0 
N/A 
0 
0 
89 
N/A 
9 
-2 
N/A 
N/A 
N/A 
N/A 
N/A 
0 
-6 
0 
0 
0 
-1 
0 
0 
0 

N/A 
N/A 
N/A 

TOTAL 
1084 
0 
0 
0 
334 
0 
319 
11 
0 
182 
0 
-526 
-9 
17 
1 
14 
0 
5 
436 
274 
0 
44 
0 
-5 
0 
0 
3688 

Final data only 
-2 
852 

Final data only 

Final data only 

Final data only 

Final data only 

Final data only 

0 
Final data only 

0 
Final data only 

0 
Final data only 

0 
N/A 

0 
Final data only 

Source: Eurostat free online database (Eurobase) 
Legend: Empty cell = not available; N/A = no data available 

We notice that for 9 Member States there are no differences, in absolute figures, between the 
final and the preliminary data for R&D expenditure which is a good indicator of the coherence of 
their R&D expenditure data. One Member State greatly overestimated the preliminary data. In the 
case of 8 Member States the final data, in absolute figures, is significantly higher while for three 

 Quality report on European statistics on research and development, 2024 Edition 

41 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
6   Cohérence and comparability 

countries there are quite significant differences between the final and the preliminary data.  

TABLE 16a:   
Overview of the relative difference between the final and preliminary R&D expenditure for 
2021, by country 

Country / Sector 
of performance 

R&D expenditure 

Percentage points 

BE 
BG 
CZ 
DK 
DE 
EE 
IE 
EL 
ES 
FR 
HR 
IT 
CY 
LV 
LT 
LU 
HU 
MT 
NL 
AT 
PL 
PT 
RO 
SI 
SK 
FI 
SE 

CH 
NO 
IS 

AL 
BA 
ME 
MK 
RS 
TR 

BES 
5.753 
0 
0 
0 
0.761 
0 
8.105 
0 
0 
0.539 
-0.013 
-3.893 
-13.706 
21.767 
0.449 
11.247 
0 
7.144 
0.124 
1.120 
0 
2.002 
0 
-0.516 
-0.001 
0 
2.804 

GOV 
5.967 
0 
0 
0 
-1.683 
-0.001 
0 
1.954 
0 
-3.860 
-1.163 
1.617 
3.957 
0.006 
0 
-12.549 
0 
55.673 
-0.332 
1.997 
0.038 
2.715 
0 
0.146 
0.002 
0 
-0.001 

HES 
12.969 
0 
0 
0 
0.235 
0 
3.843 
-0.081 
0 
1.416 
0.026 
0.665 
9.451 
-0.029 
0 
0.952 
0 
0.336 
8.135 
5.505 
0 
-0.195 
0 
0.179 
0 
0 
0 

Final data only 
-0.004 
1.323 

Final data only 
0 
0 

Final data only 
0 
0 

PNP 
-19.239 
0 
0 
0 
N/A 
0.007 
N/A 
0.247 
0 
8.376 
N/A 
1.940 
-7.624 
N/A 
N/A 
N/A 
N/A 
N/A 
N/A 
-8.377 
-0.595 
-0.155 
-0.001 
-7.447 
-1.140 
0 
0 

N/A 
N/A 
N/A 

TOTAL 
6.709 
0 
0 
0 
0.296 
0 
7.096 
0.406 
0 
0.329 
0.008 
-1.982 
-4.088 
7.174 
0.220 
1.891 
0 
5.187 
2.255 
2.113 
0 
1.242 
0 
-0.404 
0 
0 
2.016 

Final data only 
-0.002 
0.944 

Final data only 

Final data only 

Final data only 

Final data only 

Final data only 

0 
Final data only 

0 
Final data only 

0 
Final data only 

-0.134 
N/A 

0 
Final data only 

Source: Eurostat free online database (Eurobase) 
Legend: Empty cell = not available; N/A = no data available; higher than (+5); lower than (-5) 

Looking at TABLE 16a, we notice that 10 countries show differences of more than 5% points 
between their preliminary and final data, while for the rest of the countries there are differences, 
or they are not significant. Looking by sector, we can see that BES and PNP are the ones for 
which more countries present a difference of more than 5% between the preliminary and final 
data. On the other hand, for GOV, only three countries present significant differences.  

42                             Quality report on European statistics on research and development, 2024 Edition 

 
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
 
 
TABLE 17:   
Overview of difference between the final and preliminary R&D personnel and researchers, 
in full time equivalent (FTE), by country, 2021 

Coherence and comparability    6 

Country / 
Sector of 
perfor-
mance 

BE 
BG 
CZ 
DK 
DE 
EE 
IE 
EL 
ES 
FR 
HR 
IT 
CY 
LV 
LT 
LU 
HU 
MT 
NL 
AT 
PL 
PT 
RO 
SI 
SK 
FI 
SE 

CH 

NO 
IS 

AL 

BA 

ME 
MK 
RS 

TR 

R&D personnel  
(final data minus preliminary data) 

R&D researchers  
(final data minus preliminary data) 

Gender: Total 
Full time equivalent (FTE) 
TOTAL  BES 

GOV 

BES 
-1837 
0 
0 
0 
778 
0 
2560 
-25 
0 
-685 
0 
-23862 
-47 
12 
27 
-54 
0 
21 
-558 
244 
0 
194 
0 
-51 
0 
0 
0 

GOV 

93 
0 
0 
0 
2368 
0 
0 
440 
0 
-2373 
-60 
741 
-27 
0 
0 
-303 
0 
44 
-84 
-364 
0 
20 
0 
0 
0 
0 
0 

HES 
3365 
0 
0 
0 
943 
0 
1164 
751 
0 
-2510 
0 
-1466 
5 
0 
0 
16 
0 
-2 
1873 
540 
0 
-49 
0 
0 
-1 
0 
0 

Final 
data 
only 

Final 
data 
only 

Final 
data 
only 

-92 
345 

0 
0 

359 
-6 

PNP 
-1921 
0 
0 
0 

N/A 

N/A 

0 

0 
0 
765 

N/A 

-5 
-7 

N/A 
N/A 
N/A 
N/A 
N/A 

0 
-10 
0 
-23 
0 
-4 
0 
0 
0 

N/A 
N/A 
N/A 

-301 
0 
0 
0 
4089 
0 
3723 
1165 
1 
-4803 
0 
-24593 
-76 
12 
26 
-339 
0 
62 
1231 
410 
0 
141 
0 
-55 
0 
0 
0 

-1983 
0 
0 
0 
733 
0 
1861 
-4 
0 
-4051 
0 
-11910 
-40 
11 
20 
21 
0 
42 
-1246 
1166 
0 
171 
0 
-1 
0 
0 
0 

80 
0 
0 
0 
1001 
0 
0 
287 
0 
-1618 
-35 
-101 
-9 
0 
0 
-105 
0 
42 
-151 
-266 
0 
15 
0 
0 
0 
0 
-1 

HES 
2344 
0 
0 
0 
401 
0 
1229 
745 
0 
-848 
0 
-1716 
3 
0 
0 
13 
0 
-2 
1432 
582 
0 
-18 
0 
0 
0 
0 
0 

PNP 

-521 
0 
0 
0 

N/A 

N/A 

0 

0 
0 
312 

N/A 

-31 
-18 

N/A 
N/A 
N/A 
N/A 
N/A 

0 
0 
0 
-5 
0 
-2 
0 
0 

Final 
data 
only 

Final 
data 
only 

Final 
data 
only 

Final 
data 
only 

267 
339 

3 
186 

0 
0 

353 
-6 

N/A 
N/A 
N/A 

TOTAL 
-79 
0 
0 
0 
2135 
0 
3089 
1028 
0 
-6204 
0 
-13759 
-63 
11 
19 
-70 
0 
82 
35 
1481 
0 
163 
0 
-3 
0 
0 
-1 

Final 
data 
only 

356 
180 

Final 
data 
only 

Final 
data 
only 

Final 
data 
only 

Final 
data 
only 

Final 
data 
only 

Final 
data 
only 

Final 
data 
only 

Final 
data 
only 

Final 
data 
only 

Final 
data 
only 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

Final 
data 
only 

Final 
data 
only 

Final 
data 
only 

Final 
data 
only 

Final 
data 
only 

Final 
data 
only 

Final 
data 
only 

N/A 

Final 
data 
only 

N/A 

Source: Eurostat free online database (Eurobase) 
Legend: Empty cell = not available; N/A = no data available 

We notice that for ten Member States, in absolute figures, there are no differences between final 
and preliminary R&D personnel data which is a good indicator for the coherence of their R&D 
personnel data. However, in the case of 13 countries there are significant or very high differences 
between these datasets for R&D personnel, four of them overestimating the preliminary data.  

In the case of the R&D researchers’ data we notice that for 12 Member States there are no 
differences between the final and the preliminary data which is a good indicator of coherence. We 

 Quality report on European statistics on research and development, 2024 Edition 

43 

 
 
 
  
 
  
  
  
  
  
  
  
  
  
  
 
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
 
6   Cohérence and comparability 

notice again the case of overestimation of the preliminary data and 4 cases of significant 
differences between the final and preliminary data. 

TABLE 17a:   
Overview  of the  relative  difference  between  the  final  and  preliminary  R&D  personnel  and 
researchers, in full time equivalent (FTE), by country, 2021 

Country / 
Sector of 
performance 

BE 
BG 
CZ 
DK 
DE 
EE 
IE 
EL 
ES 
FR 
HR 
IT 
CY 
LV 
LT 
LU 
HU 
MT 
NL 
AT 
PL 
PT 
RO 
SI 
SK 
FI 
SE 

CH 

NO 
IS 

AL 

BA 

ME 
MK 
RS 

TR 

R&D personnel 

R&D researchers 

Gender: Total 
Full time equivalent (FTE) 

BES 
-2.211 
0 
0 
0 
0.163 
0 
12.558 
-0.141 
0 
-0.226 
0 
-10.74 
-5.562 
0.654 
0.478 
-1.829 
0 
1.775 
-0.449 
0.405 
0 
0.563 
0 
-0.448 
0 
0 
0 

Final 
data 
only 
-0.339 
12.950 

Final 
data 
only 

0 
Final 
data 
only 

GOV 
1.069 
0 
0 
0 
2.026 
0 
0 
2.970 
0 
-4.469 
-1.699 
1.800 
-12.000 
0 
0 
-20.160 
0 
122.222 
-0.870 
-6.219 
0 
0.737 
0 
0 
0 
0 
0 

Final 
data 
only 
0 
0 

Final 
data 
only 

0 
Final 
data 
only 

HES 
13.469 
0 
0 
0 
0.606 
0 
8.904 
2.719 
0 
-1.847 
0 
-1.676 
0.571 
0 
0 
1.017 
0 
-0.301 
4.852 
2.668 
0 
-0.157 
0 
0 
-0.010 
0 
0 

Final 
data 
only 
2.036 
-0.484 

Final 
data 
only 

0 
Final 
data 
only 

PNP 
-71.360 
0 
0 
0 
N/A 
0 
N/A 
0 
0 
8.484 
N/A 
-0.071 
-1.842 
N/A 
N/A 
N/A 
N/A 
N/A 
0 
-1.495 
0 
-1.769 
0 
-2.484 
0 
0 
0 

N/A 

Final 
data 
only 

0 

N/A 

TOTAL 
-0.252 
0 
0 
0 
0.545 
0 
10.723 
1.924 
0 
-0.959 
0 
-6.875 
-3.269 
0.170 
0.175 
-5.624 
0 
3.293 
0.714 
0.471 
0 
0.203 
0 
-0.315 
0 
0 
0 

Final 
data 
only 
0.517 
8.319 

Final 
data 
only 

0 
Final 
data 
only 

BES 
-4.043 
0 
0 
0 
0.265 
0 
17.771 
-0.030 
0 
-1.929 
0 
-14.134 
-6.957 
0.952 
0.588 
2.075 
0 
8.589 
-1.675 
3.346 
0 
0.695 
0 
-0.015 
0 
0 
0 

Final 
data 
only 
0.015 
15.857 

Final 
data 
only 

0 
Final 
data 
only 

GOV 
1.265 
0 
0 
0 
1.596 
0 
0 
3.362 
0 
-5.055 
-1.639 
-0.410 
-9.000 
0 
0 
-12.883 
0 
182.609 
-2.376 
-6.515 
0 
0.840 
0 
0 
0 
0 
-0.024 

Final 
data 
only 
0 
0 

Final 
data 
only 

0 
Final 
data 
only 

HES 
11.820 
0 
0 
0 
0.333 
0 
10.396 
3.335 
0 
-0.917 
0 
-2.912 
0.392 
0 
0 
0.942 
0 
-0.376 
5.660 
3.719 
0 
-0.062 
0 
0 
0 
0 
0 

Final 
data 
only 
2.470 
-0.557 

Final 
data 
only 

0 
Final 
data 
only 

PNP 
-46.853 
0 
0 
0 
N/A 
0 
N/A 
0 
0 
5.599 
N/A 
-0.629 
-9.730 
N/A 
N/A 
N/A 
N/A 
N/A 
0 
0 
0 
-0.535 
0 
-1.724 
0 
0 

N/A 

Final 
data 
only 

0 

N/A 

TOTAL 
-0.104 
0 
0 
0 
0.465 
0 
13.435 
2.320 
0 
-1.825 
0 
-7.966 
-3.877 
0.243 
0.172 
-2.183 
0 
7.854 
0.033 
2.690 
0 
0.290 
0 
-0.027 
0 
0 
-0.001 

Final 
data 
only 
0.922 
7.560 

Final 
data 
only 

0 
Final 
data 
only 

Source: Eurostat free online database (Eurobase) (% points) 
Legend: Empty cell = not available; N/A = no data available; higher than (+5); lower than (-5) 

Looking at the TABLE 17a we note that for ten countries there are significant differences (more 
than 5%) on the personnel data, being especially significant in the case of MT. We can see as 
well that more significant differences exist for researchers than for the total R&D personnel 
between the preliminary and the final data.   

44                             Quality report on European statistics on research and development, 2024 Edition 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Coherence and comparability    6 

6.3. Conclusions on Coherence and Comparability 

These two quality indicators are very important because data should be comparable to be useful. 
This chapter looks at comparability over time and non-deviation from the Frascati Manual 
definitions and methodology. The comparability of the R&D data is good as it comes out from the 
four tables in the chapter.  

In order to check and validate the datasets, these have to be coherent among the business 
domains where the EBS General Implementing Act 2020/1197 requests common R&D indicators. 
The chapter looked at the coherence between the R&D and IFATS domains arriving at the 
conclusion that more work has to be done by Eurostat and NSIs to ensure the coherence of the 
common indicators.  

Regarding the coherence between the final and the preliminary data the picture is a mixed one as 
it results from the four dedicated tables.  

 Quality report on European statistics on research and development, 2024 Edition 

45 

 
 
 
7 

     Timeless and punctuality  

7 

Timeless and punctuality 

7. Timeless and punctuality 
Timeliness and punctuality refer to time and dates, but in a different manner: the timeliness of 
statistics reflects the length of time between their availability and the event or phenomenon they 
describe. Punctuality refers to the time lag between the release date of the data and the target 
date on which they should have been delivered, with reference to dates announced in the official 
release calendar.  

7.1. Timeliness 

Table 18 uses the information provided by the countries in the SIMS reference metadata/quality 
files for the Business enterprise sector (BESSI metadata files). The reason other sectors are not 
presented separately is because usually data for all sectors come together. The SIMS template 
provides for a dedicated concept to report on the timeliness of preliminary and final R&D data.  

46                             Quality report on European statistics on research and development, 2024 Edition 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
   Timeless and punctuality      7 

TABLE 18:   
Overview of Timeliness of R&D BES data at national level for preliminary and final data, by 
country, 2021  

Timeliness of R&D 2021 data at national level  

Country 

BE 
BG 
CZ 
DK 
DE 
EE 
IE 
EL 
ES 
FR 
HR 
IT 
CY 
LV 
LT 
LU 
HU 
MT 
NL 
AT 

PL 
PT 
RO 
SI 
SK 
FI 
SE 

CH 
NO 
IS 

AL 
BA 
ME 
MK 
RS 
TR 

Time lag – First/ preliminary results25 
no release of preliminary national data 
 303 days 
 298 days 
 330 days 
 280 days 
 360 days 
 450 days 
 300 days 
 305 days 
 270 days 
 300 days 
 270 days 
no release of preliminary national data 
 240 days 
 280 days 
 330 days 
 285 days  
 300 days 
 300 days 
no release nationally (only sent to Eurostat)  
 303 days 
 297 days 
 217 days 
 320 days 
  300 days 
 300 days 
 300 days 
 195 days 

no preliminary results 
 299 days 
 332 days 

 330 days 

Time lag - Final results26 

   546 days   
 423 days 
 298 days 
 690 days 
 530 days 
 360 days 
  450 days 
 540 days 
 547 days 
 630 days 
 300 days 
  540 days  
 570 days 
 540 days 
 523 days 
 570 days 
 544 days 
 300 days 
 570 days 
570 days 

 297 days 
 356 days 
 329 days 
 420 days 
 540 days 
 300 days 
 300 days 

 360 days 
 412 days 
 427 days 

 no preliminary results 

 292 days 

Source: Eurostat Metadata Handler (MH)  
Legend: empty cell = missing 

25 Sub-chapter 14.1.1. Time lag - First result, BESSI national reference metadata reports 

26 Sub-chapter 14.1.2. Time lag - Final result, BESSI national reference metadata reports 

 Quality report on European statistics on research and development, 2024 Edition 

47 

 
 
 
  
 
 
 
  
  
  
  
  
  
  
  
  
 
 
 
 
 
 
7 

     Timeless and punctuality  

7.2. Punctuality 

Table 19 presents the punctuality of the data sent by the countries as received by Eurostat via 
the single-entry point EDAMIS. Based on the punctuality of data sent, Eurostat is calculating the 
compliance of the reporting countries with the EBS General Implementing Act 2020/1197, i.e., if 
all the datasets were sent on the deadline foreseen by the Regulation or with delays.  

TABLE 19:   
Overview of Punctuality of R&D final 2021 data, by country  

Punctuality 

  Legend 
  Very good 
  Good 
  Satisfactorily 
  Unsatisfactorily 

Country 
BE 
BG 
CZ 
DK 
DE 
EE 
IE 
EL 
ES 
FR 
HR 
IT 
CY 
LV 
LT 
LU 
HU 
MT 
NL 
AT 
PL 
PT 
RO 
SI 
SK 
FI 
SE 

LI 
CH 
NO 
IS 

AL 
BA 
ME 
MK 
RS 
TR 

Source: Eurostat single entry point (EDAMIS) 

We notice that for the reference year 2021 there have been no major issues regarding the 
punctuality of the data.  

48                             Quality report on European statistics on research and development, 2024 Edition 

 
 
 
  
  
  
 
 
  
  
  
  
  
  
 
  
 
  
 
  
 
  
 
  
 
  
 
  
 
  
 
  
 
  
 
  
 
  
 
  
 
  
 
  
 
  
 
  
 
  
 
 
  
 
  
 
  
 
  
 
 
 
 
  
 
 
 
 
 
  
 
  
 
  
 
  
 
 
   Timeless and punctuality      7 

7.3. Conclusions on Timeliness and Punctuality  

The two tables dedicated to these two quality indicators draw a complete picture: there are no 
major issues where punctuality and timeliness are concerned, except for a few Member States. In 
terms of punctuality, we do not notice issues except for DK which is assessed as satisfactorily.  

We notice that the average time lag for the national dissemination of the preliminary data is 297 
days ranging from 195 lag days to 485 lag days. In the case of final data, the average publication 
lag is 462 days ranging from 292 lag days to 690 lag days depending on the country.     

 Quality report on European statistics on research and development, 2024 Edition 

49 

 
 
 
    Timeless and punctuality    7 

8 

Accessibility and Clarity 

8. Accessibility and Clarity 

8.1. Accessibility  

The SIMS concept Accessibility refers, as it can be inferred from its name, to how accessible the 
national datasets are in terms of their availability via different dissemination products. Another 
important criterion for easy access to data is the cost policy of the NSIs for their R&D data.    

Table 20 summarises the accessibility of the data at the national level in the reporting countries 
based on the information the NSIs have provided in the BESSI metadata. The data are easily 
accessible when there are regular R&D data releases and the data are accessible as well via 
other means of data dissemination (NSIs internet sites, data extractions, ad hoc extractions). 

Other criteria for data accessibility are the access to microdata for the research community and 
research projects and a moderate access cost policy. Aggregate data is free in all reporting 
countries. 

50                             Quality report on European statistics on research and development, 2024 Edition 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
   Accessibility and Clarity      8 

TABLE 20:   
Accessibility of data at the national level, by country  

Country 

Regular releases27 

Access to microdata 
by researchers28 

Access cost policy for micro-
data29 

BE 
BG 
CZ 
DK 
DE 
EE 
IE 
EL 
ES 
FR 
HR 
IT 
CY 
LV 
LT 
LU 
HU 
MT 
NL 
AT 
PL 
PT 
RO 
SI 
SK 
FI 
SE 

CH 

NO 
IS 

AL 
BA 
ME 
MK 
RS 
TR 

Y 
Y 
Y 
Y 
Y 
N 
Y 
Y 
Y 
N 
Y 
Y 
Y 
N 
Y 
N 
Y 
Y 
N 
N 
Y 
N 
Y 
Y 
N 
Y 
Y 

Y 

Y 
Y 

Y 

Y 
Y 

Y  
Y  
Y  
Y  
Y  
Y  
n/a 
N 
Y  
Y  
Y  
Y  
Y  
Y  
Y  
Y  
Y  
N 
Y  
Y  
N 
Y  
Y  
Y  
Y  
N 
Y  

for OECD and governmental 
projects only 
Y 
Y 

N 

Y 
n/a 

no charge 
access costs 
n/a 
no charge 
access costs 
no charge 
no charge 
no charge 
access costs 
access costs 
access costs 
no charge 
access costs 
no charge 
no charge 
no charge 
access costs 
no charge 
n/a 
access costs 
n/a 
no charge 
no charge 
no charge 
access costs 
access costs 
access costs 

no charge 

access costs 
access costs 

not available 

no charge 
n/a 

Source: Eurostat Metadata Handler (MH) 
Legend: Y= yes/ N= no; n/a= not applicable; empty cell = missing 

27 Sub-concept 10.1.1. Availability of the releases, BESSI national reference metadata reports 

28 & 29 Sub-concept 10.4.1. Provisions affecting the access, BESSI national reference metadata reports 

 Quality report on European statistics on research and development, 2024 Edition 

51 

 
 
 
 
  
  
 
 
 
 
 
 
 
  
  
  
  
  
  
  
  
  
 
 
 
 
 
 
8 

Accessibility and Clarity 

8.2. Clarity 

The concept Clarity assesses the type(s) of data accompanying information available for users to 
enhance clarity (metadata, graphs, quality reports) at national level, whether the R&D data 
production methodology is well explained.  

For all countries, R&D statistical data are always accompanied by a metadata/quality report. The 
countries provide metadata accompanying the data released and each R&D statistical release 
contains a short note on methodology or an entire methodology section/chapter with definitions of 
the R&D variables and the methodological guidelines regarding the R&D survey. This information 
is nowadays available on the NSIs’ websites and links to the relevant pages are included in the 
releases. Users can download free of charge textual description of latest developments in R&D 
activities, graphical displays, comprehensive methodological notes/glossaries of terminology 
giving information on the respective national R&D survey and its scope, concepts, and definitions, 
and for some countries also a copy of the questionnaire used. At least one country has a 
dedicated webpage with links to reference documents (Commission Regulation, Frascati Manual, 
etc). Links to related publications and data tables and the link for the online R&D questionnaire 
are made available. The same NSI has put in place an online helpdesk available for authorized 
users (i.e., respondents) while the FAQ page is open to the public. Another NSI put in place a 
Statistical Information Centre to offer additional explanations and assistance to the users.  

Some countries prepare national quality reports/quality handbook/executive summary of the 
quality report; some are available in English for all users’ convenience. The national quality 
reports describe the methodology of surveys in detail. The reports published on the results also 
contain methodological information. 

The SIMS metadata templates provide for dedicated concepts to assess the opinion of users on 
the clarity of the accompanying information to the data and if there are requests on further 
clarification of most problematic issues.  The reporting countries note that further clarifications are 
sometimes requested by users who are not familiar with the Frascati Manual, or with the 
confidentiality policy applied. Questions on differences between full time equivalent and head 
count (HC) measurements, differences between researchers and technicians, questions on 
enterprises if their activities qualify or not as R&D activities, are some of the most frequent 
questions asked by users. Many users of the R&D statistics are interested in historical 
comparisons and time series, therefore the comparability of the results to historical numbers is an 
important concern.  

Most of the countries note in their metadata files that most users are satisfied with the clarity of 
the metadata and methodological information offered.  

The NSIs are continuously working to improve the clarity of the data and accompanying 
metadata/methodological notes.   

8.3. Conclusions on Accessibility and Clarity  

The last quality indicators treated in this summary report refer to how accessible the R&D data is 
and if it is clear for the user what the data are presenting. The conclusion on accessibility is that 
R&D data is very accessible both at the national and regional level. The data are disseminated 
via several dissemination products. With the major uptake of online dissemination products, the 
data and metadata are literally at the tip of the fingers of all users. The other important criterion 

52                             Quality report on European statistics on research and development, 2024 Edition 

   Accessibility and Clarity      8 

on accessibility refers to access costs. About half of the NSIs do not charge for access to 
microdata for the research community.  

The last conclusion refers to the clarity of the data, how well data is accompanied by metadata, 
graphs that explain what data present. All the NSIs publish together with the data also the 
accompanying metadata and methodology chapters/sections. Users are aware of what the data 
presents. 

 Quality report on European statistics on research and development, 2024 Edition 

53 

 
 
 
 
8 

Accessibility and Clarity 

9 

Conclusions on the Summary 
Quality Report 

9. Conclusions on the Summary Quality Report 
The purpose of this summary quality report is to provide the users and producers of the R&D data 
with a tool for assessing the quality of these statistics for the reference year 2021. It provides a 
brief description of the main quality indicators which are: relevance, accuracy, accessibility and 
clarity, timeliness and punctuality, comparability, and coherence. Each indicator has been 
presented and supported with information provided by the countries in their SIMS metadata files.    

Regarding the relevance of the R&D data, countries have mostly reported that the main users are 
institutional users, followed by the mass media and the research community. Regarding 
completeness, Eurostat’s assessment shows that the mandatory R&D datasets are complete 
both at the national and regional level.  

In their self-assessment, countries have mostly reported a high level of accuracy of their reported 
variables. They have also provided detailed information in their metadata files on the errors that 
could occur during the data collecting and processing phases. We note that the countries are 
making sustained efforts to have a high accuracy. The conclusion is that the accuracy of the 
reporting is good but of course there are margins to improve it.    

On coherence and comparability more work is necessary, both regarding the coherence with the 
IFATS domain as well as in some cases on the coherence between preliminary and final data.   
In terms of timeliness and punctuality, countries were mostly punctual on delivering their data to 
Eurostat and we can observe that there are different practices when it comes to the timeliness of 
the release of their national figures.  

Last but not least, on accessibility, countries have reported that their R&D data is very accessible 
both at national and regional level. Regarding clarity, all the NSIs publish metadata and 
methodology chapters or sections together with the data.  

54                             Quality report on European statistics on research and development, 2024 Edition 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
GETTING IN TOUCH WITH THE EU 

In person 
All over the European Union there are hundreds of Europe Direct information centres. You can find the 
address of the centre nearest you at: https://europa.eu/european-union/contact_en 

On the phone or by email 
Europe Direct is a service that answers your questions about the European Union. You can contact this 
service: 
– by freephone: 00 800 6 7 8 9 10 11 (certain operators may charge for these calls), 
– at the following standard number: +32 22999696 or  
– by email via: https://europa.eu/european-union/contact_en 

FINDING INFORMATION ABOUT THE EU 

Online 
Information about the European Union in all the official languages of the EU is available on the Europa 
website at: https://europa.eu/european-union/index_en 

EU publications 
You can download or order free and priced EU publications at: https://op.europa.eu/en/publications. 
Multiple copies of free publications may be obtained by contacting Europe Direct or your local information 
centre (see https://europa.eu/european-union/contact_en). 

EU law and related documents 
For access to legal information from the EU, including all EU law since 1952 in all the official language 
versions, go to EUR-Lex at: http://eur-lex.europa.eu 

Open data from the EU 
The EU Open Data Portal (http://data.europa.eu/euodp/en) provides access to datasets from the EU. 
Data can be downloaded and reused for free, for both commercial and non-commercial purposes. 

 
 
 
 
 
 
 
 
 
 
 
 
 
Quality report on  
European statistics on 
research and development

Research and experimental development (R&D) comprise creative and 
systematic work undertaken in order to increase the stock of knowledge 
– including knowledge of humankind, culture and society – and to devise 
new applications of available knowledge. R&D statistics play a central role 
in measuring the expenditure and the employment in R&D activities. The 
availability and quality of the R&D statistics are key to benchmarking the 
relation between investment in R&D expenditure and innovation.  This is the 
first edition of the summary quality report of R&D statistics. It aims to provide 
users with relevant information on the quality of the R&D data disseminated in 
Eurostat’s online database under the Science and technology.

For more information
https://ec.europa.eu/eurostat/

K
S
-
F
T
-
2
4
-
0
0
5
-
E
N

-

N

P

u

b

l

i

c

a

t

i

o

n

t

i

t

l

e

   ISBN: 978-92-68-19018-0

2023 edition