European Innovation Scoreboard : identification and treatment of outliers.
Date: 2024-01-15
Institutions: Directorate-General for Research and Innovation
Subjects: EU Member State, EU research policy, innovation, performance measurement, report, research and development, research infrastructure, statistics

European Innovation Scoreboard 

Identification and Treatment of Outliers 

 
 
 
 
 
 
 
 
 
European Innovation Scoreboard – Identification and Treatment of Outliers 

European Commission 

Directorate-General for Research and Innovation 
Directorate G - Common Policy Centre 
Unit G1 - Common R&I Strategy & Foresight Service 
Contact    Alexandr Hobza, Chief Economist and Head of Unit G1 
Athina Karvounaraki, Team Leader, Coordinator of European Innovation Scoreboard 2023, Unit G1 
Email  

RTD-STATISTICS@ec.europa.eu 
RTD-PUBLICATIONS@ec.europa.eu 

Directorate-General for Internal Market, Industry, Entrepreneurship and SMEs 
Directorate A - Strategy and Economic Analysis 
Unit A1 - Chief Economist 
Contact    Román Arjona, Chief Economist and Head of Unit A1 
Xosé-Luís Varela-Irimia, Coordinator of European Innovation Scoreboard 2023, Unit A1 
Email   GROW-A1@ec.europa.eu 

European Commission 
B-1049 Brussels 

Manuscript completed in December 2023  

This  document  has  been  prepared  for  the  European  Commission,  however  it  reflects  the  views  only  of  the  authors,  and  the 
European Commission shall not be liable for any consequence stemming from the reuse. 

PDF  

ISBN 978-92-68-10999-1 

doi:10.2777/038677 

KI-05-23-576-EN-N 

Luxembourg: Publications Office of the European Union, 2024 

© European Union, 2024  

is  authorised  under  a  Creative  Commons  Attribution  4.0 

The  reuse  policy  of  European  Commission  documents  is  implemented  by  Commission  Decision  2011/833/EU  of  12  December 
2011  on  the  reuse  of  Commission  documents  (OJ  L  330,  14.12.2011,  p.  39).  Unless  otherwise  noted,  the  reuse  of  this 
document 
licence 
(https://creativecommons.org/licenses/by/4.0/). This means that reuse is allowed provided appropriate credit is given and any 
changes are indicated. 
For any use or reproduction of elements that are not owned by the European Union, permission may need to be sought directly 
from the respective rightholders.  

(CC  BY  4.0) 

International 

 
 
 
 
 
 
 
 
 
EUROPEAN COMMISSION 

European Innovation Scoreboard 

Identification and Treatment of Outliers 

The report was written by 

Nordine Es-Sadki, Aishe Khalilova and Hugo Hollanders 

Maastricht University (UNU-MERIT) 

as part of the Development of the European Innovation Scoreboard project for the European Commission, 
Directorate-General for Research and Innovation 

Directorate-General for Research and Innovation 
2024          Directorate-General  for  Internal  Market,  Industry,  Entrepreneurship  and  SMEs            EN

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Table of contents 

1. 

INTRODUCTION .............................................................................................. 3 

2. 
CURRENT  METHOD  USED  FOR  OUTLIER  DEFINITION  AND 
TREATMENT IN THE EIS/RIS ...................................................................................... 4 

3. 
INNOVATION SCOREBOARD ..................................................................................... 5 

OUTLIERS  TREATMENT  METHODS:  RESULTS  FOR  THE  EUROPEAN 

Descriptives and tests of normality of the distribution of the original raw data .. 5 

No outlier treatment .................................................................................................. 9 

Quartile ranges, Box-plot method ........................................................................... 12 

Median Absolute Deviation (MAD) method ............................................................. 16 

Z-score (current method but different cut-off values) .......................... 20 

Analysing the impact for the EIS ............................................................................. 24 

4. 

RESULTS FOR THE REGIONAL INNOVATION SCOREBOARD .................. 30 

No outlier treatment .................................................................................................. 30 

Quartile ranges, Box-plot method ........................................................................... 31 

Median Absolute Deviation (MAD) method ............................................................. 31 

Z-score method (current method but different cut-off values) .............................. 32 

Analysing the impact for the RIS ............................................................................. 32 

5. 

TRANSFORMING DATA FIRST FOR SKEWNESS ........................................ 37 

6. 
METHOD ....................................................................................................................... 41 

CRITERIA  TO  DETERMINE  THE  BEST  OUTLIER 

IDENTIFICATION 

7. 

8. 

9. 

CONCLUSIONS ............................................................................................... 44 

REFERENCES ................................................................................................. 45 

ANNEXES ........................................................................................................ 46 

ANNEX A: Outliers identified by country by indicator and change in 

normalised scores per indicator ....................................................................................... 46 

ANNEX B. Outliers treatment results for EU Member States only......................... 58 

ANNEX C. Box-plot overviews EIS .......................................................................... 60 

ANNEX D. RIS 2021 results by region ..................................................................... 65 

ANNEX E. Outliers identified and replaced, by region – RIS 2021 ........................ 85 

ANNEX F. Box-plot overviews RIS........................................................................... 91 

2 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1.  Introduction 

In  business  statistics,  particularly  based  on  survey  data,  the  distribution  of  variables  is  often  highly 
skewed,  resulting  in  observations  that  differ  substantially  from  most  observations  in  the  sample. 
These units are referred to as outliers. Outliers result from various factors such as: 

•  Human errors, e.g. data entry errors 

• 

Instrument errors, e.g. measurement errors 

•  Data processing errors, e.g. data manipulation 

•  Sampling errors, e.g. extracting data from wrong sources 

•  Not an error, the value is extreme, just a ‘novelty’ in the data 

The first four factors are not prevalent in business statistics, particularly when the data originates from 
official  sources  such  as  Eurostat  and  National  Statistical  Offices  (NSOs)  as  this  data  has  been 
thoroughly  checked  before  publication.  Therefore,  the  most  common  outliers  in  economic  analysis 
and  in  statistics  used  for  the  European  Innovation  Scoreboard  (EIS)  and  the  Regional  Innovation 
Scoreboard  (RIS)  are  the  last  factor,  genuine  observed  values.  They  are  also  referred  to  as  a 
representative outlier by the European Business Statistics Manual1 published by Eurostat. These units 
represent correct values in the population. These outliers can have an impact on various statistics and 
statistical estimates such as mean values, leading to under- or over-estimated resulting values. Morris 
Hansen (1983) proposed the rule of thumb that an outlier is any observation whose removal from the 
sample  changes  the  estimate  (in  regression  analysis)  of  a  parameter  of  interest  by  10  percent  or 
more. Woolridge (2015) refers to a ‘large amount’. Notwithstanding the decision to remove, correct or 
leave  an  outlier  (for  a  discussion  on  this  topic  see  McClelland,  2000),  it  is  necessary  to  be  able  to 
detect its presence. 

An  appropriate  method  of  identifying  and  subsequently  treating  outliers  is  therefore  required.  This 
involves  first  identifying  outliers  and  afterwards  modifying  or  replacing  them  with  substituted  values. 
There  are  various  methods  available  to  treat  outliers  and  researchers  often  must  take  a  balanced 
approach as there is a trade-off between the loss of accuracy, if representative outliers are modified, 
and the bias of estimates (mean, min-max etc.), if outliers are kept as is. There are key factors when 
deciding  on  treating  an  outlier  or  not.  It  goes  both  ways  and  any  decision  whether  to  keep  the 
observation  as  is  or  replacing  it  can  have  an  impact  on  the  accuracy  of  the  data,  potential  biases, 
robustness of the data and the interpretability of the results (McClelland, 2000). 

The  objective  of  this  study  is  to  review  the  methodology  on  the  definition  of  the  outliers  taking  into 
consideration the distributions of the EIS 2021 set of indicators and propose alternative approaches. 
The  same  approaches  will  be  applied  and  tested  to  the  RIS  2021  indicator  set.  Depending  on  the 
outcome  of  this  analysis  and  agreed  by  the  European  Commission,  a  revised  method  for  treating 
outliers may be considered for future editions of the EIS and RIS. 

The  remainder  of  this  report  is  structured  as  follows.  Section  2  briefly  discusses  the  current  outlier 
method.  Section  3  analyses  the  impact  of  using  alternative  outlier  methods  on  the  EIS,  similarly 
Section  4  analyses  the  impact  of  alternative  outlier  methods  on  the  RIS.  Section  5  analysis  for  a 
selection of outlier methods what the impact on the EIS is if transforming the data first for skewness is 
done before outlier treatment method. Section 6 concludes the report and provides recommendations.   

1 Eurostat (2021). European Business Statistics Manual 2021 edition. 

https://ec.europa.eu/eurostat/web/products-manuals-and-guidelines/-/ks-gq-21-001 

3 

 
 
 
 
 
2.  Current method used for outlier definition and treatment in the EIS/RIS 

In calculating the Summary Innovation Index for both the EIS and RIS, before normalizing the data for 
the individual indicators, outliers are identified and treated. The method used is known as the z-score 
outlier treatment method. The z-score is a measure of how many standard deviations a data point is 
away  from  the  mean.  Data  points  with  a  z-score  greater  than  a  threshold  (e.g.,  +/-2  in  the  current 
EIS/RIS  measurement  framework)  are  considered  outliers.  Over  the  years,  it  has  been  noticed  that 
the  underlying  assumption  of  a  normal  distribution  does  not  apply  for  some  of  the  indicators.  The 
2021 revision brought many changes to the  indicators’ framework, which call for a thorough analysis 
of their distributions and the effect they may have on the results. 

The  methodology  for  calculating  the  Summary  Innovation  Index  is  outlined  in  detail  in  the  EIS 
methodology report. The main steps are: 

•  Step 1: Setting reference years 

•  Step 2: Imputing for missing values 

•  Step 3: Identifying and replacing outliers 

•  Step 4: Transforming data that have highly skewed distributions across countries2 

•  Step 5: Determining Maximum and Minimum scores 

•  Step 6: Calculating re-scaled scores 

•  Step 7: Calculating composite innovation indexes 

•  Step 8: Calculating relative-to-EU performance scores 

Of importance is to emphasize that outlier identification and treatment precedes data analysis. In step 
3,  outliers  are  identified  and  replaced  using  the  z-score  method.  Positive  outliers  are  defined  as 
observations  with  country  scores  higher  than  the  mean  across  all  countries  plus  twice  the  standard 
deviation3,  while  negative  outliers  are  those  with  scores  lower  than  the  mean  across  all  countries 
minus  twice  the  standard  deviation.  These  outliers  are  replaced  by  the  respective  maximum  and 
minimum  values  observed  over  all  the  years  and  all  countries  not  including  these  outliers.  The  RIS 
takes a similar approach but uses regional scores instead of country scores. 

Although it is one of the main methods used to identify outliers it has one disadvantage as the mean 
and standard deviation are themselves affected by the presence of outliers. The crux of the problem 
is  that  the  standard  deviation  is  based  on  squared  distances,  so  extreme  points  are  much  more 
influential than those close to the mean. Moreover, three problems can be identified when using the 
mean as the central tendency indicator (Miller, 1991). Firstly, it assumes that the distribution is normal 
(outliers  included).  Secondly,  the  mean  and  standard  deviation  are  strongly  impacted  by  outliers. 
Thirdly, as stated by Cousineau and Chartier (2010), this method is very unlikely to detect outliers in 
small samples. 

This  report  examines  the  treatment  of  outliers  on  changes  in  country  rankings,  placement  in 
performance group and changes in the Summary Innovation Index (SII) or Regional Innovation Index 
(RII) of countries or regions. Step 4 may also have an influence on these issues as transforming data 
before  treating  outliers  influences  the  results.  This  is  analysed  in  Section  5.  Moreover,  performance 
group threshold values (70, 100, and 125) also have an influence on country or region placement in 
performance groups. Changes in performance group threshold values are not examined in this report 
as the focus is to examine the effects of using alternative outlier treatment methods. 

2 Steps 3 and 4 are switched to examine the impact on EIS scores in Section 5 of this report. 

3  The  thresholds  of  2  or  3  are  commonly  used.  These  values  are  related  to  the  statistical  concept  of  the 

confidence level and they correspond to 95.5% and 99.7% confidence levels respectively. 

4 

 
 
 
3.  Outliers treatment methods: results for the European Innovation 

Scoreboard 

This  section  provides  the  results  of  the  following  different  approaches  to  treat  outliers  based  on  the 
EIS 2021. 

•  No outlier treatment 

•  Box-plot method 

•  MAD method 

•  Z-score (current approach but with higher threshold +/-3 instead of +/-2) 

There are advantages and disadvantages of each outlier detection method, for an overview see Seo 
(2006). In summary, both MAD and z-score method assume a normal distribution of the data. Shiffler 
(1988)  showed  that  the  maximum  value  of  the  z-score  depends  on  the  sample  size  given  as  and 
therefore (Iglewicz & Hoaglin, 1993) consider that the z-score method is not appropriate for small data 
sets (which is not applicable to the EIS nor RIS). The Box-plot method is more suitable to both normal 
distributed and skewed data (Seo, 2006). It is less sensitive to extreme observations (Seo, 2006), and 
therefore  in  general  detects  fewer  outliers  compared  to  the  MAD  or  z-score  methods.  For  each 
alternative  a  short  description  of  the  method  and  the  alternative  results  of  the  EIS  ranking  are 
provided, for elaborate examples and explanation of the details of the methods see Seo (2006). 

It is important to note that each of these methods (including the current method used for the EIS and 
RIS)  are  issued  to  identify  outliers  and  not  to  transform  all  the  data.  Identified  outliers  are  replaced 
with the first corresponding value below or equal to the threshold value. For example, if country A has 
an observed value of 50, and this value is identified as an upper outlier by the  Box-plot method then 
that value will be replaced by the first actual value observed in the dataset that is equal or below the 
upper limit value. 

Another approach is Winsorization. To winsorize data means to set the tails of the distribution equal to 
a specified percentile of the data (Ghosh and Vogt, 2012). For example, a 99% winsorization sets all 
observations  greater  than  the  99th  percentile  equal  to  the  value  at  the  99th  percentile  and  all 
observations less than the 1st percentile equal to the value at the 1st percentile. In effect, to winsorize 
data  means  to  change  the  tail  values  in  a  dataset  to  less  extreme  values.  Using  the  percentiles  to 
treat  outliers  is  similar  to  the  trimming  approach.  However,  trimming  involves  removing  the  outliers 
from the data set whereas winsorizing replaces outliers with the nearest non-outlier values. Given that 
winsorization  does  not  detect  outliers  but  treats  data  samples  regardless  of  whether  there  are  any 
observed  outliers,  this  method  is  not  considered  as  an  appropriate  approach  for  the  detection  and 
treatment of outliers. 

  Descriptives and tests of normality of the distribution of the original raw data 

Since  several  outlier  detection  methods  assume  the  normality  of  a  population,  it  is  useful  to  test 
whether  the  underlying  distribution  of  the  individual  indicators  is  normal,  or  at  least  symmetric.  This 
can be done via several approaches such as reviewing the distribution graphically using histograms 
or  Box-plots,  analysing  the  skewness  and  kurtosis  and  lastly  by  employing  statistical  tests.  In  this 
section the skewness and kurtosis are analysed together with two statistical tests, Shapiro-Wilk and 
D’Agostino-Pearson. Hair et al. (2010) and Bryne (2010) argued that data is considered to be normal 
if skewness is between ‐2 to +2 and kurtosis is between ‐7 to +7. The D'Agostino-Pearson normality 
test first computes the skewness and kurtosis to quantify how far the distribution is from Gaussian in 
terms  of  asymmetry  and  shape  (D’Agostino  et  al.,  1990).  It  then  calculates  how  far  each  of  these 
values  differs  from  the  value  expected  with  a  Gaussian  distribution  and  computes  a  single  P  value 
from  the  sum  of  these  discrepancies.  It  is  a  versatile  and  powerful  normality  test,  and  is  generally 
recommended  (Saculinggan  and  Balase,  2013).  The  Shapiro-Wilk  normality  test  is  another  well-
known option when it comes to normality tests (Yab and Sim, 2011). Unlike the D’Agostino-Pearson 
test, the Shapiro-Wilk test does not use the shape of the distribution to determine whether or not it is 

5 

 
 
normal. Instead, it compares the actual Standard Deviation (SD) of the data to the SD computed from 
the slope of the quantile–quantile plot (Q-Q plot) for the data and calculates their ratio. If the data are 
sampled from a Gaussian distribution, the two values will be similar, so the ratio will be close to 1.0, 
while a ratio much different from one would suggest a non-normal distribution. The Shapiro-Wilk test 
works very well if every value is unique but does not work as well when several values are identical 
(Yab and Sim, 2011; Saculinggan and Balase, 2013). In the database used for this report 46% of the 
values are unique.  

Descriptive statistics including the skewness, kurtosis and the Shapiro-Wilk and D’Agostino-Pearson 
of the raw data are provided in Table 1a and 1b below. 

Based  on  the  Shapiro-Wilk  and  D’Agostino-Pearson  statistical  tests,  none  of  the  distributions  are 
considered to be normal4. However, based on the threshold values from Hair et al. (2010) and Bryne 
(2010),  six  out  of  the  32  indicators  are  considered  to  be  non-normally  distributed.  These  are  1.2.1 
International  co-publications,  2.2.2  Non-R&D  Innovation  expenditure,  3.2.2  Public-private  co-
publications,  3.3.2.  Trademarks  and  3.3.3  Designs  (Kurtosis  is  within  accepted  values)  and  4.2.3 
Sales of new to market products. 

As  indicated  before,  the  z-score  and  MAD  methods  assume  normality  and  are  hence  not  the  most 
suitable outlier  detection  methods for non-normal distributions. The  Box-plot method is  better suited 
for both normal and skewed distributions. 

Sections  3.2  to  3.5  present  an  analysis  of  the  results  obtained  through  various  outlier  treatment 
methods. The discussion focuses on the main changes, while all changes are visually represented in 
the  accompanying  figures  and  tables.  Figures  1  to  4  depict  increases  in  the  Summary  Innovation 
Index (SII) highlighted in green and decreases in red, alongside changes in rankings represented by 
non-horizontal lines for each distinct outlier identification method. Meanwhile, Tables 2 to 5 illustrate 
performance  groups  or  rank  variations,  with  increases  highlighted  in  green  and  decreases  in  red. 
Each table is followed by a summary presenting an overview of the average SII change resulting from 
the  application  of  the  alternative  outlier  method.  Additionally,  two  sets  of  figures  are  provided:  one 
presenting  the  average  SII  change  based  on  the  EIS  2021  grouping,  and  the  other  showcasing  the 
average  SII  change  based  on  the  new  grouping  resulting  from  the  alternative  outlier  method.  Both 
sets of results are presented for comprehensive analysis. 

4 Log-normal transformation has also been used by first calculating the logarithm of the raw data values and then 
perform  the  tests.  The  results  are  mostly  similar  except  for  indicator  4.1.1  which  is  considered  normally 
distributed based on the D’Agostino test. 

6 

 
 
 
 
Table 1a Descriptive statistics of the raw data using EIS 2021 values (all countries, EU and non-EU, full 8 years 2013-2020)  

Indicator 
1.1.1 
1.1.2 
1.1.3 
1.2.1 
1.2.2 
1.2.3 
1.3.1 
1.3.2 
2.1.1 
2.1.2 
2.1.3 
2.2.1 
2.2.2 
2.2.3 
2.3.1 
2.3.2 
3.1.1 
3.1.2 
3.2.1 
3.2.2 
3.2.3 
3.3.1 
3.3.2 
3.3.3 
4.1.1 
4.1.2 
4.2.1 
4.2.2 
4.2.3 
4.3.1 
4.3.2 
4.3.3 

Name 
Doctorate graduates 
Tertiary education 
Lifelong learning 
International co-publications 
Most cited publications 
Foreign doctorate students 
Broadband penetration 
Digital skills 
Public R&D 
Venture capital 
Government R&D support 
Business R&D 
Non-R&D Innovation expenditure 
Innovation exp per employee 
Upgrading ICT skills 
ICT specialists 
Product innovators 
Business process innovators 
Innovative SMEs collaborating with others 
Public-private co-publications 
Job-to-job mobility 
PCT patents 
Trademarks 
Designs 
KIA employment 
Employment innovative enterprises. 
Medium high-tech exports 
Knowledge-intensive services exports 
Sales of new to market products 
Resource productivity 
PM 2.5 air emissions 
Environment-related technologies 

Mean 
0.70 
39.69 
11.40 
1947.53 
0.08 
19.91 
15.01 
29.32 
0.58 
0.09 
0.10 
0.98 
0.81 
4576.77 
21.61 
3.66 
24.48 
35.18 
11.26 
327.89 
6.75 
2.55 
6.51 
3.24 
14.29 
54.70 
45.75 
52.23 
11.17 
1.84 
0.22 
12.11 

Standard Error  Median 
0.03 
0.52 
0.50 
283.36 
0.00 
1.05 
0.56 
0.74 
0.01 
0.01 
0.01 
0.05 
0.05 
171.44 
0.51 
0.08 
0.60 
0.61 
0.40 
49.05 
0.15 
0.16 
0.40 
0.19 
0.30 
0.82 
0.87 
1.18 
0.47 
0.05 
0.02 
0.27 

0.70 
40.60 
8.50 
931.90 
0.09 
14.20 
13.00 
29.00 
0.55 
0.07 
0.07 
0.70 
0.60 
3766.72 
22.00 
3.50 
24.09 
35.94 
9.82 
142.63 
6.65 
1.36 
5.01 
2.62 
13.80 
57.05 
48.67 
50.22 
9.93 
1.58 
0.10 
11.74 

7 

Mode 
0.70 
47.60 
5.70 
481.21 
#N/A 
19.00 
8.00 
26.00 
0.21 
0.00 
0.05 
1.44 
0.01 
237.11 
22.00 
5.00 
38.62 
35.94 
9.27 
#N/A 
6.90 
0.00 
1.22 
0.00 
12.20 
50.54 
18.56 
33.50 
9.06 
0.83 
0.26 
0.00 

Sample 
Variance 
0.21 
79.23 
71.76 
24409553 
0.00 
315.36 
92.90 
159.31 
0.06 
0.01 
0.01 
0.65 
0.80 
8229873 
76.33 
1.89 
105.03 
106.72 
45.86 
731475 
6.22 
7.53 
48.51 
10.80 
26.34 
195.78 
228.44 
420.75 
66.56 
0.84 
0.07 
22.67 

Range 
3.37 
38.80 
33.40 
36339 
0.15 
86.79 
62.00 
54.00 
1.00 
0.43 
0.41 
3.86 
8.87 
12287 
40.00 
5.90 
46.04 
58.05 
41.93 
6109 
10.80 
10.62 
44.27 
21.13 
28.20 
69.22 
62.13 
79.52 
68.34 
3.93 
1.07 
29.66 

Count 
304 
296 
288 
304 
304 
288 
296 
288 
304 
256 
304 
304 
288 
280 
288 
280 
288 
288 
288 
304 
272 
304 
304 
304 
296 
288 
304 
304 
296 
280 
256 
304 

 
 
 
 
Table 1b Tests for normality of the distribution using the raw data of EIS 2021 values (all countries, EU and non-EU, full 8 years 2013-2020)  

Indicator 
1.1.1 
1.1.2 
1.1.3 
1.2.1 
1.2.2 
1.2.3 
1.3.1 
1.3.2 
2.1.1 
2.1.2 
2.1.3 
2.2.1 
2.2.2 
2.2.3 
2.3.1 
2.3.2 
3.1.1 
3.1.2 
3.2.1 
3.2.2 
3.2.3 
3.3.1 
3.3.2 
3.3.3 
4.1.1 
4.1.2 
4.2.1 
4.2.2 
4.2.3 
4.3.1 
4.3.2 
4.3.3 

Name 
Doctorate graduates 
Tertiary education 
Lifelong learning 
International co-publications 
Most cited publications 
Foreign doctorate students 
Broadband penetration 
Digital skills 
Public R&D 
Venture capital 
Government R&D support 
Business R&D 
Non-R&D Innovation expenditure 
Innovation exp per employee 
Upgrading ICT skills 
ICT specialists 
Product innovators 
Business process innovators 
Innovative SMEs collaborating with others 
Public-private co-publications 
Job-to-job mobility 
PCT patents 
Trademarks 
Designs 
KIA employment 
Employment innovative enterprises. 
Medium high-tech exports 
Knowledge-intensive services exports 
Sales of new to market products 
Resource productivity 
PM 2.5 air emissions 
Environment-related technologies 

Kurtosis 
3.05 
-0.89 
-0.10 
33.85 
-1.30 
2.87 
3.72 
-0.58 
-0.98 
2.70 
0.47 
1.61 
45.29 
0.18 
-0.54 
-0.45 
-0.67 
-0.14 
1.36 
33.02 
-0.84 
0.71 
10.43 
6.94 
3.01 
0.01 
-0.18 
-0.94 
26.15 
0.26 
2.55 
1.12 

Skewness 
1.03 
-0.03 
0.95 
5.84 
0.07 
1.55 
1.61 
0.26 
0.20 
1.55 
1.12 
1.26 
5.62 
0.84 
0.23 
0.34 
-0.03 
-0.36 
1.02 
5.75 
0.11 
1.31 
2.98 
2.10 
1.27 
-0.59 
-0.66 
0.15 
4.24 
0.98 
1.85 
0.45 

Shapiro-
Wilks 
0.93 
0.97 
0.89 
0.27 
0.94 
0.84 
0.88 
0.97 
0.97 
0.86 
0.86 
0.88 
0.56 
0.93 
0.98 
0.98 
0.98 
0.98 
0.93 
0.29 
0.98 
0.80 
0.67 
0.81 
0.92 
0.97 
0.95 
0.96 
0.64 
0.90 
0.73 
0.97 

SWTest1 
0.00 
0.00 
0.00 
0.00 
0.00 
0.00 
0.00 
0.00 
0.00 
0.00 
0.00 
0.00 
0.00 
0.00 
0.00 
0.00 
0.00 
0.00 
0.00 
0.00 
0.00 
0.00 
0.00 
0.00 
0.00 
0.00 
0.00 
0.00 
0.00 
0.00 
0.00 
0.00 

normal2 
no 
no 
no 
no 
no 
no 
no 
no 
no 
no 
no 
no 
no 
no 
no 
no 
no 
no 
no 
no 
no 
no 
no 
no 
no 
no 
no 
no 
no 
no 
no 
no 

D'Agostino 
65.91 
34.29 
33.35 
372.95 
338.69 
90.68 
103.34 
11.20 
53.68 
80.51 
47.70 
66.90 
358.63 
27.50 
8.95 
9.05 
12.20 
6.27 
47.34 
369.23 
25.17 
60.97 
213.56 
153.34 
78.12 
15.29 
19.82 
44.94 
297.16 
35.21 
93.74 
18.32 

DPTest3 
0.00 
0.00 
0.00 
0.00 
0.00 
0.00 
0.00 
0.00 
0.00 
0.00 
0.00 
0.00 
0.00 
0.00 
0.01 
0.01 
0.00 
0.04 
0.00 
0.00 
0.00 
0.00 
0.00 
0.00 
0.00 
0.00 
0.00 
0.00 
0.00 
0.00 
0.00 
0.00 

normal2 
no 
no 
no 
no 
no 
no 
no 
no 
no 
no 
no 
no 
no 
no 
no 
no 
no 
no 
no 
no 
no 
no 
no 
no 
no 
no 
no 
no 
no 
no 
no 
no 

Notes: 1. SWTEST: p-value of the Shapiro-Wilk test, 2. p-value threshold (alpha) of 0.05, 3. DPTest: p-value of the D’Agostino-Pearson test.  

8 

 
 
 
  No outlier treatment 

Before  other  alternatives  to  the  current  method  are  explored,  this  section  examines  what  the  EIS 
results would look like if outliers were not treated, and the data are taken as is. 

Using no outlier treatment leads to significant changes in the ranking and for three countries changes 
in  performance  group  membership.  Both  Belgium  and  Denmark  move  from  the  Innovation  Leader 
group to the Strong Innovator group but lose no place in ranking. This shows the first peculiarity of the 
results, there are only three Innovation Leader countries, Sweden, Switzerland and Finland. Sweden 
also takes first place instead of Switzerland. 

The reason for the change in first place is two-fold. First the current EIS method identifies and treats 
more outliers for Sweden compared to Switzerland, 35 vs. 18, all of which are upper limits. In the case 
of  Sweden,  the  most  affected  indicators  are  1.1.3  Lifelong  learning,  with  all  8  years  being  an  upper 
limit outlier. In the case of  Switzerland,  the same holds for the following  indicators: 2.2.3 Innovation 
expenditures per employee, 2.3.2 ICT Specialists, and 3.3.1 PCT patents. Secondly, when the upper 
limits are substituted with the nearest highest values just below or equal to the upper limit, Sweden's 
performance  remains  at  least  on  par  with  that  of  Switzerland.  The  reordering,  with  Switzerland  now 
surpassing Sweden, suggests that Switzerland excels in other indicators compared to Sweden, which 
experienced  less  pronounced  effects  from  the  outlier  treatment.  See  Annex  A.1  for  an  overview  of 
outliers identified by country by indicator. The annex also includes information on the changes in MIN 
and MAX values and on the application of square root transformation. 

The  third  group  membership  change  is  Hungary,  whose  ranking  does  not  change  but  improves  in 
terms  of  group  membership  moving  from  the  Emerging  Innovator  group  to  the  Moderate  Innovator 
group. This is not that surprising as Hungary had an SII of 68.2 which is close to the threshold of 70. 
Lithuania,  the  Netherlands,  Portugal,  Sweden,  Iceland,  Israel  and  the  UK  have  all  improved  their 
ranking.  Iceland  makes  the  strongest  improvement  in  ranking,  rising  4  places  from  13  to  9.  The 
Netherlands and Israel move up 2 places, the  other  countries move one place  up. Estonia, Greece, 
Luxembourg, Austria, Slovenia, Bosnia and Herzegovina, Switzerland and the United Kingdom have 
moved down in ranking. 

Not treating outliers leads to different results compared to the method currently used for the EIS. Not 
treating  outliers mostly  benefits  Emerging Innovators  as their average  SII increases with 5.4 (based 
on the EIS 2021 performance grouping) percentage points and it has more negative impacts for the 
Innovation Leaders. Their average SII decreases with 6.4 percentage points. The average SII of the 
Moderate Innovator group increases with 0.6 percentage points and for the Strong Innovator group it 
decreases with 2.0. 

In summary, one country moves up one performance group (Hungary) and two countries move down 
(Belgium and Denmark), 7 countries improve in rank and 8 countries decrease in rank. 

9 

 
 
 
 
Figure 1 No outlier treatment method – EIS 2021

Note: Increases in SII shown in the middle with green lines, decreases with red. Rank changes are shown with non-horizontal line changes.

10 

 
 
 
 
Table 2 EIS 2021 results using the current method vs. no outlier treatment 

Country 
Sweden 
Switzerland 
Finland 
Denmark 
Belgium 
Netherlands 
Luxembourg 
United Kingdom 
Iceland 
Norway 
Germany 
Israel 
Estonia 
Austria 
France 
Ireland 
Cyprus 
Italy 
Portugal 
Slovenia 
Malta 
Spain 
Czechia 
Lithuania 
Greece 
Serbia 
Croatia 
Hungary 
Poland 
Slovakia 
Latvia 
Turkey 
Montenegro 
Bulgaria 
North Macedonia 
Ukraine 
Romania 
Bosnia and 
Herzegovina 

No outlier treatment 

EIS 2021 results 

Differences 

SII 

Rank  Performance group 

SII 

Rank  Performance group  SII 

Rank 

134.1 
131.3 
126.9 
122.2 
121.6 
120.4 
118.4 
118.0 
117.5 
117.4 
116.0 
115.6 
114.2 
110.7 
106.9 
104.2 
101.3 
94.1 
90.9 
90.6 
90.5 
86.7 
83.5 
81.0 
80.4 
75.7 
73.0 
72.7 
67.6 
66.1 
64.1 
59.1 
53.4 
52.6 
48.7 
40.6 
39.3 

Innovation Leader 
1 
Innovation Leader 
2 
Innovation Leader 
3 
Strong Innovator 
4 
Strong Innovator 
5 
Strong Innovator 
6 
Strong Innovator 
7 
Strong Innovator 
8 
9 
Strong Innovator 
10  Strong Innovator 
11  Strong Innovator 
12  Strong Innovator 
13  Strong Innovator 
14  Strong Innovator 
15  Strong Innovator 
16  Strong Innovator 
17  Strong Innovator 
18  Moderate Innovator 
19  Moderate Innovator 
20  Moderate Innovator 
21  Moderate Innovator 
22  Moderate Innovator 
23  Moderate Innovator 
24  Moderate Innovator 
25  Moderate Innovator 
26  Moderate Innovator 
27  Moderate Innovator 
28  Moderate Innovator 
29  Emerging Innovator 
30  Emerging Innovator 
31  Emerging Innovator 
32  Emerging Innovator 
33  Emerging Innovator 
34  Emerging Innovator 
35  Emerging Innovator 
36  Emerging Innovator 
37  Emerging Innovator 

137.5 
140.2 
134.1 
128.4 
127.9 
123.2 
123.8 
123.7 
114.7 
119.4 
119.0 
110.1 
119.5 
116.5 
107.8 
106.2 
100.5 
95.3 
90.0 
91.4 
89.4 
86.5 
81.6 
80.9 
81.1 
72.9 
70.9 
68.2 
62.6 
61.6 
58.2 
52.1 
49.4 
47.1 
43.6 
31.6 
33.6 

Innovation Leader 
2 
Innovation Leader 
1 
Innovation Leader 
3 
Innovation Leader 
4 
Innovation Leader 
5 
Strong Innovator 
8 
Strong Innovator 
6 
7 
Strong Innovator 
13  Strong Innovator 
10  Strong Innovator 
11  Strong Innovator 
14  Strong Innovator 
9 
Strong Innovator 
12  Strong Innovator 
15  Strong Innovator 
16  Strong Innovator 
17  Strong Innovator 
18  Moderate Innovator 
20  Moderate Innovator 
19  Moderate Innovator 
21  Moderate Innovator 
22  Moderate Innovator 
23  Moderate Innovator 
25  Moderate Innovator 
24  Moderate Innovator 
26  Moderate Innovator 
27  Moderate Innovator 
28  Emerging Innovator 
29  Emerging Innovator 
30  Emerging Innovator 
31  Emerging Innovator 
32  Emerging Innovator 
33  Emerging Innovator 
34  Emerging Innovator 
35  Emerging Innovator 
38  Emerging Innovator 
37  Emerging Innovator 

-3.5 
-8.9 
-7.1 
-6.2 
-6.3 
-2.9 
-5.4 
-5.7 
2.9 
-2.0 
-3.0 
5.5 
-5.4 
-5.7 
-0.9 
-2.0 
0.8 
-1.1 
0.9 
-0.8 
1.1 
0.2 
1.8 
0.1 
-0.7 
2.8 
2.1 
4.5 
5.1 
4.4 
5.9 
7.0 
3.9 
5.5 
5.0 
9.0 
5.7 

38.0 

38  Emerging Innovator 

34.7 

36  Emerging Innovator 

3.2 

1 
-1 
0 
0 
0 
2 
-1 
-1 
4 
0 
0 
2 
-4 
-2 
0 
0 
0 
0 
1 
-1 
0 
0 
0 
1 
-1 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
2 
0 

-2 

Note:  Performance  groups  or  rank  increases  highlighted  in  green,  decreases  in  red.  The  difference  columns  show  the 
differences of the SII and ranking of the countries between the current method and the alternative method. 

Average change in SII 
Performance group 
Emerging Innovator 
Moderate Innovator 
Strong Innovator 
Innovation Leader 

Rank and performance group change 
Change 
Increase 
Decrease 
No change 
Total countries 

Based on old grouping 
5.4 
0.6 
-2.0 
-6.4 

Based on new grouping 
5.5 
1.0 
-2.6 
-6.5 

Rank change 
7 
8 
23 
38 

11 

Performance group change 
1 
2 
35 
38 

 
 
 
 
 
  Quartile ranges, Box-plot method 

The  first  alternative  approach  to  treat  outliers  is  the  identification  of  outliers  based  on  the  quartile 
range. Box plots can be used for this approach (see Figure 1). Quartiles (Q1, Q2, Q3) divide a data 
set  into  four  groups,  each  containing  about  25%  (or  a  quarter)  of  the  data  points.  There  are  three 
quartiles: Q1, Q2, and Q3. Q1 (also known as the first quartile or lower quartile) is the 25th percentile 
of  the  data.  Q2  (the  second  quartile)  is  the  50th  percentile  or  median  of  the  data.  Q3  (the  third  or 
upper quartile) is the 75th percentile of the data. 

The  Interquartile  Range  (IQR)  is  the  distance  between  the  first  and  third  quartile.  Subtract  the  first 
quartile from the third quartile to find the interquartile range. IQR = Q3 – Q1. 

The upper and lower fences represent values more and less than 75th and 25th percentiles (3rd and 
1st quartiles), respectively. Any value that is 1.5 x IQR greater than the third quartile is designated as 
an outlier and any value that is 1.5 x IQR less than the first quartile is also designated as an outlier. 

Figure 1 Box-plot with outliers 

Results of treating the EIS 2021 with the Box-plot method 

Using  the  Box-plot  method  would  lead  to  one  performance  group  change  as  Luxembourg  would 
improve and join the Innovation Leaders. No other performance group change is observed. There are 
several rank changes. Bulgaria, Germany, Estonia, Lithuania and the Netherlands gain one place in 
the  ranking  if  the  Box-plot  method  was  used.  Czechia,  Greece,  Montenegro  and  Norway  lose  one 
place in the ranking and the United Kingdom loses two places in ranking from 7 to 9. 

The  improvement  in  rank  of  the  Netherlands  can  be  attributed  to  a  very  small  change  in  average 
relative  to  EU  performance  scores  where  the  Netherlands  improves  from  123.2  to  123.9  (a  0.7 
percentage  point  increase).  Estonia  also  sees  improvement,  with  a  notable  increase  of  3.6 
percentage points, moving from 119.5 to 123.1. On the other hand, the UK experiences a drop of two 
places, with the relative to EU performance score going down to 122.1 from 123.7. The explanation 
for  these  shifts  in  rankings  is  due  to  several  factors.  Firstly,  the  Box-plot  method  identifies  and 
replaces  fewer  outliers  compared  to  the  EIS  method  (z-score  +/-  2  cut-off  values)  as  illustrated  in 
Table 6. This leads to higher MAX values and lower MIN values (see Table 26).  

In  the  case  of  the  Netherlands,  there  are  no  changes  in  outliers  identified  and  replaced  when 
comparing the EIS method (z-score +/-2) with the Box-plot method. However, for the UK there are 3 
fewer  outliers  identified  and  replaced,  all  related  to  indicator  2.1.3  Government  R&D  support. 
Surprisingly,  for  Estonia  9  more  outliers  are  identified  and  replaced  across  all  indicators  and  all  8 
years.  Specifically,  three  more  upper  limit  outliers  have  been  identified  and  treated  for  each  of  the 
following three indicators: 2.2.2 Non-R&D Innovation expenditure, 3.3.2 Trademarks and 4.3.2 PM 2.5 

12 

 
 
 
 
 
air  emissions.  In  the  case  of  3.1.1  Product  innovation,  the  current  EIS  method  (z-score  +/2  cut-off 
values) identifies one outlier, which was not identified using the Box-plot method. The replacement of 
upper  limit  outliers  is  expected  to  negatively  impact  Estonia’s  average  normalised  score.  This  is 
indeed the case as the average normalised score for all indicators does decrease from 0.622 to 0.615 
(-1.1%).  Meanwhile,  the  EU  average  normalised  score  decreases  from  0.520  to  0.499  (-4.0%). 
Because  the  EU  average  decreases  more  than  Estonia’s  average,  the  relative  to  EU  performance 
score of Estonia is higher using the Box-plot method compared to the current EIS method (z-score +/- 
2  cut-off  values),  from  119.5  to  123.1  (as  indicated  above).  The  UK  average  normalised  score 
decreases from 0.643 to 0.610 (-5.1%) and for the Netherlands from 0.641 to 0.619 (-3.4%).  

The  changes  in  rankings  of  Estonia  and  the  UK  are  mostly  driven  by  changes  in  the  minimum-
maximum (MIN-MAX). These changes stem from variations in distributions following the treatment of 
outliers.  The  normalised  scores  for  individual  indicators  depend  on  the  MIN  and  MAX  values  of  all 
countries in all years. In general, when employing the  Box-plot method with fewer outliers treated at 
both  upper  and  lower  limits,  the  range  of  MIN-MAX  values  tends  to  be  higher.  Nevertheless,  for 
individual  indicators  the  change  in  the  MIN-MAX  range  varies.  Some  indicators  experience  more 
significant changes compared to others. The more substantial changes are typically attributed to the 
absence  of  a  square  transformation  when  using  the  Box-plot  method,  whereas  the  current  EIS 
method (z-score +/- 2 cut-off values) necessitated a square root transformation. In such cases more 
outliers are identified and replaced resulting in a less skewed distribution, eliminating the need for a 
square  root  transformation.  However,  the  consequence  is  a  substantial  change  in  the  MIN-MAX 
ranges, often leading to larger ranges and, consequently, impacting the overall results.  

In situations where the application of a square root transformation remains unaffected by skewness, 
changes in scores for countries only arise when there is a change in the MIN value. If the MIN value 
remains  constant  but  the  MAX  value  changes,  the  adjustment  in  the  MIN-MAX  range  impacts  all 
countries  uniformly.  This  is  because  there  is  only  a  denominator  change  in  the  calculation  [(Xi-
MIN)/(MAX-MIN)], where X represents the country score in year i, MAX denotes the maximum values 
observed  throughout  the  entire  time  period,  and  MIN  represents  the  minimum  value  observed  over 
the entire time period. 

In  cases  where  the  MIN  value  decreases,  countries  with  lower  performance  generally  experience  a 
greater  advantage  compared  to  medium  and  high-performing  countries.  This  holds  true  unless  the 
absolute relative increase in the MAX value significantly surpasses the absolute relative decrease in 
the  MIN  value.  This  occurs  because  for  countries  with  lower  performance,  the  change  in  the 
numerator  has  a  relatively  more  positive  impact  than  the  effect  of  the  changed  denominator  (the 
modified  MIN-MAX  range),  particularly  when  the  MAX  value  does  not  change  much.  For  instance, 
consider  indicator  4.2.1  Medium  high-tech  exports.  Using  the  Box-plot  method  results  in  the 
identification and replacement of 18 fewer outliers compared to the current EIS method (z-score +/- 2 
cut-off values). This increases the MIN-MAX range with no need for a square root transformation. In 
this case the increase in the MIN-MAX range is solely attributable to a lower MIN value, decreasing 
from  16.6  to  8.16  (a  decrease  of  50.9%).  This  shift  disproportionately  benefits  countries  with  lower 
rankings. For example, Estonia, ranked 28th, experiences a 15.9% increase in the normalized score 
for this indicator, while the UK, ranked 16th, sees a 6.2% improvement.  

Explaining the  implications of the application  of a square root transformation can be shown with the 
changes in indicator 2.2.2 Non-R&D Innovation expenditures. Estonia is the second-best performing 
country on this indicator while the UK ranks 26th. Using the Box-plot outlier method 10 more outliers 
are identified and replaced, which mitigates the data’s skewness and eliminates the need for a square 
root transformation (unlike the current EIS method with z-score +/- 2 cut-off values). In this case the 
larger MIN-MAX range enhances Estonia's comparative advantage over the UK. Estonia’s normalised 
score increases  by 5.2%  whereas  the UK’s score  decreases significantly by  35.2%. See  Annex A.2 
for  an  overview  of  outliers  identified  by  country  by  indicator  using  the  Box-plot  method.  The  annex 
also provides information on changes in MIN and MAX values, along with insights into the application 
of the square root transformation. 

The  Box-plot  method  benefits  both  Innovation  Leaders  and  Emerging  Innovators  more  with  an 
average  increase  in  SII  of  2.1  and  1.4  respectfully,  based  on  the  EIS  2021  performance  grouping. 
Average SII  of the  Moderate Innovator group  increases with 0.8  and for the Strong  Innovator  group 
with 0.7. 

In  summary,  one  country  moves  up  one  performance  group  (Luxembourg),  5  countries  improve  in 
rank and 5 countries decrease in rank. 

13 

 
 
Figure 2 Box-plot outlier treatment method – EIS 2021 

Note: Increases in SII shown in the middle with green lines, decreases with red. Rank changes are shown with non-horizontal line changes. 

14 

 
 
 
EIS 2021 results 
Rank  Performance group  SII  Rank  Outliers 

Differences 

Table 3 EIS 2021 results using the current method vs. the Box-plot method 

Country 
Switzerland 
Sweden 
Finland 
Denmark 
Belgium 
Luxembourg 
Netherlands 
Estonia 
United Kingdom 
Germany 
Norway 
Austria 
Iceland 
Israel 
France 
Ireland 
Cyprus 
Italy 
Slovenia 
Portugal 
Malta 
Spain 
Lithuania 
Czechia 
Greece 
Serbia 
Croatia 
Hungary 
Poland 
Slovakia 
Latvia 
Turkey 
Bulgaria 
Montenegro 
North 
Macedonia 
Bosnia and 
Herzegovina 
Romania 
Ukraine 

Box-plot 

Rank  Performance group 

Innovation Leader 
1 
Innovation Leader 
2 
Innovation Leader 
3 
Innovation Leader 
4 
Innovation Leader 
5 
Innovation Leader 
6 
Strong Innovator 
7 
Strong Innovator 
8 
9 
Strong Innovator 
10  Strong Innovator 
11  Strong Innovator 
12  Strong Innovator 
13  Strong Innovator 
14  Strong Innovator 
15  Strong Innovator 
16  Strong Innovator 
17  Strong Innovator 
18  Moderate Innovator 
19  Moderate Innovator 
20  Moderate Innovator 
21  Moderate Innovator 
22  Moderate Innovator 
23  Moderate Innovator 
24  Moderate Innovator 
25  Moderate Innovator 
26  Moderate Innovator 
27  Moderate Innovator 
28  Emerging Innovator 
29  Emerging Innovator 
30  Emerging Innovator 
31  Emerging Innovator 
32  Emerging Innovator 
33  Emerging Innovator 
34  Emerging Innovator 

SII 
144.3 
140.6 
135.0 
129.7 
129.0 
125.4 
123.9 
123.1 
122.1 
121.0 
118.8 
116.6 
113.6 
112.7 
105.7 
105.6 
104.4 
95.4 
91.6 
91.0 
90.3 
87.9 
82.6 
82.2 
81.9 
73.8 
71.0 
68.1 
64.6 
62.8 
59.7 
54.1 
49.5 
48.9 

SII 
140.2 
137.5 
134.1 
128.4 
127.9 
123.8 
123.2 
119.5 
123.7 
119.0 
119.4 
116.5 
114.7 
110.1 
107.8 
106.2 
100.5 
95.3 
91.4 
90.0 
89.4 
86.5 
80.9 
81.6 
81.1 
72.9 
70.9 
68.2 
62.6 
61.6 
58.2 
52.1 
47.1 
49.4 

Innovation Leader 
Innovation Leader 
Innovation Leader 
Innovation Leader 
Innovation Leader 
Strong Innovator 
Strong Innovator 
Strong Innovator 
Strong Innovator 
Strong Innovator 
Strong Innovator 
Strong Innovator 
Strong Innovator 
Strong Innovator 
Strong Innovator 
Strong Innovator 
Strong Innovator 

1 
2 
3 
4 
5 
6 
8 
9 
7 
11 
10 
12 
13 
14 
15 
16 
17 
18  Moderate Innovator 
19  Moderate Innovator 
20  Moderate Innovator 
21  Moderate Innovator 
22  Moderate Innovator 
25  Moderate Innovator 
23  Moderate Innovator 
24  Moderate Innovator 
26  Moderate Innovator 
27  Moderate Innovator 
Emerging Innovator 
28 
Emerging Innovator 
29 
Emerging Innovator 
30 
Emerging Innovator 
31 
Emerging Innovator 
32 
Emerging Innovator 
34 
Emerging Innovator 
33 

4.1 
3.1 
1.0 
1.3 
1.1 
1.7 
0.7 
3.6 
-1.6 
2.0 
-0.6 
0.1 
-1.1 
2.5 
-2.1 
-0.5 
3.9 
0.1 
0.2 
1.0 
0.9 
1.3 
1.7 
0.6 
0.8 
0.9 
0.1 
-0.1 
2.1 
1.2 
1.5 
2.0 
2.4 
-0.6 

45.5 

35  Emerging Innovator 

43.6 

35 

Emerging Innovator 

1.9 

36.2 
34.8 
33.8 

36  Emerging Innovator 
37  Emerging Innovator 
38  Emerging Innovator 

34.7 
33.6 
31.6 

36 
37 
38 

Emerging Innovator 
Emerging Innovator 
Emerging Innovator 

1.5 
1.2 
2.2 

0 
0 
0 
0 
0 
0 
1 
1 
-2 
1 
-1 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
2 
-1 
-1 
0 
0 
0 
0 
0 
0 
0 
1 
-1 

0 

0 
0 
0 

5 
22 
8 
5 
5 
2 
0 
-8 
3 
5 
12 
1 
17 
0 
0 
5 
2 
1 
6 
0 
0 
0 
-4 
0 
-1 
-6 
0 
3 
0 
0 
0 
-3 
2 
4 

0 

1 
15 
0 

Note:  Performance  groups  or  rank  increases  highlighted  in  green,  decreases  in  red.  The  difference  columns  show  the 
differences between the current method and the alternative method of the SII, ranking and the identified and treated outliers by 
country. 

Average change in SII 
Performance group 
Emerging Innovator 
Moderate Innovator 
Strong Innovator 
Innovation Leader 

Rank and performance group change 
Change 
Increase 
Decrease 
No change 
Total countries 

Based on new grouping 
1.4 
0.8 
0.6 
2.0 

Performance group change 
1 
0 
37 
38 

Based on old grouping 
1.4 
0.8 
0.7 
2.1 

Rank change 
5 
5 
28 
38 

15 

 
 
 
 
 
  
  Median Absolute Deviation (MAD) method 

Absolute  deviation  from  the  median  was  (re-)discovered  and  popularized  by  Hampel  (1974)  who 
attributes the idea to Carl Friedrich Gauss (1777–1855). The median (M) is, like the mean, a measure 
of  central  tendency  but  offers  the  advantage  of  being  insensitive  to  the  presence  of  outliers.  One 
indicator  as  discussed  in  Leys  et  al.  (2013)  of  this  insensitivity  is  the  “breakdown  point”  (see,  also 
Donoho & Huber, 1983). The estimator’s breakdown point is the maximum proportion of observations 
that can be set to infinity without forcing the estimator to result in a false value (infinite or null in the 
case  of  an  estimator  of  scale).  For  example,  when  a  single  observation  has  an  infinite  value,  the 
mean  of all observations becomes infinite; hence the  mean’s breakdown point is 0. By contrast, the 
median value remains unchanged. The median becomes meaningless only when more than 50% of 
the observations are infinite. With a breakdown point of 0.5, the median is the location estimator that 
has the highest breakdown point. The same logic can be applied for the Median Absolute Deviation 
as an estimator of scale (see the formula below for a definition). Moreover, the MAD is unaffected by 
the sample size. These two properties have led Huber (1981) to describe the MAD as the “single most 
useful ancillary estimate of scale” (p.107). It is for example more robust than the classical interquartile 
range  (see  Rousseeuw  &  Croux,  1993),  which  has  a  breakdown  point  of  25%  only.  The  MAD  is 
defined as follows (Huber, 2011): 

MAD = b (𝑚𝑒𝑑𝑖𝑎𝑛|X𝑖–− 𝑥̃| 𝑖=1,2,...,𝑛) 

Where X𝑖 is the value in the series, 𝑥̃ is the sample median. Usually, b = 1.4826 a constant linked to 
the assumption of normality of the data, disregarding the abnormality induced by outliers (Rousseeuw 
&  Croux,  1993).  If  another  underlying  distribution  is  assumed,  this  value  changes  to  b  =  1/Q(0.75), 
where  Q(0.75)  is  the  0.75  quantile  of  that  underlying  distribution.  In  case  of  normality,  1/Q(0.75)  = 
1.4826 Leys et al. (2013). 

The  rejection  criterion  of  a  value  is  discussed  in  Leys  et  al.  (2013).  As  for  the  mean  and  standard 
deviation, it is necessary to define a level of decision: This remains the unavoidable subjective aspect 
of the decision. Depending on the stringency of the researcher’s criteria, which should be defined and 
justified by the researcher, Miller (1991) proposes the values of 3 (very conservative), 2.5 (moderately 
conservative)  or  even  2  (poorly  conservative).  The  limit  as  proposed  in  Leys  et  al.  (2013)  of  2.5  is 
used. The decision criterion becomes: 

M−2.5 ⁎ MAD < X𝑖 < M + 2.5 ⁎ MAD 

Results of treating the EIS 2021 with the MAD method 

Using  the  MAD  method  would  lead  to  two  performance  group  placement  changes,  the  Netherlands 
would improve and move to the Innovation Leaders, Croatia would worsen and move to the Emerging 
Innovators group. No other performance group change is observed. There are several rank changes. 
Bulgaria, Germany, France, Lithuania, Austria and Romania would gain one place in the ranking if the 
MAD  method  was  used.  The  Netherlands  improves  with  two  places  in  the  ranking.  Czechia, 
Luxembourg, Bosnia and Herzegovina, Israel, Montenegro and the UK would have lost one place in 
the ranking and Norway two places. 

The  improvement  of  the  Netherlands  with  two  ranks  surpassing  Luxembourg  and  the  UK  can  be 
attributed to several factors. Firstly, the Netherlands average normalised score increases from 0.641, 
using the current EIS method (z-score +/- 2 cut-off values), to 0.648.  

The increase is primarily because of an increased performance in two indicators 3.2.2 Public-private 
co-publications  and  3.3.1  PCT  patents,  which  increase  with  15.7%  and  21.9%  respectively.  This 
improvement  results  from  the  identification  and  treatment  of  15  additional  outliers  for  3.2.2  Public-
private  co-publications  across  all  countries  and  all  years,  and  36  more  for  3.3.1  PCT  patents.  The 
treatment of outliers resulted in a less skewed distribution. Using the current EIS method (z-score +/-2 
cut-off  values)  both  indicators  exhibited  skewness,  necessitating  a  square  root  transformation.  The 
square root transformation reduces the variance in the dataset and thus narrows the MIN-MAX range. 
Conversely, no square root transformation is applied when using the MAD outlier method, resulting in 
an increased MIN-MAX range compared to the range when using the current EIS method (z-score +/- 
2 cut-off values).  

16 

 
 
In the case of indicator 3.2.2 Public-private co-publications both MIN and MAX values increase, with 
the  MAX  value  increasing  substantially  more.  As  explained  earlier,  this  disproportionately  impacts 
lower-performing  countries  negatively,  while  simultaneously  benefiting  higher-ranked  countries. 
Although,  the  MIN  value  of  indicator  3.3.1  PCT  patents  remains  unchanged  (zero),  the  MAX  value 
increases  substantially,  once  again  benefiting  higher-ranked  countries  on  this  indicator5.  Since  the 
Netherlands is ranked higher for both indicators, its performance increases relatively more compared 
to the UK and Luxembourg, as illustrated in the overview below. 

Indicator 3.2.2 Public-private co-
publications 

Country 

Rank 

Change in normalised score 
(MAD vs. current method) 

Netherlands 

UK 

Luxembourg 

6th 

14th 

7th 

15.7% 

-4.7% 

11.4% 

Rank 

7th 

13th 

17th 

3.3.1 PCT patents 

Change in normalised score (MAD 
vs. current method) 

21.9% 

-5.5% 

-28.3% 

Similarly,  for  Norway  the  decline  in  two  ranks  is  primarily  attributable  to  a  noteworthy  decrease  in 
performance  on  indicator  3.3.2  Trademarks  by  41.7%.  Using  the  MAD  outlier  method  led  to  the 
identification  and  treatment  of  5  additional  outliers  for  this  indicator,  resulting  in  a  less  skewed 
distribution,  eliminating  the  need  for  a  square  root  transformation.  Contrastingly,  the  current  EIS 
method  (z-score  +/-  2  cut-off  values)  necessitated  a  square  root  transformation  was  applied.  Not 
applying the square root transformation increases the MIN-MAX range.  

Norway  ranks  32  out  of  38  on  this  indicator.  The  MIN  value  decreased  from  0.38  to  0.15  (-61.6%) 
benefiting  low  performing  countries.  However,  the  MAX  value  increased  substantially  from  4.46  to 
13.9  (212.4%).  Consequently,  the  larger  MIN-MAX  range  disproportionately  impacts  countries  with 
lower performance in this instance. See Annex A.3 for an overview of outliers identified by country by 
indicator  using  the  MAD  method.  The  annex  also  includes  information  on  the  changes  in  MIN  and 
MAX, as well as insights into the application of square root transformation. 

The MAD mostly benefits the Innovation Leaders whose average increase in SII is 0.8, based on the 
EIS  2021  performance  grouping.  Average  SII  of  the  Emerging  Innovators  and  Moderate  Innovator 
groups  decreases  with  1.1  and  1.2  percentage-point  on  average.  No  difference  is  observed  for  the 
Strong Innovators. 

In  summary,  one  country  moves  up  one  performance  group  (The  Netherlands)  and  one  country 
moves down (Croatia), 7 countries improve in rank and 7 countries decrease in rank. 

5 Note that in this case the MIN value stays the same, but the normalized score does change differently across 
countries because of a change in the MAX value. This is because of the use of the square root transformation 
when using the current EIS method (z-score +/- 2 cut-off values) but this is not necessary following the MAD 
outlier  treatment  method.  Thus,  the  data  values  used  to  calculate  the  normalized  scores  using  the  min-max 
procedure are different with the MAD method, and thus a difference in the MAX value does lead to changes in 
normalized scores in this case. 

17 

 
 
 
 
Figure 3 MAD outlier treatment method – EIS 2021 

Note: Increases in SII shown in the middle with green lines, decreases with red. Rank changes are shown with non-horizontal line changes. 

18 

 
 
 
 
Table 4 EIS 2021 results using the current method vs. the MAD method 

MAD 

EIS 2021 results 

Differences 

SII  Rank   Performance group  SII  Rank   Performance group 

Country 
140.3 
Switzerland 
138.9 
Sweden 
134.1 
Finland 
129.7 
Denmark 
129.3 
Belgium 
125 
Netherlands 
122.6 
Luxembourg 
122.1 
United Kingdom 
120.8 
Estonia 
120 
Germany 
119.9 
Austria 
118.5 
Norway 
115.5 
Iceland 
106.6 
France 
106.1 
Israel 
105.2 
Ireland 
102.1 
Cyprus 
95 
Italy 
91 
Slovenia 
88.5 
Portugal 
87.8 
Malta 
85.3 
Spain 
80.4 
Czechia 
79.7 
Lithuania 
79.4 
Greece 
71.5 
Serbia 
68.9 
Croatia 
67.2 
Hungary 
62.7 
Poland 
60.2 
Slovakia 
56.1 
Latvia 
51.6 
Turkey 
46.9 
Bulgaria 
46.9 
Montenegro 
43.7 
North Macedonia 
Romania 
32.9 
Bosnia and Herzegovina  32.4 
29.8 
Ukraine 

Innovation Leader 
Innovation Leader 
Innovation Leader 
Innovation Leader 
Innovation Leader 
Innovation Leader 

1 
2 
3 
4 
5 
6 
7  Strong Innovator 
8  Strong Innovator 
9  Strong Innovator 
10  Strong Innovator 
11  Strong Innovator 
12  Strong Innovator 
13  Strong Innovator 
14  Strong Innovator 
15  Strong Innovator 
16  Strong Innovator 
17  Strong Innovator 
18  Moderate Innovator 
19  Moderate Innovator 
20  Moderate Innovator 
21  Moderate Innovator 
22  Moderate Innovator 
23  Moderate Innovator 
24  Moderate Innovator 
25  Moderate Innovator 
26  Moderate Innovator 
27  Emerging Innovator 
28  Emerging Innovator 
29  Emerging Innovator 
30  Emerging Innovator 
31  Emerging Innovator 
32  Emerging Innovator 
33  Emerging Innovator 
34  Emerging Innovator 
35  Emerging Innovator 
36  Emerging Innovator 
37  Emerging Innovator 
38  Emerging Innovator 

140.2 
137.5 
134.1 
128.4 
127.9 
123.2 
123.8 
123.7 
119.5 
119 
116.5 
119.4 
114.7 
107.8 
110.1 
106.2 
100.5 
95.3 
91.4 
90 
89.4 
86.5 
81.6 
80.9 
81.1 
72.9 
70.9 
68.2 
62.6 
61.6 
58.2 
52.1 
47.1 
49.4 
43.6 
33.6 
34.7 
31.6 

Innovation Leader 
Innovation Leader 
Innovation Leader 
Innovation Leader 
Innovation Leader 

1 
2 
3 
4 
5 
8  Strong Innovator 
6  Strong Innovator 
7  Strong Innovator 
9  Strong Innovator 
11  Strong Innovator 
12  Strong Innovator 
10  Strong Innovator 
13  Strong Innovator 
15  Strong Innovator 
14  Strong Innovator 
16  Strong Innovator 
17  Strong Innovator 
18  Moderate Innovator 
19  Moderate Innovator 
20  Moderate Innovator 
21  Moderate Innovator 
22  Moderate Innovator 
23  Moderate Innovator 
25  Moderate Innovator 
24  Moderate Innovator 
26  Moderate Innovator 
27  Moderate Innovator 
28  Emerging Innovator 
29  Emerging Innovator 
30  Emerging Innovator 
31  Emerging Innovator 
32  Emerging Innovator 
34  Emerging Innovator 
33  Emerging Innovator 
35  Emerging Innovator 
37  Emerging Innovator 
36  Emerging Innovator 
38  Emerging Innovator 

SII 
0.1 
1.3 
0.1 
1.3 
1.4 
1.8 
-1.1 
-1.6 
1.3 
1 
3.4 
-0.9 
0.8 
-1.2 
-4 
-0.9 
1.6 
-0.3 
-0.3 
-1.5 
-1.6 
-1.2 
-1.2 
-1.1 
-1.7 
-1.5 
-2 
-1 
0.1 
-1.4 
-2.1 
-0.5 
-0.2 
-2.5 
0.1 
-0.6 
-2.3 
-1.8 

Rank  Outliers 

0 
0 
0 
0 
0 
2 
-1 
-1 
0 
1 
1 
-2 
0 
1 
-1 
0 
0 
0 
0 
0 
0 
0 
0 
1 
-1 
0 
0 
0 
0 
0 
0 
0 
1 
-1 
0 
1 
-1 
0 

-36 
3 
-1 
-13 
0 
-10 
-4 
-1 
-12 
-8 
-3 
5 
6 
-1 
0 
0 
-5 
-4 
3 
-1 
0 
0 
0 
-5 
-1 
-11 
0 
-1 
-1 
0 
0 
-3 
0 
0 
0 
12 
1 
0 

Note:  Performance  groups  or  rank  increases  highlighted  in  green,  decreases  in  red.  The  difference  columns  show  the 
differences between the current method and the alternative method of the SII, ranking and the identified and treated outliers by 
country. 

Average change in SII 
Performance group 
Emerging Innovator 
Moderate Innovator 
Strong Innovator 
Innovation Leader 

Based on old grouping 
-1.1 
-1.2 
0.0 
0.8 

Based on new grouping 
-1.2 
-1.2 
-0.1 
1.0 

Rank and performance group change 
Change 
Increase 
Decrease 
No change 
Total countries 

Rank change 
7 
7 
24 
38 

Performance group change 
1 
1 
36 
38 

19 

 
 
 
 
 
 
 
 Z-score (current method but different cut-off values) 

A Z-score measures the distance between  a data point  and the mean  using standard deviations. Z-
score can be positive or negative. The sign tells you whether the observation is above or below the 
mean. For example, a Z-score of +2 indicates that the data point falls two standard deviations above 
the  mean,  while  a  -2  signifies  it  is  two  standard  deviations  below  the  mean.  This  is  the  current 
approach as used in the EIS and RIS. A Z-score of zero equals the mean. Statisticians also refer to Z-
score as standard scores. 

Z-scores  can  quantify  the  outliers  in  a  dataset  in  case  the  data  follow  the  normal  distribution.  The 
basic z score formula for a sample is: 

For example, there is a test score of 190. The test has a mean (μ) of 150 and a standard deviation (σ) 
of 25. Assuming a normal distribution, the z score would be: 

z = (x – μ) / σ 

• 

z = (x – μ) / σ 

•  = (190 – 150) / 25 = 1.6. 

The z score shows how many standard  deviations from the  mean the score  is. In this example, the 
score is 1.6 standard deviations above the mean. 

If  all  data  was  converted,  the  center  becomes  0  and  the  Z-score  corresponding  to  each  data  point 
represents the distance from the center in terms of standard deviation.  In this analysis the z-score is 
used  to  identify  outliers,  and  only  the  outliers  are  replaced  with  the  next  to  highest  or  lowest  value 
within the allowed range of the cut-off values. 

The further away an observation’s Z-score is from zero, the more unusual it is. Currently the EIS and 
RIS  use  cut-off  values  of  +/-2.  The  Z-score  method  tends  to  detect  more  outliers  in  case  of  non-
normal  distribution  compared  to  the  Box-plot  method  (Seo,  2006).  An  alternative  cut-off  value  for 
finding outliers are Z-scores of +/-3 or further from zero, and thus a less strict rule of outlier treatment. 
In this case, any Z-score greater than +3 or less than  -3 is considered as an outlier and is replaced 
with  the  corresponding  value  of  an  observation  that  falls  with  the  allowed  range.  For  example  if 
country A has a z-score of 3.4 in the year 2022, which for example corresponds to the value 50, the 
value of 50 is replaced with the first observed corresponding value below or equal to the z-score of 3, 
for instance a z-score of 2.99 corresponds with a value of 47 (which can be a value from any country 
in any year), then the observed value of 50 of country A in the year 2022 is replaced with 47. 

Results of treating the EIS 2021 with the Z-score method (+/- 3 cut-off values) 

Using the Z-score method with +/-3 as cut-off values has led to several changes in the ranking. The 
figure  shows  improvement  in  SII  in  green,  and  rank  changes  by  non-horizontal  line  changes.  The 
table shows performance groups or rank increases highlighted in green, decreases in red. 

Using  the  Z-score  method  would  lead  to  two  performance  group  changes,  Hungary  would  improve 
and  move  to  the  Moderate  Innovators,  Denmark  would  worsen  and  move  to  the  Strong  Innovators 
group, however just below the threshold with an SII of 124.9. No other performance group change is 
observed. There are several rank changes. The Netherlands, Israel and Ukraine would have gained 
two places in ranking if the Z-score method was used. Belgium, Bulgaria, Germany and Norway gain 
one  place  in  the  ranking.  Estonia  and  Luxembourg  would  have  lost  two  places  in  the  ranking. 
Denmark, Austria, Romania, Bosnia and Herzegovina, Iceland and Montenegro one place. 

Ukraine’s  increase  in  average  normalised  score  is  the  highest  (21.3%)  when  comparing  the  current 
EIS  method  (z-score  +/-  2  cut-off  values),  with  the  Z-score  method  using  +/-  3  cut-off  values.  The 
latter  method  identifies  and  replaces  262  fewer  outliers.  As  a  result,  more  indicators  still  have  a 
skewed  distribution  and  necessitate  a  squared  root  transformation.  Those  indicators  that  required  a 
squared root transformation have the most impact on changes in the results.  

20 

 
 
For  example,  substantial  positive  changes  of  varying  magnitudes  are  evident  across  countries  for 
indicator  3.3.3  Designs.  The  Z-score  method  with  +/-  3  as  cut-off  values  identifies  4  fewer  outliers 
compared to the current EIS method (z-score with +/- 2 cut-off values). While using the current EIS 
method  (z-score  +/-  2  cut-off  values)  indicator  3.3.3  Designs  does  not  necessitate  a  square  root 
transformation. However, such a transformation is necessary when using the Z-score method with +/- 
3  cut-off  values.  This  has  led  to  Ukraine’s  normalised  score  to  increases  with  1094.5%.  See  Annex 
A.4 for an overview of outliers identified by country by indicator using the Z-score (+/- 3 cut-off values) 
method.  The  annex  also  includes  information  on  the  changes  in  MIN  and  MAX  values,  as  well  as 
insights into the application of square root transformation. 

The  Z-score  method  with  +/-  3  cut-off  mostly  benefits  the  Emerging  Innovators  whose  average 
increase  in  SII  is  4.4,  based  on  the  EIS  2021  performance  grouping.  Average  SII  of  the  Moderate 
Innovators also increases with 0.8. Innovation Leaders SII average decreases the most with 2.3, and 
Strong Innovators average SII decreases with 0.7. 

In summary, one country moves up one performance group (Hungary) and one country moves down 
(Denmark), 7 countries improve in rank and 8 countries decrease in ranking. 

21 

 
 
Figure 4 Z-score outlier treatment method – EIS 2021 

Note: Increases in SII shown in the middle with green lines, decreases with red. Rank changes are shown with non-horizontal line changes. 

22 

 
 
 
 
 
Table 5 EIS 2021 results using the current method (Z-score with cut-off values of +/-2) vs. the 
Z-score method with cut-off values of +/-3 

Z-score (cut-off values +/-3) 

EIS 2021 results 

Differences 

SII  Rank 

SII  Rank 

Country 
138.8 
Switzerland 
137.2 
Sweden 
130.5 
Finland 
125.2 
Belgium 
124.9 
Denmark 
122 
Netherlands 
121.9 
United Kingdom 
121.3 
Luxembourg 
118.9 
Norway 
117.8 
Germany 
117.2 
Estonia 
116.1 
Israel 
114.2 
Austria 
111.4 
Iceland 
106.4 
France 
106 
Ireland 
102.4 
Cyprus 
94.4 
Italy 
91.5 
Slovenia 
91.5 
Portugal 
89.5 
Malta 
87.7 
Spain 
84.1 
Czechia 
81.4 
Greece 
80.9 
Lithuania 
74.4 
Serbia 
72.7 
Croatia 
71.8 
Hungary 
67 
Poland 
65.7 
Slovakia 
63.4 
Latvia 
55.4 
Turkey 
51.9 
Bulgaria 
51.5 
Montenegro 
47.5 
North Macedonia 
39.5 
Ukraine 
Bosnia and Herzegovina  39.3 
Romania 

38.6 

 Performance 
group 
Innovation Leader 
1 
Innovation Leader 
2 
Innovation Leader 
3 
Innovation Leader 
4 
Strong Innovator 
5 
Strong Innovator 
6 
Strong Innovator 
7 
Strong Innovator 
8 
9 
Strong Innovator 
10  Strong Innovator 
11  Strong Innovator 
12  Strong Innovator 
13  Strong Innovator 
14  Strong Innovator 
15  Strong Innovator 
16  Strong Innovator 
17  Strong Innovator 
18  Moderate Innovator 
19  Moderate Innovator 
20  Moderate Innovator 
21  Moderate Innovator 
22  Moderate Innovator 
23  Moderate Innovator 
24  Moderate Innovator 
25  Moderate Innovator 
26  Moderate Innovator 
27  Moderate Innovator 
28  Moderate Innovator 
29  Emerging Innovator 
30  Emerging Innovator 
31  Emerging Innovator 
32  Emerging Innovator 
33  Emerging Innovator 
34  Emerging Innovator 
35  Emerging Innovator 
36  Emerging Innovator 
37  Emerging Innovator 

140.2 
137.5 
134.1 
127.9 
128.4 
123.2 
123.7 
123.8 
119.4 
119 
119.5 
110.1 
116.5 
114.7 
107.8 
106.2 
100.5 
95.3 
91.4 
90.0 
89.4 
86.5 
81.6 
81.1 
80.9 
72.9 
70.9 
68.2 
62.6 
61.6 
58.2 
52.1 
47.1 
49.4 
43.6 
31.6 
34.7 

 Performance 
group 
Innovation Leader 
1 
Innovation Leader 
2 
Innovation Leader 
3 
Innovation Leader 
5 
Innovation Leader 
4 
Strong Innovator 
8 
Strong Innovator 
7 
6 
Strong Innovator 
10  Strong Innovator 
11  Strong Innovator 
9 
Strong Innovator 
14  Strong Innovator 
12  Strong Innovator 
13  Strong Innovator 
15  Strong Innovator 
16  Strong Innovator 
17  Strong Innovator 
18  Moderate Innovator 
19  Moderate Innovator 
20  Moderate Innovator 
21  Moderate Innovator 
22  Moderate Innovator 
23  Moderate Innovator 
24  Moderate Innovator 
25  Moderate Innovator 
26  Moderate Innovator 
27  Moderate Innovator 
28  Emerging Innovator 
29  Emerging Innovator 
30  Emerging Innovator 
31  Emerging Innovator 
32  Emerging Innovator 
34  Emerging Innovator 
33  Emerging Innovator 
35  Emerging Innovator 
38  Emerging Innovator 
36  Emerging Innovator 

SII 

Rank  Outliers 

-1.4 
-0.3 
-3.5 
-2.7 
-3.5 
-1.2 
-1.8 
-2.4 
-0.5 
-1.2 
-2.3 
6 
-2.2 
-3.3 
-1.4 
-0.1 
1.9 
-0.9 
0.2 
1.5 
0.1 
1.2 
2.5 
0.3 
0 
1.4 
1.8 
3.6 
4.4 
4.1 
5.2 
3.3 
4.8 
2 
3.9 
7.9 
4.5 

0 
0 
0 
1 
-1 
2 
0 
-2 
1 
1 
-2 
2 
-1 
-1 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
1 
-1 
0 
2 
-1 

18 
33 
15 
7 
10 
4 
7 
11 
21 
5 
10 
8 
1 
19 
7 
5 
5 
1 
6 
6 
3 
1 
0 
0 
0 
5 
0 
3 
0 
0 
7 
11 
2 
8 
1 
0 
2 

38 

Emerging Innovator  33.6 

37 

Emerging Innovator  5.1 

-1 

20 

Note:  Performance  groups  or  rank  increases  highlighted  in  green,  decreases  in  red.  The  difference  columns  show  the 
differences between the current method and the alternative method of the SII, ranking and the identified and treated outliers by 
country. 

Average change in SII 
Performance group 
Emerging Innovator 
Moderate Innovator 
Strong Innovator 
Innovation Leader 

Rank and performance group change 
Change 
Increase 
Decrease 
No change 
Total countries 

Based on old grouping 
4.4 
0.8 
-0.7 
-2.3 

Rank change 
7 
8 
23 
38 

Based on new grouping 
4.5 
1.1 
-0.9 
-2.0 

Performance group change 
1 
1 
36 
38 

23 

 
 
  
 
 
 
 
  Analysing the impact for the EIS 

3.6.1. 

Impact on performance group placement and changes in ranking 

Of  most  importance,  is  the  impact  on  performance  group  placement  and  changes  in  ranking.  No 
outlier treatment leads to substantial changes with 2 countries moving to a lower performance group 
and  one  country  improving.  MAD  and  Z-score  (cut-off  values  +/-3)  lead  to  2  countries  changing 
performance  group  changes  and  the  Box-plot  method  leads  to  one  country  changing  performance 
group. 

Table 6: Performance group changes, by outlier treatment method 

Change 

Increase 

Decrease 

No change 
Total countries 

No outlier treatment 

Box-plot 

1 

2 

35 
38 

1 

0 

37 
38 

MAD 

1 

1 

36 
38 

Z-score 
(cut-off values +/-3) 

1 

1 

36 
38 

More changes are observed when rank changes are considered. The results show a similar trend as 
in Table 6. The fewest rank changes are observed with the Box-plot method. Not treating outliers and 
the Z-score method with +/- 3 cut-off values lead to most changes, and more countries changing more 
than 2 ranks (up or down). 

Table 7: Rank changes, by outlier treatment method 

Change 

No outlier treatment 

Box-plot 

MAD 

Total  Of which =>2   Total  Of which =>2  

Total  Of which =>2  

Z-score 
(cut-off values +/-3)  
Of which 
=>2  

Total 

Increase 

Decrease 

No change 
Total countries 

7 

8 

23 
38 

4 

3 

5 

5 

28 
38 

1 

1 

1 

1 

7 

7 

24 
38 

7 

8 

23 
38 

3 

2 

The current outlier treatment method (z-score with cut-off values +/-2) used in the EIS replaces in total 
347 data  points  of  the 9,352  in total (3.7%). Together with the MAD  outlier method the  EIS method 
treats  far  more  outliers  compared  to  other  methods  considered  in  this  report.  Table  8  gives  an 
overview  of  the  number  of  outliers  replaced  by  method  for  both  the  upper  limits  and  lower  limits. 
Another interesting result of the current outlier method vs. other methods is the high number of lower 
limit outliers replaced (47 datapoints). On the other hand, the Z-score method with cut-off values +/-3 
replaces  very  few  lower  limit  datapoints  with  just  2  replacements  (4.1.2  Employment  innovative 
enterprises), as it is less strict than using +/-2 as cut-off values. In total, the Z-score method (cut-off 
values +/-3) replaces 85 datapoints, the lowest of all methods considered. The MAD method replaces 
most datapoints with 438 in total. 

24 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Table 8: Number of outliers identified and replaced, by outlier treatment method 

Outlier treatment method 
EIS 2021–- Current method (z-score with cut-off values +/-2) 
Box-plot 
MAD 
Z-score (cut-off values +/-3) 

Upper limit 
300 
232 
404 
83 

Lower limit 
47 
13 
34 
2 

Total 
347 
245 
438 
85 

All  methods  replace  far  more  upper  limit  outliers  than  lower  limit  outliers.  One  might  expect  that, 
particularly Emerging and  Moderate  Innovators  would benefit  from replacing upper limit  outliers, but 
this  is  not  necessarily  the  case  when  observing  the  average  change  in  SII  based  on  the  EIS  2021 
performance groups presented in Table 9. 

For  example,  the  MAD  method  that  replaces  most  of  the  upper  limit  outliers,  results  in  lower  SII 
averages  for  both  the  Emerging  and  Moderate  Innovators  groups.  No  outlier  treatment  leads  to  the 
most  significant  changes  measured  by  the  change  in  average  SII  across  the  different  performance 
groups.  Particularly,  Innovation  Leaders  are  punished  by  lowering  their  SII  by  an  average  of  6.4. 
Emerging  Innovators  on  the  other  hand,  benefit  from  no  outlier  treatment  as  their  SII  average 
increases by 5.4. 

Table 9: Average change in SII based on EIS 2021 grouping, by outlier treatment method 

Performance group 
Emerging Innovator 
Moderate Innovator 
Strong Innovator 
Innovation Leader 
Sum of absolute SII change  

No outlier 
treatment 
5.4 
0.6 
-2.0 
-6.4 
14.4 

Box-plot 
1.4 
0.8 
0.7 
2.1 
5.0 

MAD 
-1.1 
-1.2 
0.0 
0.8 
3.2 

Z-score 
(cut-off values +/-
3) 
4.4 
0.8 
-0.7 
-2.3 
8.2 

The reason behind this can be explained by the information in Table 10. Significantly more upper limit 
outliers are identified for both Innovation Leaders and Strong Innovators compared to Emerging and 
Moderate  innovators,  regardless  of  the  outlier  method  used.  When  outliers  are  substituted  with  the 
nearest  observed  value  below  the  upper  limits,  it  results  in  lower  maximum  values  used  in  the 
normalization process (i.e. Step 5, see section 2). Consequently, this leads to smaller denominators in 
the  calculation  [(Xi-MIN)/(MAX-MIN)],  where  X  represents  the  country  score  in  year  i,  MAX  denotes 
the  maximum  values  observed  in  the  entire  time  period,  and  MIN  denotes  the  minimum  value 
observed  in  the  entire  time  period.  The  smaller  denominators  benefit  Emerging  and  Moderate 
innovators  more  as  their  scores  are  less  impacted  by  the  outlier  treatment  compared  to  Innovation 
Leaders and Strong Innovators. The Z-score method (cut-off values +/-3) shows similar results, where 
Innovation Leaders and Strong Innovators average  SII decreases, and the average  SII of  Emerging 
and Moderate Innovators increases. The Box-plot method replaces a total of 245 datapoints (2.6% of 
9,352),  indicating  more  replacements  compared  to  the  Z-score  (cut-off  values  +/-3).  However, 
changes in SII averages, along with the MAD method, are lower compared to other methods. 

25 

 
 
 
 
 
 
 
Table 10: Number of outliers treated by performance group, by outlier treatment method 

Based on EIS 2021 grouping 

Performance 
group 
Emerging 
Innovator 
Moderate 
Innovator 
Strong 
Innovator 
Innovation 
Leader 
Total 

EIS 2021 (Z-score 
cut-off value +/-2) 

Up. 
Lim. 

Low. 
Lim.  Tot. 

Box-plot 
Low. 
Lim.  Tot. 

Up. 
Lim. 

MAD 
Low. 
Lim.  Tot. 

Up. 
Lim. 

Z-score (cut-off 
value +/-3) 

Sum 

Up. 
Lim. 

Low. 
Lim.  Tot. 

Up. 
Lim. 

Low. 
Lim.  Tot. 

27 

32 

59 

25 

12 

37 

32 

19 

51 

3 

44 

0 

44 

48 

142 

15 

157 

117 

0 

1 

48 

63 

0 

63 

22 

118 

175 

15 

190 

54 

87 
300 

0 
47 

87 
347 

42 
232 

0 
13 

42 
245 

134 
404 

0 
34 

134 
438 

4 
83 

2 

0 

0 

0 
2 

5 

87 

65 

152 

22 

177 

0 

177 

54 

488 

31 

519 

4 

267 
85  1,019 

0 

267 
96  1,115 

Based on new grouping  

EIS 2021 (Z-score 
cut-off value +/-2) 

Box-plot 
Low. 
Lim.  Tot. 

59 

32 

27 

Up. 
Lim. 

Up. 
Lim. 

Up. 
Lim. 

Low. 
Lim.  Tot. 

Performance 
group 
Emerging 
Innovator 
Moderate 
Innovator 
Strong 
Innovator 
Innovation 
87 
Leader 
347 
Total 
Note: Up. Lim. is upper limit, Low. Lim is lower limit. Tot. is total outliers. 

87 
300 

66 
232 

66 
245 

148 
404 

0 
47 

0 
13 

157 

142 

161 

44 

15 

44 

48 

48 

63 

93 

94 

25 

12 

37 

32 

0 

0 

1 

MAD 
Low. 
Lim.  Tot. 

Z-score (cut-off 
value +/-3) 

Sum 

Up. 
Lim. 

Low. 
Lim.  Tot. 

Up. 
Lim. 

Low. 
Lim.  Tot. 

19 

51 

3 

0 

63 

22 

15 

0 
34 

176 

54 

148 
438 

4 
83 

2 

0 

0 

0 
2 

5 

87 

65 

152 

22 

177 

0 

177 

54 

450 

31 

481 

4 

305 
85  1,019 

0 

305 
96  1,115 

3.6.2. 

Indicators and countries most affected by outlier treatment 

Table 11  provides the number of  outliers  identified and replaced  by indicator. The results show that 
several indicators have a lot of identified outliers and thus changes in how these are treated affect the 
results  in  terms  of  performance  and  rank  changes.  The  indicators  with  most  outliers  identified  and 
treated are provided below based on the last column of Table 11: 

Ranked by total sum of outliers identified and replaced (all methods combined, > 50 cases) 

1.  4.3.2 Air emissions by fine particulate matter (PM2.5) in Industry 

2.  3.3.1 PCT patent applications per billion GDP (in PPS) 

3.  3.3.2 Trademark applications per billion GDP (in PPS) 

4.  2.2.2 Non-R&D innovation expenditures (percentage of turnover) 

5.  4.3.3 Development of environment-related technologies, percentage of all technologies 

6.  2.1.3 Direct government funding and government tax support for business R&D (percentage 

of GDP) 

7.  3.2.2 Public-private co-publications per million population 

8.  4.3.1 Resource productivity 

9.  1.3.1 Broadband Penetration 

Table  12  provides  the  number  of  outliers  identified  and  replaced  by  country.  The  results  show  that 
only for a few countries no outliers have been identified by any of the methods. This is the case for 
Czechia,  Croatia,  Slovakia  and  Ukraine.  Furthermore,  there  are  several  other  countries  with  few 
identified  outliers.  On  the  other  hand,  there  are  countries  which  have  many  identified  and  treated 
outliers such as Iceland, Luxembourg, Israel, Switzerland and Sweden. More detailed information on 
the identified outliers by indicator, by country and the impact on the MIN-MAX ranges is provided in 
Annex A. 

26 

 
 
 
 
Table 11 Outliers identified and replaced, by indicator – EIS 2021 

EIS 2021 
(z-score +/-2) 

Limit 

Box-plot 

Limit 

MAD 

Limit 

Z-score (+/- 3) 

Limit 

Total sum of outliers identified 
and replaced (all methods 
combined) 

Limit 

# 
304 
296 
288 
304 
304 
288 
296 
288 
304 
256 
304 
304 
288 
280 
288 
280 
288 
288 
288 
304 
272 
304 
304 
304 
296 
288 
304 
304 
296 
280 
256 
304 
9,352 

Upper 
8 
2 
18 
8 
0 
9 
14 
8 
3 
13 
20 
8 
10 
18 
10 
12 
2 
2 
9 
8 
1 
21 
16 
10 
9 
0 
0 
3 
10 
14 
22 
12 
300 

Lower 
0 
1 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
1 
0 
7 
5 
0 
0 
1 
0 
0 
0 
0 
8 
18 
0 
0 
0 
0 
6 
47 

Ind. 
1.1.1 
1.1.2 
1.1.3 
1.2.1 
1.2.2 
1.2.3 
1.3.1 
1.3.2 
2.1.1 
2.1.2 
2.1.3 
2.2.1 
2.2.2 
2.2.3 
2.3.1 
2.3.2 
3.1.1 
3.1.2 
3.2.1 
3.2.2 
3.2.3 
3.3.1 
3.3.2 
3.3.3 
4.1.1 
4.1.2 
4.2.1 
4.2.2 
4.2.3 
4.3.1 
4.3.2 
4.3.3 
TOTAL 
1.1.1  Doctorate  graduates;  1.1.2  Tertiary  education;  1.1.3  Lifelong  learning;  1.2.1  International  co-publications;  1.2.2  Most  cited  publications;  1.2.3  Foreign  doctorate  students;  1.3.1  Broadband 
penetration;  1.3.2  Digital  skills;  2.1.1  Public  R&D;  2.1.2  Venture  capital;  2.1.3  Government  R&D  support;  2.2.1  Business  R&D;  2.2.2  Non-R&D  Innovation  expenditure;  2.2.3  Innovation  exp  per 
employee; 2.3.1 Upgrading ICT skills; 2.3.2 ICT specialists; 3.1.1 Product innovators; 3.1.2 Business process innovators; 3.2.1 Innovative SMEs collaborating with others; 3.2.2 Public-private co-
publications; 3.2.3 Job-to-job mobility; 3.3.1 PCT patents; 3.3.2 Trademarks; 3.3.3 Designs; 4.1.1 KIA employment; 4.1.2 Employment innovative enterprises; 4.2.1 Medium high-tech exports; 4.2.2 
Knowledge-intensive services exports; 4.2.3 Sales of new to market products; 4.3.1 Resource productivity; 4.3.2 PM 2.5 air emissions; 4.3.3 Environment-related technologies 

Total  %  Upper 
3% 
1% 
6% 
3% 
0% 
3% 
5% 
3% 
1% 
5% 
7% 
3% 
3% 
6% 
4% 
4% 
3% 
2% 
3% 
3% 
1% 
7% 
5% 
3% 
3% 
3% 
6% 
1% 
3% 
5% 
9% 
6% 
4% 

Total  % 
0% 
0% 
7% 
7% 
0% 
6% 
7% 
0% 
0% 
6% 
8% 
6% 
9% 
7% 
0% 
0% 
0% 
2% 
3% 
8% 
0% 
19% 
7% 
3% 
3% 
2% 
6% 
0% 
4% 
10% 
16% 
7% 
5% 

Total  % 
0% 
0% 
0% 
4% 
0% 
3% 
5% 
0% 
0% 
5% 
3% 
3% 
7% 
1% 
0% 
0% 
0% 
2% 
3% 
5% 
0% 
7% 
7% 
3% 
3% 
1% 
0% 
0% 
4% 
4% 
12% 
6% 
3% 

Total  % 
0% 
0% 
0% 
3% 
0% 
3% 
1% 
0% 
0% 
3% 
0% 
3% 
1% 
0% 
0% 
0% 
0% 
0% 
0% 
3% 
0% 
0% 
3% 
2% 
3% 
1% 
0% 
0% 
1% 
0% 
2% 
0% 
1% 

Lower 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
2 
0 
0 
0 
0 
0 
0 
2 

Lower 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
5 
0 
0 
0 
0 
0 
0 
0 
5 
18 
0 
0 
0 
0 
6 
34 

Lower 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
5 
0 
0 
0 
0 
0 
0 
0 
2 
0 
0 
0 
0 
0 
6 
13 

Upper 
1 
0 
0 
8 
0 
8 
4 
0 
0 
7 
1 
8 
3 
0 
0 
0 
0 
0 
1 
8 
0 
0 
10 
6 
8 
0 
0 
0 
3 
0 
6 
1 
83 

Upper 
1 
0 
19 
20 
0 
16 
20 
1 
0 
15 
25 
18 
25 
19 
0 
0 
0 
1 
9 
23 
0 
57 
21 
10 
9 
0 
0 
0 
11 
28 
41 
15 
404 

Lower 
0 
1 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
1 
0 
7 
15 
0 
0 
1 
0 
0 
0 
0 
17 
36 
0 
0 
0 
0 
18 

Upper 
11 
2 
37 
47 
0 
41 
52 
10 
3 
47 
54 
42 
58 
39 
10 
12 
2 
4 
27 
54 
1 
99 
68 
35 
34 
0 
0 
3 
35 
53 
100 
39 

Total 
11 
3 
37 
47 
0 
41 
52 
10 
3 
47 
54 
42 
58 
39 
11 
12 
9 
19 
27 
54 
2 
99 
68 
35 
34 
17 
36 
3 
35 
53 
100 
57 

1 
0 
0 
11 
0 
8 
14 
1 
0 
12 
8 
8 
20 
2 
0 
0 
0 
1 
8 
15 
0 
21 
21 
9 
8 
0 
0 
0 
11 
11 
31 
11 
232 

8 
3 
18 
8 
0 
9 
14 
8 
3 
13 
20 
8 
10 
18 
11 
12 
9 
7 
9 
8 
2 
21 
16 
10 
9 
8 
18 
3 
10 
14 
22 
18 
347 

1 
0 
19 
20 
0 
16 
20 
1 
0 
15 
25 
18 
25 
19 
0 
0 
0 
6 
9 
23 
0 
57 
21 
10 
9 
5 
18 
0 
11 
28 
41 
21 
438 

1 
0 
0 
11 
0 
8 
14 
1 
0 
12 
8 
8 
20 
2 
0 
0 
0 
6 
8 
15 
0 
21 
21 
9 
8 
2 
0 
0 
11 
11 
31 
17 
245 

1 
0 
0 
8 
0 
8 
4 
0 
0 
7 
1 
8 
3 
0 
0 
0 
0 
0 
1 
8 
0 
0 
10 
6 
8 
2 
0 
0 
3 
0 
6 
1 
85 

27 

 
 
 
  
 
 
 
 
 
 
 
 
 
 
 
 
Table 12 Outliers identified and replaced, by country – EIS 2021 

Country 
Belgium 
Bulgaria 
Czechia 
Denmark 
Germany 
Estonia 
Ireland 
Greece 
Spain 
France 
Croatia 
Italy 
Cyprus 
Latvia 
Lithuania 
Luxembourg 
Hungary 
Malta 
Netherlands 
Austria 
Poland 
Portugal 
Romania 
Slovenia 
Slovakia 
Finland 
Sweden 
Bosnia and Herzegovina 
Iceland 
Israel 
North Macedonia 
Montenegro 
Norway 
Serbia 
Switzerland 
Turkey 
Ukraine 
United Kingdom 

EIS 2021 (Z-score +/-2) 
Lower 
limit 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
1 
0 
0 
0 
0 
0 
0 
0 
0 
0 
22 
0 
0 
0 
0 
1 
8 
0 
1 
8 
6 
0 
0 
0 
0 
0 

Upper 
limit 
7 
2 
0 
12 
5 
10 
8 
0 
1 
8 
0 
1 
7 
9 
0 
26 
3 
17 
4 
1 
0 
10 
0 
7 
0 
15 
35 
2 
27 
24 
0 
0 
15 
8 
18 
11 
0 
7 

Total 
7 
2 
0 
12 
5 
10 
8 
0 
1 
8 
0 
1 
8 
9 
0 
26 
3 
17 
4 
1 
0 
10 
22 
7 
0 
15 
35 
3 
35 
24 
1 
8 
21 
8 
18 
11 
0 
7 

Upper 
limit 
2 
0 
0 
7 
0 
18 
3 
1 
1 
8 
0 
0 
5 
9 
4 
24 
0 
17 
4 
0 
0 
10 
0 
1 
0 
7 
13 
2 
18 
24 
0 
0 
9 
14 
13 
14 
0 
4 

Box-plot 
Lower 
limit 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
1 
0 
0 
0 
0 
0 
0 
0 
0 
0 
7 
0 
0 
0 
0 
0 
0 
0 
1 
4 
0 
0 
0 
0 
0 
0 

Total 
2 
0 
0 
7 
0 
18 
3 
1 
1 
8 
0 
0 
6 
9 
4 
24 
0 
17 
4 
0 
0 
10 
7 
1 
0 
7 
13 
2 
18 
24 
1 
4 
9 
14 
13 
14 
0 
4 

Upper 
limit 
7 
2 
0 
25 
13 
22 
8 
1 
1 
9 
0 
5 
12 
9 
5 
30 
4 
17 
14 
4 
1 
11 
0 
4 
0 
16 
32 
2 
21 
24 
0 
0 
10 
19 
54 
14 
0 
8 

28 

MAD 
Lower 
limit 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
1 
0 
0 
0 
0 
0 
0 
0 
0 
0 
10 
0 
0 
0 
0 
0 
8 
0 
1 
8 
6 
0 
0 
0 
0 
0 

Total 
7 
2 
0 
25 
13 
22 
8 
1 
1 
9 
0 
5 
13 
9 
5 
30 
4 
17 
14 
4 
1 
11 
10 
4 
0 
16 
32 
2 
29 
24 
1 
8 
16 
19 
54 
14 
0 
8 

Z-score (+/-3) 
Lower 
limit 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
2 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 

Upper 
limit 
0 
0 
0 
2 
0 
0 
3 
0 
0 
1 
0 
0 
3 
2 
0 
15 
0 
14 
0 
0 
0 
4 
0 
1 
0 
0 
2 
1 
16 
16 
0 
0 
0 
3 
0 
0 
0 
0 

Total 
0 
0 
0 
2 
0 
0 
3 
0 
0 
1 
0 
0 
3 
2 
0 
15 
0 
14 
0 
0 
0 
4 
2 
1 
0 
0 
2 
1 
16 
16 
0 
0 
0 
3 
0 
0 
0 
0 

Total sum of outliers identified 
and replaced (all methods 
combined) 
Lower 
limit 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
3 
0 
0 
0 
0 
0 
0 
0 
0 
0 
41 
0 
0 
0 
0 
1 
16 
0 
3 
20 
12 
0 
0 
0 
0 
0 

Upper 
limit 
16 
4 
0 
46 
18 
50 
22 
2 
3 
26 
0 
6 
27 
29 
9 
95 
7 
65 
22 
5 
1 
35 
0 
13 
0 
38 
82 
7 
82 
88 
0 
0 
34 
44 
85 
39 
0 
19 

Total 
16 
4 
0 
46 
18 
50 
22 
2 
3 
26 
0 
6 
30 
29 
9 
95 
7 
65 
22 
5 
1 
35 
41 
13 
0 
38 
82 
8 
98 
88 
3 
20 
46 
44 
85 
39 
0 
19 

 
 
 
 
3.6.3. 

Impact on distribution 

The different outlier methods also lead to different distributions. With the current method (z-score +/-2) 
the overall highest score is 140.2, the overall  lowest score is 31.6, see Table 13. When outliers are 
not treated these are resp. 134.1 and 38.0. The range is thus much smaller (108.6 vs. 96.1). This is 
also the case for Z-score (cut-off values +/-3) which identifies and treats the fewest outliers. The SII 
range  becomes  smaller  because  the  maximum  and  minimum  values  identified  in  Step  5  of  the  EIS 
methodology  (see  Section  2)  become  more  extreme  in  case  fewer  outliers  are  treated,  i.e.  higher 
maximum  values  and  lower  minimum  values.  Therefore,  during  the  normalisation  procedure,  where 
the indicator value (minus the minimum value) is divided by the maximum score minus the minimum 
score, the normalised scores become more centred resulting in smaller SII ranges. On the contrary, 
the range is increased for the  Box-plot and MAD methods (both 110.5). A possible implication could 
be that thresholds also need to be revised. 

Table 13 MAX and MIN values of SII for each outlier treatment method – EIS results 

SII values 
No treatment 
EIS 2021–- Current method (z-score +/-2 cut-off values) 
Box-plot 
MAD 
Z-score (+/-3 cut-off values) 

MAX 
134.1 
140.2 
144.3 
140.3 
138.8 

MIN 
38.0 
31.6 
33.8 
29.8 
38.6 

Range 
96.1 
108.6 
110.5 
110.5 
100.1 

Outliers treated 

347 
245 
438 
85 

3.6.4. 

Additional analysis on skewness and distribution 

Since three out of the five countries with most outliers are non-EU Member States the outlier analysis 
has  been  repeated  for  only  EU  Member  States,  see  Annex  B.  The  results  show  that  there  are  very 
few  differences  compared  to  the  results  in  section  3.3  to  3.5  when  only  EU  Member  States  are 
included in the outlier identification and treatment procedure. 

Lastly, a Box-plot overview is another useful tool to visualise the distribution of a dataset and it allows 
a straightforward identification of the outliers and comparison of outlier treatment methods. Moreover, 
they provide a quick and easy way to visualize the median, quartiles, and dispersion of outliers in a 
dataset (see the no treatment Box-plots). The Box-plot overviews in Annex C compare the four outlier 
treatment methods for each of the EIS indicators and include no treatment of outliers as baseline. The 
Box-plot  overviews  show  the  no  treatment  distribution  and  the  distributions  of  the  data  after  outlier 
treatment. As said before, outliers are replaced with the nearest observed values closest to the upper 
or lower limit. The upper and lower limits are determined by the outlier treatment method. The results 
show that regardless the indicator, the MAD and the current EIS approach (Z-score method with +/- 2 
cut-off values) treat most outliers followed by the Box-plot method. The Z-score method with +/- 3 cut-
off values identifies and treats fewer outliers compared to the other methods as it has a less strict rule 
to identify outliers. 

29 

 
 
 
 
 
4.  Results for the Regional Innovation Scoreboard 

This  section  provides  the  results  of  the  following  different  approaches  to  treat  outliers  based  on  the 
RIS 2021.The RIS uses correction factor to align the results of the RIS with the EIS. The correction 
factor is applied before the last Step 8 (see section 2) when performance scores relative to the EU are 
calculated.   

1.  No outlier treatment 

2.  Box-plot method 

3.  MAD method 

4.  Z-score (current approach but with higher threshold +/-3 instead of +/-2) 

Given the substantial number of changes for each alternative explored, only summary results will be 
discussed in  Sections 4.1 to 4.4 using the four main  performance groups. The first table presents a 
summary  of  the  average  RII  change  for  each  outlier  treatment  method.  Two  sets  of  figures  are 
included:  one  indicating  the  average  RII  change  according  to  the  RIS  2021  grouping,  and  the  other 
displaying  the  average  RII  change  based  on  the  new  grouping  resulting  from  the  alternative  outlier 
method.  Both  sets  of  results  are  included  for  comprehensive  comparison.    The  second  part  of  the 
table in each section gives the performance and rank group changes. 

Annex  D  provides  details  on  the  full  changes.  The  tables  in  Annex  D  show,  similarly  to  the  EIS, 
changes  in  performance  groups,  rank  increases  (highlighted  in  green),  or  decreases  (highlighted  in 
red). 

  No outlier treatment 

Using  no  outlier  treatment  has  led  to  several  changes  in  the  ranking,  see  Table  14.  No  outlier 
treatment increases the Regional Innovation Index (RII) average for the Emerging Innovators and the 
Moderate Innovators, the RII average decreases for Strong Innovators and Innovation Leaders. The 
Emerging Innovators benefit most with an average increase of 7.1, based on the RIS 2021 grouping. 
The Moderate Innovators average RII increases with 1.8. The average RII for the Innovation Leaders 
and Strong Innovators decreases with 5.9 and 2.8 respectively. 

Most regions change ranks, with just 29 regions keeping the same rank as the RIS 2021 ranking. In 
total, 91 regions improve their rank, and 120 regions worsen their rank. Eighteen regions increase in 
their  performance  group  placement,  and  similarly  eighteen  regions  decrease  in  their  performance 
group placement. All other regions stay in the same performance group. 

Table 14 Summary results for no outlier treatment based on the RIS 2021 

Average change in RII 
Performance group 
Emerging Innovator 
Moderate Innovator 
Strong Innovator 
Innovation Leader 

Rank and performance group change 
Change 
Increase 
Decrease 
No change 
Total regions 

Based on old grouping 
7.1 
1.8 
-2.8 
-5.9 

Based on new grouping 
6.9 
2.3 
-3.0 
-5.1 

Performance group change 
18 
18 
204 
240 

Rank change 
91 
120 
29 
240 

30 

 
 
 
  Quartile ranges, Box-plot method 

Using  the  Box-plot  method  has  led  to  several  changes  in  the  ranking,  see  Table  15.  The  Box-plot 
method  increases  the  RII  average  for  three  performance  groups,  Emerging  Innovators,  Moderate 
Innovators, and Innovation Leaders. The Emerging Innovators benefit most with an average increase 
of  3.7,  based  on  the  RIS  2021  grouping.  The  Innovation  Leaders  average  RII  increases  marginally 
with 0.1, the Strong Innovators average RII decreases with 0.1 and the Moderate Innovator average 
RII increases with 1.3. 

In  total,  204  out  of  the  240  regions  change  ranks,  with  94  regions  improving  their  rank  and  110 
regions worsening their rank. Nine regions increase in their performance group placement, and three 
regions  decrease  in  their  performance  group  placement.  All  other  regions  stay  in  the  same 
performance group. 

Table 15 Summary results for the Box-plot method based on the RIS 2021 

Average change in RII 
Performance group 
Emerging Innovator 
Moderate Innovator 
Strong Innovator 
Innovation Leader 

Based on old grouping 
3.7 
1.3 
-0.1 
0.1 

Based on new grouping 
3.7 
1.2 
0.1 
0.3 

Rank and performance group change 
Change 
Increase 
Decrease 
No change 
Total regions 

Rank change 
94 
110 
36 
240 

  Median Absolute Deviation (MAD) method 

Performance group change 
9 
3 
228 
240 

Using the MAD method has led to several changes in the ranking, see Table 16. Based on the RIS 
2021 grouping, the MAD method decreases the RII average for three performance groups, Moderate 
Innovators  (-0.9),  Strong  Innovators  (-0.4)  and  Innovation  Leaders  (-0.6).  Only  the  Emerging 
Innovators average RII increases with 0.7. 

Most regions change ranks, with just 16 regions keeping the same rank as the RIS 2021 ranking. In 
total,  103  regions  improve  their  rank,  and  121  regions  worsen  their  rank  (see  section  4.5.2  for  an 
analysis  of  those  indicators  driving  these  changes  the  most)  Eleven  regions  increase  in  their 
performance  group  placement,  and  six  regions  decrease  in  their  performance  group  placement.  All 
other regions stay in the same performance group. 

Table 16 Summary results for the MAD method based on the RIS 2021 

Average change in RII 
Performance group 
Emerging Innovator 
Moderate Innovator 
Strong Innovator 
Innovation Leader 

Rank and performance group change 
Change 
Increase 
Decrease 
No change 
Total regions 

Based on old grouping 
0.7 
-0.9 
-0.4 
-0.6 

Based on new grouping 
0.5 
-1.4 
-0.2 
-0.1 

Rank change 
103 
121 
16 
240 

31 

Performance group change 
11 
6 
223 
240 

 
 
 
 
 
 
  Z-score method (current method but different cut-off values) 

Using the Z-score method with +/- 3 as cut-off values has led to several changes in the ranking, see 
Table 17. The Z-score method increases the RII average for the Emerging Innovators the most with 
6.4,  followed  by  the  Moderate  Innovators  with  2.1.  For  both  the  Innovation  Leaders  and  Strong 
innovators the average RII decreases with 1.2 and 3.5 respectively. 

Most regions change ranks, with just 32 regions keeping the same rank as the RIS 2021 ranking. In 
total, 92 regions improve their rank, and 116 regions worsen their rank. Fourteen regions increase in 
their performance group placement, and nine regions decrease in their performance group placement. 
All other regions stay in the same performance group. 

Table 17 Summary results for the Z-score method based on the RIS 2021 

Average change in RII 
Performance group 
Emerging Innovator 
Moderate Innovator 
Strong Innovator 
Innovation Leader 

Rank and performance group change 
Change 
Increase 
Decrease 
No change 
Total regions 

Based on old grouping 
6.4 
2.1 
-1.2 
-3.5 

Rank change 
92 
116 
32 
240 

Based on new grouping 
6.5 
2.1 
-1.0 
-3.3 

Performance group change 
14 
9 
217 
240 

  Analysing the impact for the RIS 

4.5.1. 

Impact on performance group placement and changes in ranking 

Changes in performance group placement show that the no outlier treatment leads to most changes 
with  36  regions  in  total  changing  performance  group,  18  regions  moving  up  and  18  regions  moving 
down, see Table 18. The Box-plot and MAD outlier treatment methods lead to fewer changes, 12 and 
17  in  total  respectively.  The  Z-score  method  (cut-off  values  +/-3)  leads  to  14  regions  moving  to  a 
higher performance group and 9 regions to a lower performance group. 

Table 18 Performance group changes, by outlier treatment method 

Change 

Increase 

Decrease 

No change 

Total regions 

No outlier 
treatment 
18 

18 

204 

240 

Box-plot 
9 

3 

228 

240 

MAD 
11 

6 

223 

240 

Z-score 
(cut-off values +/-
3) 
14 

9 

217 

240 

More  changes  are  observed  when  rank  changes  are  considered  in  Table  19.  The  MAD  outlier 
treatment methods leads to most changes with 224 regions in total changing their rank, 103 regions 
moving up and 121 regions moving down. The other methods lead to fewer changes, but results are 
not  that  different  from  each  other.  The  results  do  show  that  regardless  of  the  outlier  method,  more 
regions  decrease  in  their  ranking  compared  to  the  original  method.  Fewest  rank  changes  are 
observed with the Box-plot method and z-score method (cut-off values +/-3). 

32 

 
 
 
 
 
 
 
 
Table 19 Rank changes, by outlier treatment method 

Change 

No outlier treatment 
Of which 
=>5  

Total 

Increase 

Decrease 

No change 
Total countries 

7 

8 

23 
38 

4 

3 

Box-plot 

MAD 

Z-score 
(cut-off values +/-3) 

Total 

5 

5 

28 
38 

Of which 
=>5  

1 

1 

Total 

7 

7 

24 
38 

Of which 
=>5  

1 

1 

Of which 
=>5  

3 

2 

Total 

7 

8 

23 
38 

The current outlier treatment method used in the RIS replaces in total 1,566 data points of the 40,810 
in total (3.8%). Table 20 gives an overview of the number of outliers replaced by method for both the 
upper limits and lower limits. The MAD outlier treatment method treats far more outliers compared to 
the other outlier methods, with in total 2,004 replaced values (4.9%). The current RIS method (z-score 
with cut-off values +/-2) treats more outliers compared to the Box-plot and Z-score (with +/- 3 cut-off 
values).  Of  interest  are  the  few  lower  limit  replacements  observed  for  the  z-score  method  (cut-off 
values +/-3). A reason for this is the less strict threshold of +/- 3, which leads to fewer observations to 
be identified as outliers. 

Table 20 Number of outliers replaced, by outlier treatment method 

Outlier treatment method 
RIS 2021–- Current method (z-score with cut-off values +/-2) 
Box-plot 
MAD 
Z-score (cut-off values +/-3) 

Upper limit 
1,379 
1,163 
1,912 
400 

Lower limit 
187 
30 
92 
9 

Total 
1,566 
1,193 
2,004 
409 

Percent 
replaced 
3.8% 
2.9% 
4.9% 
1.0% 

Not  replacing  any  outliers  leads  to  the  most  significant  changes  when  examining  the  change  in 
average RII across the different performance groups, see Table 21. Particularly, Emerging Innovators 
benefit the most, as their average RII increases by 7.1.  

The reason was discussed before, when analysing the EIS (for an explanation see section 3.6.1). The 
results in Table 22  show that far more upper  limit  outliers are identified for  both Innovation Leaders 
and  Strong  Innovators  compared  to  Emerging  and  Moderate  innovators  regardless  of  the  outlier 
method used.  

Moreover,  far  more  lower  limit  outliers  are  identified  for  Emerging  and  Moderate  Innovators,  which 
was  less  the  case  for  the  EIS  results.  This  improves  the  performance  of  Emerging  and  Moderate 
Innovators  as  lower  limit  outliers  are  replaced  with  values  just  above  or  equal  to  the  lower  limit 
threshold, thus increasing their region scores. The replacement of lower limit outliers benefits regions 
part  of  Emerging  and  Moderate  Innovators  more  compared  to  Strong  Innovators  and  Innovation 
leaders as  they have very  few  lower limit outliers.  Strong Innovators  only have  4 lower limit  outliers 
(based on RIS2019 grouping) and Innovation Leaders have no lower limit outliers (see Table 22).  

The z-score method (cut-off values +/-3) also leads to significant changes in RII (Table 21), benefiting 
both  Emerging  and  Moderate  Innovators  and  punishing  Strong  Innovators  and  Innovation  Leaders. 
The  MAD  outlier  treatment  method  results  in  the  lowest  changes  in  average  RII  across  the  four 
performance  groups.  The  biggest  decrease  observed  is  using  no  outlier  treatment  method  for  the 
Innovation Leaders, whose RII average decreases with 5.9. 

One of the consistent results observed is that Emerging Innovators average RII increases for all the 
outlier  treatment  measures  considered.  Another  noteworthy  result  is  that  for  Strong  Innovators  the 
average RII decreases for all methods. Similarly, the average RII of the Innovation Leaders decreases 
for all other methods except for the Box-plot method. 

33 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
Table 21 Average change in RII based on RIS 2021 grouping, by outlier treatment method 

Performance group 

Emerging Innovator 

Moderate Innovator 

Strong Innovator 

Innovation Leader 
Sum of absolute RII change 

No outlier 
treatment 
7.1 

1.8 

-2.8 

-5.9 

17.5 

Box-plot 
3.7 

1.3 

-0.1 

0.1 

5.2 

MAD 
0.7 

-0.9 

-0.4 

-0.6 

2.7 

Z-score (cut-off 
value +/-3) 
6.4 

2.1 

-1.2 

-3.5 

13.3 

Table 22: Number of outliers treated by performance group (), by outlier treatment method 

Based on RIS 2021 grouping 

Performance 
group 

RIS 2021 (Z-score 
cut-off value +/-2) 

Up. 
Lim. 

Low. 
Lim.  Tot. 

Box-plot 
Low. 
Lim.  Tot. 

Up. 
Lim. 

MAD 

Z-score (cut-off 
value +/-3) 

Sum 

Up. 
Lim. 

Low. 
Lim.  Tot. 

Up. 
Lim. 

Low. 
Lim.  Tot. 

Up. 
Lim. 

Low. 
Lim.  Tot. 

Emerging Innovator 

105 

180 

285 

110 

26 

136 

175 

88 

263 

Moderate Innovator 

Strong Innovator 

Innovation Leader 

162 

298 

763 

5 

2 

0 

167 

138 

300 

259 

763 

616 

3 

1 

0 

141 

253 

260 

465 

616 

974 

3 

1 

0 

34 

38 

74 

256 

466 

974 

223 

9 

0 

0 

0 

9 

43 

38 

424 

303 

727 

591 

11 

602 

74  1,096 

223  2,576 

4 

0 

1,100 

2,576 

378  4,687 

318  5,005 

Total 

1,328 

187  1,515  1,123 

30  1,153  1,867 

92  1,959 

369 

Based on new grouping 

Performance 
group 

RIS 2021 (Z-score 
cut-off value +/-2) 

Up. 
Lim. 

Low. 
Lim.  Tot. 

Box-plot 
Low. 
Lim.  Tot. 

Up. 
Lim. 

MAD 

Z-score (cut-off 
value +/-3) 

Sum 

Up. 
Lim. 

Low. 
Lim.  Tot. 

Up. 
Lim. 

Low. 
Lim.  Tot. 

Up. 
Lim. 

Low. 
Lim.  Tot. 

Emerging Innovator 

95 

178 

273 

100 

26 

126 

176 

88 

264 

Moderate Innovator 

Strong Innovator 

Innovation Leader 

170 

295 

768 

7 

2 

0 

177 

149 

297 

255 

768 

619 

3 

1 

0 

152 

235 

256 

464 

619 

992 

2 

2 

0 

24 

44 

90 

237 

466 

992 

211 

9 

0 

0 

0 

9 

33 

44 

395 

301 

696 

598 

12 

610 

90  1,104 

211  2,590 

5 

0 

1,109 

2,590 

378  4,687 

318  5,005 

Total 
Note: Up. Lim. Is upper limit, Low. Lim is lower limit. Tot. is total outliers. 

187  1,515  1,123 

30  1,153  1,867 

1,328 

92  1,959 

369 

34 

 
 
 
 
Table 23 Outliers identified and replaced, by indicator – RIS 2021 

RIS 2021 
(z-score +/-2) 

Box-plot 

MAD 

Z-score (+/-3) 

Total sum of outliers identified 
and replaced (all methods 
combined) 

Limit 

Limit 

Limit 

Limit 

Limit 

Upper  Lower  Total 

Ind. 

1.1.2 

1.1.3 

1.2.1 

1.2.2 

1.3.2 

2.1.1 

2.2.1 

2.2.2 

2.2.3 

2.3.2 

3.1.1 

3.1.2 

3.2.1 

3.2.2 

3.3.1 

3.3.2 

3.3.3 

4.1.1 

4.1.2 

4.2.3 

4.3.2 

# 

1,952 

1,960 

1,960 

1,960 

1,960 

1,960 

1,960 

1,904 

1,864 

1,960 

1,960 

1,960 

1,960 

1,960 

1,960 

1,960 

1,960 

1,960 

1,864 

1,904 

1,922 

Upper  Lower  Total 

68 

126 

102 

15 

19 

106 

73 

20 

32 

119 

30 

34 

97 

93 

103 

48 

84 

42 

0 

78 

90 

1 

0 

0 

35 

2 

0 

0 

0 

0 

0 

24 

36 

0 

0 

0 

0 

0 

28 

58 

0 

3 

69 

126 

102 

50 

21 

106 

73 

20 

32 

119 

54 

70 

97 

93 

103 

48 

84 

70 

58 

78 

93 

% 

4% 

6% 

5% 

3% 

1% 

5% 

4% 

1% 

2% 

6% 

3% 

4% 

5% 

5% 

5% 

2% 

4% 

4% 

3% 

4% 

5% 

Upper  Lower  Total 

17 

64 

102 

2 

0 

83 

60 

83 

93 

121 

1 

1 

30 

94 

94 

54 

106 

10 

0 

118 

30 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

21 

9 

0 

0 

17 

64 

102 

2 

0 

83 

60 

83 

93 

121 

1 

1 

30 

94 

94 

54 

106 

31 

9 

118 

30 

% 

1% 

3% 

5% 

0% 

0% 

4% 

3% 

4% 

5% 

6% 

0% 

0% 

2% 

5% 

5% 

3% 

5% 

2% 

0% 

6% 

2% 

Upper  Lower  Total 

19 

173 

142 

2 

0 

119 

119 

102 

108 

161 

1 

9 

64 

181 

210 

75 

160 

18 

0 

124 

125 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

23 

0 

0 

0 

0 

0 

21 

48 

0 

0 

19 

173 

142 

2 

0 

119 

119 

102 

108 

161 

1 

32 

64 

181 

210 

75 

160 

39 

48 

124 

125 

% 

1% 

9% 

7% 

0% 

0% 

6% 

6% 

5% 

6% 

8% 

0% 

2% 

3% 

9% 

11% 

4% 

8% 

2% 

3% 

7% 

7% 

9 

9 

37 

2 

0 

45 

23 

13 

26 

35 

1 

0 

20 

37 

34 

36 

31 

0 

0 

39 

3 

% 

0% 

0% 

2% 

0% 

0% 

2% 

1% 

1% 

1% 

2% 

0% 

0% 

1% 

2% 

2% 

2% 

2% 

0% 

0% 

2% 

0% 

Upper 

Lower 

Total 

113 

372 

383 

21 

19 

353 

275 

218 

259 

436 

33 

44 

211 

405 

441 

213 

381 

70 

0 

359 

248 

1 

0 

0 

35 

2 

0 

0 

0 

0 

0 

24 

59 

0 

0 

0 

0 

0 

70 

124 

0 

3 

114 

372 

383 

56 

21 

353 

275 

218 

259 

436 

57 

103 

211 

405 

441 

213 

381 

140 

124 

359 

251 

9 

9 

37 

2 

0 

45 

23 

13 

26 

35 

1 

0 

20 

37 

34 

36 

31 

0 

9 

39 

3 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

9 

0 

0 

9 

TOTAL  40,810 

1,379 

187 

1,566 

3.8% 

1,163 

30 

1,193 

2.9% 

1,912 

92 

2,004 

4.9% 

400 

409 

1.0% 

4,854 

318 

5,172 

1.1.2  Tertiary  education;  1.1.3  Lifelong  learning;  1.2.1  International  co-publications;  1.2.2  Most  cited  publications;1.3.2  Digital  skills;  2.1.1  Public  R&D;  2.2.1  Business  R&D;  2.2.2  Non-R&D 
Innovation expenditure; 2.2.3 Innovation exp per employee; 2.3.2 ICT specialists; 3.1.1 Product innovators; 3.1.2 Business process innovators; 3.2.1 Innovative SMEs collaborating with others; 3.2.2 
Public-private co-publications; 3.3.1 PCT patents; 3.3.2 Trademarks; 3.3.3 Designs; 4.1.1 KIA employment; 4.1.2 Employment innovative enterprises; 4.2.3 Sales of new to market products; 4.3.2 
PM 2.5 air emissions 

35 

 
 
 
 
 
 
 
 
 
 
 
 
4.5.2. 

Indicators and countries most affected by outlier treatment 

Table 23 provides the number of outliers identified and replaced by indicator for the RIS. The results 
show that several indicators have a lot of identified outliers and thus changes in how these are treated 
affect  the  results  in  terms  of  performance  and  rank  changes.  The  indicators  with  most  outliers 
identified and treated are  provided  below  based on the  last column of table 22. Except for indicator 
3.3.1, all other indicators are different from the indicators with most outliers found for the EIS. 

Ranked by total sum of outliers identified and replaced (all methods combined, > 300 cases) 

1.  3.3.1 PCT patent applications per billion GDP (in PPS) 

2.  2.3.2 ICT specialists (as a percentage of total employment) 

3.  3.2.2 Public-private co-publications per million population 

4.  1.2.1 International scientific co-publications per million population 

5.  3.3.3 Design applications per billion GDP (in PPS) 

6.  1.1.3. Lifelong learning 

7.  4.2.3 Sales of new-to market and new-to enterprise innovations as percentage of turnover 

8.  2.1.1 R&D expenditure in the public sector (percentage of GDP) 

Annex E provides the number of outliers identified and replaced by region. The results show that most 
outliers  identified  are  in  capital  regions  such  as  Stockholm  (SE11),  Hovedstaden  (DK01),  Helsinki-
Uusimaa (FI1B), Zürich (CH04) and Oslo of Akershus (NO01). 

4.5.3. 

Impact on distribution 

The different outlier methods also lead to different distributions. With the current method (z-score +/-2) 
the overall highest score is 155.2, the overall  lowest score is 17.9, see Table 24. When outliers are 
not treated these are resp. 153.9 and 23.1. The range is thus much smaller (137.3 vs. 130.9), as also 
observed with the EIS results. This is also the case for Z-score (cut-off values +/-3) which identifies 
and treats the fewest outliers. The SII range becomes smaller because the  maximum and minimum 
values identified in Step 5 of the EIS/RIS methodology (see Section 2) become more extreme in case 
fewer  outliers  are  treated,  i.e.  higher  maximum  values  and  lower  minimum  values.  On  the  contrary, 
the range is increased for the Box-plot and MAD methods (137.9 and 138.4).  

Table 24 MAX and MIN values of SII for each outlier treatment method – RIS results 

RII values 
No treatment 
RIS 2021–- Current method (z-score +/-2 cut-off values) 
Box-plot 
MAD 
Z-score (+/-3 cut-off values) 

MAX 
153.9 
155.2 
158.3 
156.4 
155.0 

MIN 
23.1 
17.9 
20.4 
18.1 
22.6 

Range 
130.9 
137.3 
137.9 
138.4 
132.4 

Outliers treated 

1,515 
1,153 
1,959 
378 

4.5.4. 

Additional analysis on skewness and distribution 

Box-plot  overviews  for  the  RIS  are  provided  in  Annex  F.  Similarly  to  the  EIS,  the  results  show  that 
regardless of the indicator the MAD and the current EIS approach (Z-score method with +/- 2 cut-off 
values)  treat  most  outliers  followed  by  the  Box-plot  method.  The  Z-score  method  with  +/-  3  cut-off 
values identifies and treats fewer outliers compared to the other methods as it has a less strict rule to 
identify outliers. 

36 

 
 
 
 
 
 
5.  Transforming data first for skewness  

This report has examined the treatment of outliers before the distribution was analysed for skewness. 
This  follows  the  methodological  steps  to  construct  the  SII  and  RII  (see  Section  2.1).  An  alternative 
approach is to first adjust for skewness and then treat any remaining outliers. 

This  analysis  is  conducted  for  three  methods  for  the  EIS:  the  current  EIS  method,  the  Box-plot 
method  and  the  MAD  method.  Table  25  summarizes  the  results.  The  table  shows  performance 
groups  increases  highlighted  in  green,  decreases  in  red  compared  to  the  current  method  EIS  2021 
(2nd column). 

For the current method (z-score +/- 2) transforming the data first for skewness (3rd column in Table 
26)  leads  to  three  countries  decreasing  in  performance  group,  Belgium,  Denmark  and  Cyprus. 
Hungary  increases  in  performance  group.  The  results  for  the  Box-plot  method  show  that  treating 
outliers first leads just to one country changing performance group (Luxembourg increases), whereas 
treating the data first for skewness leads to three performance group changes. Belgium and Denmark 
decrease in performance group and Hungary increases in performance group. For the MAD method, 
results are similar as for the current method but transforming data first for skewness. 

All  countries  changing  performance  group  perform  close  to  threshold  values,  e.g.  for  both  Belgium 
and  Denmark,  depending  on  the  outlier  treatment  method  or  the  order  (transforming  data  first  for 
skewness, then outlier treatment) their SII is just above or below the threshold for Innovation Leaders 
(SII  >  125).  The  same  holds  for  the  other  countries  changing  performance  group  particularly  for 
Hungary  whose  SII  is  for  all  approaches  just  below  or  above  70  (the  threshold  for  Moderate 
Innovator). See Table 26 for the SII results. 

Table  27 shows the changes in ranking compared to  the EIS 2021 ranking (2nd column). The table 
shows  rank  increases  highlighted  in  green,  and  decreases  in  red.  More  changes  in  ranking  are 
observed,  which  is  logical.  The  direction  of  change  is  identical  regardless  of  approach  (skewness 
transformation first or outlier treatment first). 

37 

 
 
 
 
 
Table  25  Treating  data  first  for  skewness  and  afterwards  for  outliers,  performance  group 
changes by method  

EIS 2021 

EIS 2021 

Box-plot 

Box-plot 

MAD 

MAD 

Method    

Outliers first 
(current method) 

Transformation 
first 

Outliers first 

Transformation 
first 

Outliers first 

Transformation 
first 

Country 

Group 

Group 

Group 

Group 

Group 

Group 

Strong Inn. 

Strong Inn. 

Strong Inn. 

Strong Inn. 

Strong Inn. 

Strong Inn. 

Moderate Inn. 
Emerging Inn. 

Strong Inn. 
Emerging Inn. 

Strong Inn. 
Emerging Inn. 

Strong Inn. 
Emerging Inn. 

Strong Inn. 
Emerging Inn. 

Inn. Leader 
Emerging Inn. 

Strong Inn. 
Emerging Inn. 

Inn. Leader 
Emerging Inn. 

Strong Inn. 
Strong Inn. 
Strong Inn. 
Strong Inn. 

Strong Inn. 
Strong Inn. 
Strong Inn. 
Strong Inn. 

Strong Inn. 
Strong Inn. 
Strong Inn. 
Strong Inn. 

Inn. Leader 
Strong Inn. 
Strong Inn. 
Strong Inn. 

Inn. Leader 
Strong Inn. 
Strong Inn. 
Strong Inn. 

Strong Inn. 
Inn. Leader 
Emerging Inn. 
Emerging Inn. 
Moderate Inn.  Moderate Inn.  Moderate Inn.  Moderate Inn.  Moderate Inn.  Moderate Inn. 
Inn. Leader 
Strong Inn. 
Strong Inn. 
Strong Inn. 
Moderate Inn.  Moderate Inn.  Moderate Inn.  Moderate Inn.  Moderate Inn.  Moderate Inn. 
Moderate Inn.  Moderate Inn.  Moderate Inn.  Moderate Inn.  Moderate Inn.  Moderate Inn. 
Strong Inn. 
Strong Inn. 
Moderate Inn.  Moderate Inn.  Moderate Inn.  Moderate Inn. 
Emerging Inn.  Moderate Inn. 
Moderate Inn.  Moderate Inn.  Moderate Inn.  Moderate Inn.  Moderate Inn.  Moderate Inn. 
Moderate Inn. 
Strong Inn. 
Emerging Inn. 
Emerging Inn. 
Moderate Inn.  Moderate Inn.  Moderate Inn.  Moderate Inn.  Moderate Inn.  Moderate Inn. 
Strong Inn. 
Inn. Leader 
Strong Inn. 
Emerging Inn.  Moderate Inn. 
Emerging Inn.  Moderate Inn. 
Emerging Inn.  Moderate Inn. 
Moderate Inn.  Moderate Inn.  Moderate Inn.  Moderate Inn.  Moderate Inn.  Moderate Inn. 
Strong Inn. 
Strong Inn. 
Strong Inn. 
Strong Inn. 
Emerging Inn. 
Emerging Inn. 
Moderate Inn.  Moderate Inn.  Moderate Inn.  Moderate Inn.  Moderate Inn.  Moderate Inn. 
Emerging Inn. 
Emerging Inn. 
Moderate Inn.  Moderate Inn.  Moderate Inn.  Moderate Inn.  Moderate Inn.  Moderate Inn. 
Emerging Inn. 
Emerging Inn. 
Inn. Leader 
Inn. Leader 
Inn. Leader 
Inn. Leader 

Belgium 
Bulgaria 
Czechia 
Denmark 
Germany 
Estonia 
Ireland 
Greece 
Spain 
France 
Croatia 
Italy 
Cyprus 
Latvia 
Lithuania 
Luxembourg 
Hungary 
Malta 
Netherlands 
Austria 
Poland 
Portugal 
Romania 
Slovenia 
Slovakia 
Finland 
Sweden 
Bosnia and 
Herzegovina 
Iceland 
Israel 
North Macedonia  Emerging Inn. 
Emerging Inn. 
Montenegro 
Strong Inn. 
Norway 
Moderate Inn.  Moderate Inn.  Moderate Inn.  Moderate Inn.  Moderate Inn.  Moderate Inn. 
Serbia 
Inn. Leader 
Switzerland 
Emerging Inn. 
Turkey 
Emerging Inn. 
Ukraine 
United Kingdom 
Strong Inn. 
Note: Performance groups increases highlighted in green, decreases in red. 

Emerging Inn. 
Strong Inn. 
Strong Inn. 
Emerging Inn. 
Emerging Inn. 
Strong Inn. 

Emerging Inn. 
Strong Inn. 
Strong Inn. 
Emerging Inn. 
Emerging Inn. 
Strong Inn. 

Emerging Inn. 
Strong Inn. 
Strong Inn. 
Emerging Inn. 
Emerging Inn. 
Strong Inn. 

Emerging Inn. 
Strong Inn. 
Strong Inn. 
Emerging Inn. 
Emerging Inn. 
Strong Inn. 

Emerging Inn. 
Strong Inn. 
Strong Inn. 
Emerging Inn. 
Emerging Inn. 
Strong Inn. 

Inn. Leader 
Emerging Inn. 
Emerging Inn. 
Strong Inn. 

Inn. Leader 
Emerging Inn. 
Emerging Inn. 
Strong Inn. 

Inn. Leader 
Emerging Inn. 
Emerging Inn. 
Strong Inn. 

Inn. Leader 
Emerging Inn. 
Emerging Inn. 
Strong Inn. 

Inn. Leader 
Emerging Inn. 
Emerging Inn. 
Strong Inn. 

Inn. Leader 
Strong Inn. 
Emerging Inn. 

Strong Inn. 
Strong Inn. 
Emerging Inn. 

Strong Inn. 
Strong Inn. 
Emerging Inn. 

Strong Inn. 
Strong Inn. 
Emerging Inn. 

Emerging Inn. 
Inn. Leader 
Inn. Leader 

Emerging Inn. 
Inn. Leader 
Inn. Leader 

Emerging Inn. 
Inn. Leader 
Inn. Leader 

Emerging Inn. 
Inn. Leader 
Inn. Leader 

Emerging Inn. 
Strong Inn. 
Strong Inn. 

Emerging Inn. 

Emerging Inn. 

Emerging Inn. 

Emerging Inn. 

Strong Inn. 

38 

 
 
 
 
Table 26 Treating data first for skewness and afterwards for outliers, SII results by method  

Method 

Country 
Belgium 
Bulgaria 
Czechia 
Denmark 
Germany 
Estonia 
Ireland 
Greece 
Spain 
France 
Croatia 
Italy 
Cyprus 
Latvia 
Lithuania 
Luxembourg 
Hungary 
Malta 
Netherlands 
Austria 
Poland 
Portugal 
Romania 
Slovenia 
Slovakia 
Finland 
Sweden 
Bosnia and Herzegovina 
Iceland 
Israel 
North Macedonia 
Montenegro 
Norway 
Serbia 
Switzerland 
Turkey 
Ukraine 
United Kingdom 

EIS 2021 
Outliers first 
(current 
method) 
SII 
127.9 
47.1 
81.6 
128.4 
119.0 
119.5 
106.2 
81.1 
86.5 
107.8 
70.9 
95.3 
100.5 
58.2 
80.9 
123.8 
68.2 
89.4 
123.2 
116.5 
62.6 
90.0 
33.6 
91.4 
61.6 
134.1 
137.5 
34.7 
114.7 
110.1 
43.6 
49.4 
119.4 
72.9 
140.2 
52.1 
31.6 
123.7 

EIS 2021 

Box-plot 

Box-plot 

MAD 

MAD 

Transformation 
first 
SII 
123.2 
50.3 
83.8 
123.7 
116.9 
117.8 
105.0 
81.8 
87.4 
105.1 
73.3 
95.5 
99.0 
60.6 
82.0 
118.6 
70.1 
89.7 
119.4 
113.1 
64.3 
90.4 
37.1 
91.3 
65.1 
129.3 
133.0 
34.3 
110.0 
108.5 
43.4 
49.9 
117.1 
74.5 
134.5 
53.8 
37.2 
117.9 

Outliers first 
SII 
129.0 
49.5 
82.2 
129.7 
121.0 
123.1 
105.6 
81.9 
87.9 
105.7 
71.0 
95.4 
104.4 
59.7 
82.6 
125.4 
68.1 
90.3 
123.9 
116.6 
64.6 
91.0 
34.8 
91.6 
62.8 
135.0 
140.6 
36.2 
113.6 
112.7 
45.5 
48.9 
118.8 
73.8 
144.3 
54.1 
33.8 
122.1 

Transformation 
first 
SII 
122.8 
53.8 
84.7 
122.0 
116.8 
117.3 
104.7 
83.4 
88.6 
104.5 
74.8 
96.0 
101.5 
63.9 
83.4 
119.6 
71.7 
90.9 
118.6 
112.0 
66.9 
91.7 
39.4 
91.5 
67.8 
127.8 
133.3 
38.4 
110.0 
111.2 
47.1 
52.1 
116.9 
75.9 
134.0 
56.5 
41.5 
116.9 

Outliers 
first 
SII 
129.3 
46.9 
80.4 
129.7 
120.0 
120.8 
105.2 
79.4 
85.3 
106.6 
68.9 
95.0 
102.1 
56.1 
79.7 
122.6 
67.2 
87.8 
125.0 
119.9 
62.7 
88.5 
32.9 
91.0 
60.2 
134.1 
138.9 
32.4 
115.5 
106.1 
43.7 
46.9 
118.5 
71.5 
140.3 
51.6 
29.8 
122.1 

Transformation 
first 
SII 
122.0 
51.7 
84.1 
121.7 
116.0 
117.1 
103.6 
82.1 
87.9 
104.7 
73.1 
95.4 
99.6 
61.3 
82.1 
117.1 
70.6 
89.5 
117.6 
112.0 
65.4 
89.7 
37.9 
90.5 
66.5 
127.3 
131.1 
35.3 
109.4 
108.3 
45.0 
48.9 
115.0 
74.9 
132.5 
54.8 
38.0 
116.3 

39 

 
 
 
 
 
Table  27  Treating  data  first  for  skewness  and  afterwards  for  outliers,  ranking  changes  by 
method (EIS) 

Method 

Box-plot 

Outliers first 
Rank 
5 
33 
24 
4 
10 
8 
16 
25 
22 
15 
27 
18 
17 
31 
23 
6 
28 
21 
7 
12 
29 
20 
37 
19 
30 
3 
2 

EIS 2021 
Transformation 
first 
Rank 
5 
33 
23 
4 
11 
9 
16 
25 
22 
15 
27 
18 
17 
31 
24 
7 
28 
21 
6 
12 
30 
20 
37 
19 
29 
3 
2 

EIS 2021 
Outliers first 
(current method) 
Rank 
5 
34 
23 
4 
11 
9 
16 
24 
22 
15 
27 
18 
17 
31 
25 
6 
28 
21 
8 
12 
29 
20 
37 
19 
30 
3 
2 

Country 
Belgium 
Bulgaria 
Czechia 
Denmark 
Germany 
Estonia 
Ireland 
Greece 
Spain 
France 
Croatia 
Italy 
Cyprus 
Latvia 
Lithuania 
Luxembourg 
Hungary 
Malta 
Netherlands 
Austria 
Poland 
Portugal 
Romania 
Slovenia 
Slovakia 
Finland 
Sweden 
Bosnia and 
Herzegovina 
Iceland 
Israel 
North Macedonia 
Montenegro 
Norway 
Serbia 
Switzerland 
Turkey 
Ukraine 
United Kingdom 
Note: Performance groups increases highlighted in green, decreases in red 

36 
13 
14 
35 
34 
11 
26 
1 
32 
38 
9 

36 
13 
14 
35 
33 
10 
26 
1 
32 
38 
7 

38 
13 
14 
35 
34 
10 
26 
1 
32 
36 
8 

Box-plot 
Transformation 
first 
Rank 
4 
33 
23 
5 
11 
8 
15 
25 
22 
16 
27 
18 
17 
31 
24 
6 
28 
21 
7 
12 
30 
19 
37 
20 
29 
3 
2 

38 
14 
13 
35 
34 
10 
26 
1 
32 
36 
9 

MAD 

Outliers first 
Rank 
5 
33 
23 
4 
10 
9 
16 
25 
22 
14 
27 
18 
17 
31 
24 
7 
28 
21 
6 
11 
29 
20 
36 
19 
30 
3 
2 

37 
13 
15 
35 
34 
12 
26 
1 
32 
38 
8 

MAD 
Transformation 
first 
Rank 
4 
33 
23 
5 
10 
8 
16 
24 
22 
15 
27 
18 
17 
31 
25 
7 
28 
21 
6 
12 
30 
20 
37 
19 
29 
3 
2 

38 
13 
14 
35 
34 
11 
26 
1 
32 
36 
9 

40 

 
 
 
 
 
6.  Criteria to determine the best outlier identification method 

For  choosing  the  best  outlier  identification  method,  some  criteria  can  be  used  based  on  the  key 
results and statistics for the EIS as provided in Table 28. 

1.  Less  skewed  distribution.  One  of  the  assumptions  when  calculating  normalised  scores  is 
that  data  are  normally  distributed.  Taking  the  less  strict  definition  by  Hair  et  al.  (2010)  and 
Bryne (2010) just six out  of the 32  indicators  are  non-normally  distributed (see  section 3.1). 
Using  the  stricter  current  EIS  approach  of  transforming  the  data  with  a  square  root 
transformation if skewness is above 1, 16 indicators are considered non-normally distributed. 
If the data are normally distributed the average of the normalised scores should be (close to) 
0.500, halfway 0, the minimum score, and 1, the maximum score. Not treating outliers leads 
to  a  relatively  low  average  normalised  score  across  all  years,  countries  and  indicators  of 
0.401  and  for  16  indicators  data  distributions  are  skewed  (see  Table  28).  Outlier  treatment 
increases the average normalised score for all methods and reduces the number of indicators 
with  skewed  data  distribution,  even  to  zero  indicators  for  the  MAD  method.  Thus  outlier 
treatment  methods  which  lead  to  more  normal  distributions  (less  skewed)  should  be 
preferred. This assessment is based on the skewness threshold of bigger than 1 as used in 
the current methodology. 

2.  Equal contribution of indicators to the SII or RII. Another evaluation criterion could be to 
choose the treatment method which ensures that indicators contribute most equally to the SII 
(Vertesy,  2016)6.  For  this,  differences  in  average  normalised  scores  per  indicator  should  be 
minimal, or the standard deviation across these average normalised scores should be lowest.  

3.  Minimizing  outlier  treatment.  Minimizing  outlier  treatment  is  also  important  (McClelland, 

2000) to not make too many changes to the actual observed values.  

4.  Consistency  with  previous  EIS  reports.  The  smallest  differences  in  results,  compared  to 
the  current  method  (z-score  +/-2  cut-off  values),  are  observed  when  utilizing  the  MAD  and 
Box-plot  methods.  However,  these  methods  also  result  in  a  greater  dispersion  between 
maximum and minimum SII values. Consequently, this can lead a more apparent divergence 
between countries, although it may not necessarily indicate an actual divergence but rather a 
consequence  of  the  outlier  treatment  method.  The  z-score  (+/-3  cut-off  values)  method 
primarily  favours  Emerging  Innovators  and  penalizes  Innovation  Leaders.  A  result  also 
reflected in the smaller range between the maximum and minimum SII values (see Table 28). 
The  smaller  range  suggests  that  these  methods  lead  to  more  converged  results  between 
countries and thus may give the impression of convergence. 

For  the  EIS,  considering  criterion  1,  the  MAD  and  Box-plot  methods  show  the  best  performance  by 
demonstrating  the  lowest  skewness,  followed  by  the  current  EIS  method  (z-score  with  +/-2  cut-off 
values).  Criterion  2,  based  on  the  standard  deviation  value,  is  mainly  satisfied  by  the  current  EIS 
method (z-score with +/-2 cut-off values) and the MAD method, with the Box-plot and z-score with +/-
3  cut-off  values  also  meeting  this  criterion.  Concerning  criterion  3,  the  MAD  method  performs  least 
favourably, indicated by the highest number of outliers (438). The optimal method in this case is the z-
score with +/-3 cut-off values, followed by the Box-plot method and the current EIS method (z-score 
with +/-2 cut-off values). As for criterion 4, the current method (z-score with +/-2 cut-off values), the z-
score with +/-3 cut-off values, and the Box-plot method emerge as the most effective approaches for 
the EIS. When comparing the two alternative outlier treatment methods with the current EIS method 
(z-score with +/-2 cut-off values), the Box-plot method results in only one change in the performance 
group,  while  the  z-score  method  (+/-3  cut-off  values)  would  lead  to  two  countries  changing  their 
performance  group.  Additionally,  the  Box-plot  method  has  fewer  rank  changes  compared  to  the  z-
score with +/-3 cut-off values. 

6  This study  has  shown  that  the  different indicators  do  no contribute  equally to  the overall  average normalised 
score.  Vértesy,  Dániel.  "A  Critical  Assessment  of  the  Quality  and  Validity  of  Composite  Indicators  of 
Innovation."  In  21st  International  Conference  on  Science  and  Technology  Indicators-STI  2016.  Book  of 
Proceedings. 2016. 

41 

 
 
 
Table  29  presents  key  results  and  statistics  for  the  RIS.  When  considering  criterion  1,  excluding 
outlier treatment results in relatively low average normalized scores (0.367), and data distributions for 
12 indicators are skewed. The use of the z-score method (with cut-off values +/-3) does not perform 
well  on  this  criterion,  as  the  number  of  indicators  with  skewed  data  distributions  remains  at  12.  In 
contrast,  the  MAD  and  Box-plot  methods  perform  the  best  on  this  criterion,  displaying  the  lowest 
skewness. Criterion 2 is primarily satisfied by the MAD method, followed by the current RIS method 
(z-score  with  +/-2  cut-off  values),  and  the  z-score  method  with  +/-3  cut-off  values  follows  suit. 
Regarding criterion 3, the MAD method detects and treats the highest number of outliers, followed by 
the current RIS method (z-score with +/-2 cut-off values), and the Box-plot method comes third. The 
z-score method with +/-3 cut-off values performs the best on this criterion, identifying and treating the 
fewest outliers. Concerning criterion 4, the current method (z-score with +/-2 cut-off values), the Box-
plot method, and the z-score with +/-3 cut-off values emerge as the most effective approaches for the 
RIS.  The  Box-plot  method  exhibits  the  lowest  number  of  changes  in  the  performance  group  (12), 
followed  by  the  MAD  method  (17).  Similarly,  the  Box-plot  method  also  demonstrates  the  lowest 
number of changes in the sub-performance group (23). 

Table 28 Key results and statistics for different outlier treatment methods (EIS) 

Average of normalises scores 
Standard deviation 
Number of indicators with skewed data distribution 
Average degree of skewness 
MIN 
MAX 
MIN-MAX range 
Numbers of outliers detected 
  Upper 
  Lower 
  Total 

No treatment 
0.401 
0.107 
16 
1.344 
38.0 
134.1 
96.1 

Average change in SII 
  Innovation Leaders 
  Strong Innovators 
  Moderate Innovators 
  Emerging Innovators 
Sum of absolute SII change  

Rank change 
  Increase 
  Decrease 
  Total 
Performance group change 
  Increase 
  Decrease 
  Total 

-6.4 
-2.0 
0.6 
5.4 
14.4 

7 
8 
15 

1 
2 
3 

EIS 2021  

(Z-score +/-2)  Box-plot 

0.438 
0.058 
6 
0.544 
31.6 
140.2 
108.6 

300 
47 
347 

0.426 
0.073 
3 
0.485 
33.8 
144.3 
110.5 

232 
13 
245 

2.1 
0.7 
0.8 
1.4 
5.0 

5 
5 
10 

1 
0 
1 

MAD 
0.435 
0.052 
0 
0.380 
29.8 
140.3 
110.5 

404 
34 
438 

0.8 
0.0 
-1.2 
-1.1 
3.2 

7 
7 
14 

1 
1 
2 

Z-score  
(+/-3) 
0.433 
0.073 
11 
0.696 
38.6 
138.8 
100.1 

83 
2 
85 

-2.3 
-0.7 
0.8 
4.4 
8.2 

7 
8 
15 

1 
1 
2 

42 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Table 29 Key results for different outlier treatment methods (RIS) 

No treatment 

(Z-score +/-2)  Box-plot 

RIS 2021 

Average of normalises scores 

Standard deviation 

Number of indicators with skewed data 
distribution 

Average degree of skewness 

MIN 

MAX 

MIN-MAX range 

Numbers of outliers detected 
  Upper 

  Lower 

  Total 

Average change in RII 

  Innovation Leaders 
  Strong Innovators 

  Moderate Innovators 

  Emerging Innovators 

Sum of absolute RII change 

Rank change 

  Increase 

  Decrease 

  Total 

Performance group (4) change 

  Increase 
  Decrease 

  Total 

Performance sub-group (12) change 

  Increase 

  Decrease 
  Total 

0.462 

0.060 

6 

0.604 
23.1 

155.2 

130.9 

1379 

187 

1566 

0.367 

0.113 

12 

1.921 
23.1 

153.9 

130.9 

-5.9 
-2.8 

1.8 

7.1 

17.5 

91 

120 

211 

18 
18 

36 

17 

55 
72 

0.455 

0.075 

5 

0.547 
23.1 

158.3 

130.9 

1163 

30 

1193 

0.1 
-0.1 

1.3 

3.7 

5.2 

94 

110 

204 

9 
3 

12 

3 

20 
23 

MAD 

0.438 

0.056 

0 

0.418 
23.1 

156.4 

130.9 

1912 

92 

2004 

-0.6 
-0.4 

-0.9 

0.7 

2.7 

103 

121 

224 

11 
6 

17 

19 

25 
44 

Z-score 
(+/-3) 

0.459 

0.062 

12 

0.908 
23.1 

155.0 

130.9 

400 

9 

409 

-3.5 
-1.2 

2.1 

6.4 

13.3 

92 

116 

208 

14 
9 

23 

13 

46 
59 

43 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
7.  Conclusions  

This  report  examines  various  methods  for  identifying  and  treating  statistical  outliers  in  the  data  to 
assess  their  impact  on  the  EIS  2021  and  RIS  2021  results.  The  findings  indicate  that  the  choice  of 
outlier  treatment  method  can  lead  to  differences  in  ranks  and  performance  group  placement 
compared  to  the  current  approach.  For  both  the  EIS  and  RIS,  most  changes  in  performance  group 
and  ranking  are  observed  when  no  outlier  treatment  is  applied  or  when  the  z-score  method  (cut-off 
values +/-3) is employed. This outcome is expected, given that fewer outliers are identified with the z-
score method (cut-off values +/-3), and none are identified without any outlier treatment. Any method 
that identifies more outliers, and subsequently addresses them by substituting the nearest observed 
values within the upper and lower limits impacts the results. 

It is however  in  general not recommended to not treat outliers as they  may have a  disproportionate 
effect  on  the  analyses  (McClelland  2014).  The  present  analysis  has  shown  that  it  is  not 
straightforward  to  recommend  one  outlier  treatment  over  another  as  each  outlier  treatment  leads  to 
different  countries  being  affected.  This  is  even  more  so  for  regions.  For  instance,  the  MAD  method 
identifies and replaces the most outliers, for both the EIS (4.7%) and RIS (4.9%). On the other hand, 
the z-score (+/-3 cut-off values) method replaces the fewest outliers but results in similar changes in 
number and direction, albeit affecting different countries (Denmark moving down and Hungary moving 
up). Consequently, the identification of more outliers does not necessarily translate to more changes 
in performance group placement or ranking when compared to the current methodology (z-score with 
cut-off values of +/-2). For example, for the EIS, the MAD method identifies 91 additional observations 
compared to the current methodology (z-score with cut-off values of +/-2). However, this only leads to 
two countries changing performance group; the Netherlands moves up and Croatia moves down (just 
below threshold of 125 with 124.9). This suggests that, at a certain point, identifying and treating more 
outliers (bringing the upper and lower limits closer to the average) results in minimal changes in the 
EIS and RIS results. 

For  both  the  EIS  and  RIS,  in  evaluating  various  approaches  to  handling  outliers  against  the  current 
method (z-score with +/-2  cut-off values), the  Box-plot method emerges as  a reliable  method, when 
examining the criteria  outlined  in section  6. On  the  other hand, the  MAD method demonstrates less 
favourable results, primarily attributed to a higher incidence of treated outliers. The z-score method, 
when employing a +/-3 cut-off, indicates persistence of skewed distributions in 11 indicators. In terms 
of  consistency  with  the  current  methodology,  both  the  current  method  (z-score  with  +/-2  cut-off 
values)  and  the  Box-plot  method  distinguish  themselves  for  their  stability  and  effectiveness  in 
managing outliers. 

For  future  studies,  it  is  recommended  to  build  on  the  analysis  in  Section  5  and  further  examine  the 
effect  on  the  results  of  switching  the  order  of  Step  3  (Identifying  and  replacing  outliers)  and  Step  4 
(Transforming data that have highly skewed distributions across countries). As discussed in sections 
3.3  to  3.5,  several  of  the  changes  in  performance  group  or  ranking  in  the  EIS  are  a  result  of  the 
application of the square root transformation. Alternative methods for transforming highly skewed data 
could  be  reviewed  as  well.  Future  studies  should  also  consider  that  these  methodological  changes 
affect  the  distribution  of  the  data  which  may  also  require  changes  in  performance  group  threshold 
values. 

44 

 
 
 
 
8.  References 

Byrne,  B.  M.  (2010).  Structural  equation  modelling  with  AMOS:  Basic  concepts,  applications,  and 
programming. New York: Routledge. 

Cousineau, D., & Chartier, S. (2010). Outliers detection and treatment: a review. International Journal 
of Psychological Research, 3(1), 58-67. 

D’agostino,  R.  B.,  Belanger,  A.,  &  D’Agostino  Jr,  R.  B.  (1990).  A  suggestion  for  using  powerful  and 
informative tests of normality. The American Statistician, 44(4), 316-321. 

Ghosh, D.,  &  Vogt,  A. (2012). Outliers: An evaluation of methodologies. In Joint statistical  meetings 
(Vol. 12, No. 1, pp. 3455-3460). 

Hampel, F. R. (1974). The influence curve and its role in robust estimation. Journal of the American 
statistical association, 69(346), 383-393. 

Hansen, M., Madow, W., and Tepping, B. (1983), \An Evaluation of Model-Dependent and Probability-
Sampling Inferences in Sample Surveys,” J. Amer. Stat. Assoc., 78, 776-793. 

Hair, J., Black, W. C., Babin, B. J. & Anderson, R. E. (2010) Multivariate data analysis (7th ed.). Upper 
Saddle River, New Jersey: Pearson Educational International. 

Huber, P. J. (2011). Robust statistics. In International encyclopaedia of statistical science (pp. 1248-
1251). Springer, Berlin, Heidelberg. 

Huber, P. J., & Donoho, D. (1983). The notion of breakdown point. A Festschrift for Erich L. Lehmann. 

Iglewicz, B., & Hoaglin, D. C. (1993). Volume 16: how to detect and handle outliers. Quality Press. 

Leys, C., Ley, C., Klein, O., Bernard, P., & Licata, L. (2013). Detecting outliers: Do not use standard 
deviation around the mean, use absolute deviation around the median. Journal of experimental social 
psychology, 49(4), 764-766. 

McClelland, G. H. (2000). Nasty data: Unruly, ill-mannered observations can ruin your analysis. In H. 
T.  Reis,  &  C.  M.  Judd  (Eds.),  Handbook  of  research  methods  in  social  and  personality  psychology 
(pp. 393–411). Cambridge: Cambridge University Press. 

Miller,  J.  (1991).  Reaction  time  analysis  with  outlier  exclusion:  Bias  varies  with  sample  size.  The 
quarterly journal of experimental psychology, 43(4), 907-912. 

Rousseeuw, P. J., & Croux, C. (1993). Alternatives to the median absolute deviation. Journal of the 
American Statistical association, 88(424), 1273-1283. 

Saculinggan, M., & Balase, E. A. (2013, April). Empirical power comparison of goodness of fit tests for 
normality  in  the  presence  of  outliers.  In  Journal  of  Physics:  Conference  Series  (Vol.  435,  No.  1,  p. 
012041). IOP Publishing. 

Seo,  S.  (2006).  A  review  and  comparison  of  methods  for  detecting  outliers  in  univariate  data  sets 
(Doctoral dissertation, University of Pittsburgh). 

Shiffler, R. E. (1988). Maximum Z scores and outliers. The American Statistician, 42(1), 79-80. 

Tukey, J. W. (1977). Exploratory data analysis (Vol. 2, pp. 131-160). 

Vértesy,  D.  (2016).  A  Critical  Assessment  of  the  Quality  and  Validity  of  Composite  Indicators  of 
Innovation. In 21st International Conference on Science and Technology Indicators-STI 2016. Book of 
Proceedings. 

Wooldridge, J. M. (2015). Introductory econometrics: A modern approach. Cengage learning. 

Yap, B. W., & Sim, C. H. (2011). Comparisons of various types of normality tests. Journal of Statistical 
Computation and Simulation, 81(12), 2141-2155. 

45 

 
 
 
9.  ANNEXES 

  ANNEX A: Outliers identified by country by indicator and change in normalised scores per indicator 

Annex A.1: EIS current method  

Number of outliers identified by country by indicator EIS current method (vs. no outlier outliers) 

Green highlighted cells show the number of identified and treated outlier using the current EIS outlier method (z-score +/- 2 cut-off values) 

Country/Ind. 
Belgium 
Bulgaria 
Czechia 
Denmark 
Germany 
Estonia 
Ireland 
Greece 
Spain 
France 
Croatia 
Italy 
Cyprus 
Latvia 
Lithuania 
Luxembourg 
Hungary 
Malta 
Netherlands 
Austria 
Poland 
Portugal 
Romania 
Slovenia 
Slovakia 
Finland 
Sweden 
Bosnia and 
Herzegovina 
Iceland 
Israel 
North Macedonia 
Montenegro 
Norway 
Serbia 
Switzerland 
Turkey 
Ukraine 
United Kingdom 
Total 

1.1.1  1.1.2  1.1.3  1.2.1  1.2.2  1.2.3  1.3.1  1.3.2  2.1.1  2.1.2  2.1.3  2.2.1  2.2.2  2.2.3  2.3.1  2.3.2  3.1.1  3.1.2  3.2.1  3.2.2  3.2.3  3.3.1  3.3.2  3.3.3  4.1.1  4.1.2  4.2.1  4.2.2  4.2.3  4.3.1  4.3.2  4.3.3 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
4 
0 
0 
0 

0 
0 
0 
0 
0 
0 
0 
4 
0 
0 
0 
8 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
2 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 

1 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
3 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
2 
8 

0 
0 
0 
0 
0 
0 
0 
8 
0 
0 
0 
18 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 

0 
8 
0 
0 
0 
0 
0 
0 
0 
0 
0 
8 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
8 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 

0 
0 
0 
0 
0 
0 
0 
1 
0 
0 
0 
9 

0 
0 
0 
3 
0 
0 
0 
0 
1 
0 
0 
0 
0 
0 
0 
0 
0 
0 
1 
0 
0 
2 
0 
0 
0 
1 
3 

0 
0 
0 
0 
0 
0 
0 
3 
0 
0 
0 
14 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 

0 
8 
0 
0 
0 
0 
0 
0 
0 
0 
0 
8 

0 
0 
0 
3 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
3 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
2 
1 
0 
5 
0 
3 
0 
0 
0 
0 
0 
0 
0 
0 
0 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
2 
13 

0 
0 
0 
0 
0 
0 
2 
0 
0 
8 
0 
0 
0 
0 
0 
0 
3 
0 
0 
1 
0 
0 
0 
3 
0 
0 
0 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
3 
20 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 

0 
0 
8 
0 
0 
0 
0 
0 
0 
0 
0 
8 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 

0 
0 
0 
0 
0 
0 
6 
0 
4 
0 
0 
10 

3 
0 
0 
2 
5 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
8 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
18 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
1 
0 
0 
2 
0 

0 
2 
0 
0 
0 
6 
0 
0 
0 
0 
0 
11 

0 
0 
0 
0 
0 
1 
0 
0 
0 
0 
0 
0 
1 
0 
0 
0 
0 
0 
0 
0 
0 
0 
7 
0 
0 
0 
0 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
9 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
4 
8 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
12 

46 

1 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
1 
0 
0 
0 
0 
0 
0 
0 
0 
0 
5 
0 
0 
0 
0 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
7 

3 
0 
0 
0 
0 
1 
0 
0 
0 
0 
0 
0 
1 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
1 
0 

0 
0 
0 
0 
0 
1 
0 
0 
0 
0 
2 
9 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 

0 
8 
0 
0 
0 
0 
0 
0 
0 
0 
0 
8 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
1 
0 
0 
0 
0 

0 
1 
0 
0 
0 
0 
0 
0 
0 
0 
0 
2 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
5 
8 

0 
0 
8 
0 
0 
0 
0 
0 
0 
0 
0 
21 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
8 
0 
8 
0 
0 
0 
0 
0 
0 
0 
0 
0 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
16 

0 
1 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
4 
0 
5 
0 
0 
0 
0 
0 
0 
0 
0 
0 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
10 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
1 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 

0 
0 
8 
0 
0 
0 
0 
0 
0 
0 
0 
9 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
8 
0 
0 
0 
0 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
8 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 

0 
8 
0 
0 
4 
6 
0 
0 
0 
0 
0 
18 

0 
0 
0 
0 
0 
0 
3 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
3 

0 
0 
0 
0 
0 
0 
3 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 

0 
0 
0 
0 
0 
0 
0 
0 
7 
0 
0 
10 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
1 
0 
0 
0 
0 
0 
0 
3 
0 
0 
0 
0 
0 
0 
0 
0 

0 
0 
0 
0 
0 
8 
0 
2 
0 
0 
0 
14 

0 
0 
0 
0 
0 
4 
0 
0 
0 
0 
0 
0 
0 
8 
0 
0 
0 
0 
0 
0 
0 
8 
0 
0 
0 
0 
0 

0 
0 
0 
0 
0 
0 
2 
0 
0 
0 
0 
22 

0 
1 
0 
4 
0 
4 
0 
0 
0 
0 
0 
0 
1 
0 
0 
0 
0 
1 
0 
0 
0 
0 
0 
0 
0 
0 
0 

2 
0 
0 
1 
4 
0 
0 
0 
0 
0 
0 
18 

 
 
 
 
Additional statistics EIS current method (z-score +/- 2 cut-off values) vs. no outlier outliers 

Ind. 

EIS 

MIN 

MAX 

RANGE 

No outlier 
treatment 

MIN 

MAX 

1.1.1  1.1.2  1.1.3  1.2.1  1.2.2  1.2.3  1.3.1  1.3.2  2.1.1  2.1.2  2.1.3  2.2.1  2.2.2  2.2.3  2.3.1  2.3.2  3.1.1  3.1.2  3.2.1  3.2.2  3.2.3  3.3.1  3.3.2  3.3.3  4.1.1  4.1.2  4.2.1  4.2.2  4.2.3  4.3.1  4.3.2  4.3.3 

0.03  22.52 

0.90 

8.74 

0.01 

0.20 

0.00 

8.00 

0.11 

0.00 

0.00 

0.05 

0.10  237.1 

5.00 

1.10 

7.71  14.89 

1.20 

2.80 

2.00 

0.00 

0.38 

0.00 

5.70  29.26  16.60  14.50 

1.42 

0.62 

0.12 

3.66 

1.60  57.00  28.10  63.62 

0.16  55.30  34.00  53.00 

1.07 

0.25 

0.30 

2.44 

1.61  9872  38.00 

6.40  44.78  54.59  24.70  27.72  11.70 

2.77 

4.46 

9.64  24.50  80.15  70.28  93.19  23.81 

3.66 

0.85  21.50 

1.57  34.48  27.20  54.88 

0.15  55.09  34.00  45.00 

0.96 

0.25 

0.30 

2.39 

1.51  9635  33.00 

5.30  37.06  39.70  23.50  24.93 

9.70 

2.77 

4.07 

9.64  18.80  50.90  53.68  78.68  22.39 

3.04 

0.73  17.84 

0.17  21.50 

0.90 

8.74 

0.01 

0.45 

0.00 

8.00 

0.11 

0.00 

0.01 

0.22 

0.10  237.1 

4.00 

1.10 

2.84 

7.57 

1.10 

2.80 

1.60 

0.00 

0.38 

0.00 

2.39  10.93 

8.16  14.50 

1.19 

0.62 

0.12 

0.00 

1.84  60.30  34.30  190.8 

0.16 

9.33 

7.87  62.00 

1.11 

0.65 

0.64 

1.98 

2.98  12524  44.00 

7.00  48.87  65.62 

6.57  78.21  12.40 

3.26 

6.66 

4.60 

5.82  80.15  70.28  94.02 

8.35 

4.55 

1.04  29.66 

RANGE 

1.68  38.80  33.40  182.1 

0.15 

8.88 

7.87  54.00 

1.00 

0.65 

0.63 

1.75 

2.88  12287  40.00 

5.90  46.04  58.05 

5.47  75.41  10.80 

3.26 

6.28 

4.60 

3.43  69.22  62.13  79.52 

7.16 

3.93 

0.92  29.66 

Change in 
RANGE 

Percentage 
change 

Change in 
MIN value 

Change in 
MAX value 

Skewness 
EIS 

Skewness 
No outlier 
treatment 

Square root 
trans-
formation 
EIS 

Square root 
trans-
formation 
No outlier 
treatment 

0.11 

4.33 

6.20  127.2 

0.00  -46.22  -26.13 

9.00 

0.04 

0.40 

0.33 

-0.64 

1.37  2653 

7.00 

0.60 

8.97  18.34  -18.03  50.49 

1.10 

0.49 

2.21 

-5.05  -15.37  18.33 

8.44 

0.83  -15.23 

0.89 

0.19  11.82 

6.7%  12.6%  22.8% 

%  0.0% -83.9% -76.8%  20.0%  4.2% 

231.8

157.1
% 

110.3

202.6

% -26.6%  90.9%  27.5%  21.2%  11.3%  24.2%  46.2% -76.7% 

%  11.3%  17.6%  54.2% -52.3% -81.7%  36.0%  15.7%  1.1% -68.0%  29.3%  25.9%  66.2% 

505.5

%  -4.6%  0.0%  0.0%  0.0% 

122.6
% 

--  0.0%  0.0% 

17992
% 

-- 

347.2

%  0.0%  0.0% -20.0%  0.0% -63.2% -49.1%  -8.7%  0.0% -20.0% 

--  0.0% 

-- -58.1% -62.6% -50.9%  0.0% -16.1%  0.0%  0.0%  -100% 

15.2%  5.8%  22.1% 

%  0.0% -83.1% -76.8%  17.0%  3.7% 

199.9

157.1
% 

112.1

182.1

% -18.9%  85.1%  26.9%  15.8%  9.4%  9.1%  20.2% -73.4% 

%  6.0%  17.6%  49.5% -52.3% -76.2%  0.0%  0.0%  0.9% -64.9%  24.4%  22.3%  37.9% 

0.392  -0.044  0.810  1.166  0.073  0.809  0.826  0.146  0.188  0.881  0.852  0.695  1.651  0.536  0.098  0.227  0.033  -0.320  0.611  1.357  0.101  1.030  1.507  0.743  0.378  -0.299  -0.524  0.144  0.789  0.783  1.459  0.325 

1.026  -0.027  0.948  5.836  0.073  1.549  1.614  0.264  0.195  1.553  1.124  1.265  5.619  0.844  0.232  0.342  -0.028  -0.359  1.015  5.753  0.106  1.314  2.983  2.102  1.270  -0.593  -0.662  0.145  4.239  0.983  1.845  0.454 

No 

No 

No 

Yes 

No 

No 

No 

No 

No 

No 

No 

No 

Yes 

No 

No 

No 

No 

No 

No 

Yes 

No 

Yes 

Yes 

No 

No 

No 

No 

No 

No 

No 

Yes 

No 

Yes 

No 

No 

Yes 

No 

Yes 

Yes 

No 

No 

Yes 

Yes 

Yes 

Yes 

No 

No 

No 

No 

No 

Yes 

Yes 

No 

Yes 

Yes 

Yes 

Yes 

No 

No 

No 

Yes 

No 

Yes 

No 

47 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Change in normalised score EIS current method (z-score +/- 2 cut-off values) vs. no outlier (No outlier - current method) 

Changes bigger than 25% are highlighted in green, changes smaller than 25% are highlighted in red. 

Belgium 
Bulgaria 
Czechia 
Denmark 
Germany 
Estonia 
Ireland 
Greece 
Spain 
France 
Croatia 
Italy 
Cyprus 
Latvia 
Lithuania 
Luxembourg 
Hungary 
Malta 
Netherlands 
Austria 
Poland 
Portugal 
Romania 
Slovenia 
Slovakia 
Finland 
Sweden 
Bosnia and 
Herzegovina 
Iceland 
Israel 
North Macedonia 
Montenegro 
Norway 
Serbia 
Switzerland 
Turkey 
Ukraine 
United Kingdom 

-1% 

-6% 

-4% 

-3% 

-14% 

-15% 

3% 
-3% 
-7% 
0% 
0% 
-8% 

-11% 
-16% 
-19% 
-18% 
-10% 
-19% 

1.1.1  1.1.2  1.1.3  1.2.1  1.2.2  1.2.3  1.3.1  1.3.2  2.1.1  2.1.2  2.1.3  2.2.1  2.2.2  2.2.3  2.3.1  2.3.2  3.1.1  3.1.2  3.2.1  3.2.2  3.2.3  3.3.1  3.3.2  3.3.3  4.1.1  4.1.2  4.2.1  4.2.2  4.2.3  4.3.1  4.3.2  4.3.3 
-7% 
-16% 
-2% 
17% 
-2% 
-16% 
-7% 
-26% 
-3% 
-26% 
-7% 
-6% 
-8% 
-20% 
-7% 
17% 
-7% 
-28% 
-8% 
-16% 
-4% 
17% 
6% 
0% 
0% 
31% 
-7% 
53% 
-8% 
17% 
-8% 
-20% 
0% 
31% 
-6% 
53% 
-8% 
-6% 
-6% 
-16% 
-7% 
53% 
-12% 
-5% 
53%  20% 
-7% 
-12% 
-6% 
-6% 
-6% 
-23% 
-8% 
-23% 

-26% 
-10% 
-10%  20%  84%  36% 
-5% 
-10% 
-11% 
-10% 
-14% 
-10% 
-24% 
-10% 
-25% 
-10% 
-10% 
-24% 
-10%  45%  31%  15% 
-10% 
-10% 
-8% 
-10% 
-10% 
-12% 
0% 
-10% 
-10%  25%  18%  19% 
-10% 
-14% 
-2% 
-10% 
-8% 
-10% 
-11% 
-1% 
1% 
-10%  14%  90% 
8% 
-11% 
-2% 
-10% 
-12% 
-11% 
1% 
-10% 
-10% 
-16% 
-19% 
-4% 
-10%  68%  179%  36% 
-2%  11% 
-10% 
1% 
0%  57% 
-10%  220% 
-10% 
-8% 
-5% 
-10% 
8% 
-10%  43%  49% 
-24% 
-16% 
-5% 
-14% 
-16% 
-8% 

-18% 
-4% 
-4%  191%  357%  40% 
-4%  120%  29% 
3% 
-13% 
-4% 
-8%  83% 
-20% 
-4%  15%  80% 
-4% 
-16%  150%  18% 
-6%  20%  45% 
-4% 
-4%  92%  126%  37% 
2%  63%  28% 
-4% 
-4% 
-4% 
-1% 
-4%  22%  473%  42% 
-4%  45% 
-2%  16% 
0%  608%  86% 
-4% 
-4%  182%  395%  114% 
5%  180%  55% 
-4% 
-22%  115%  36% 
-4% 
-4%  21% 
7% 
-5% 
-4%  380%  114%  64% 
-12%  16% 
-4% 
-5% 
-4%  76% 
-21% 
-8% 
-4%  71%  33%  20% 
-4%  21% 
3%  26% 
-4%  22%  220%  81% 
-6% 
-4%  392% 
-4%  169%  133%  52% 
-14% 
-23%  93% 
-4% 
-24% 
-9%  32% 
-4% 

8% 
5% 
-3%  108%  25% 
1% 
-9%  29% 
7% 
-14%  16% 
3% 
-13% 
1% 
0%  16% 
-11% 
3% 
-21%  55% 
-7%  11%  44% 
-7%  118%  11% 
4% 
-6%  22%  17% 
7% 
7% 
5% 
5% 
-4%  88%  29% 
-4%  18%  17% 
9% 
-22%  37% 
0% 
-7%  227% 
6% 
-18%  24% 
8% 
-17%  24% 
4% 
8% 
-12% 
-3%  542% 
9% 
-4%  49%  14% 
4% 
6% 
3% 
0% 
7%  11% 
5% 
8% 

-22% 
-10% 
0% 
0%  96%  11% 
0%  31%  11% 
0% 
0% 
-4% 
-6% 
0%  59% 
0%  35% 
5% 
-8% 
4% 
0% 
0%  293%  53% 
-21% 
0%  35% 
0% 
-6%  20% 
0%  78%  25% 
0%  39%  20% 
5% 
0%  44% 
0%  71% 
5% 
-26% 
0%  103% 
-26% 
0% 
-1% 
0%  28% 
-6% 
-20% 
0%  29% 
-25% 
-12% 
0% 
0% 
5% 
0% 
-12% 
0%  215% 
-19% 
0% 
6% 
-10% 
0%  138% 
-8% 
0%  73% 
0%  74%  11% 
-21% 
0%  19% 
-2% 
-3% 
0% 

-15% 
-22%  24% 
-13% 
-22% 
-14% 
-22% 
-13% 
-6% 
-11% 
-22% 
-14% 
-22% 
-6% 
-22% 
-12% 
-22% 
-9% 
-22% 
-13% 
-22% 
-9% 
-22% 
-13% 
-22% 
-11% 
-22% 
-8% 
-22% 
-12% 
-22% 
-10% 
-22% 
-14% 
-22% 
-13% 
-22% 
-11% 
-22% 
-11% 
-22% 
-22% 
-13% 
-22%  65% 
-14% 
-22% 
-10% 
-22% 
-15% 
-22% 
-14% 
-5% 

-35%  31% 
-35% 
0% 
-35%  38% 
-25% 
-35% 
-8% 
-35% 
-35% 
-1% 
-35%  63% 
-35%  141% 
-35%  35% 
-35%  37% 
-35%  161% 
-35% 
-9% 
-35%  35% 
-35%  51% 
-35%  75% 
-33% 
-13% 
-35%  118% 
-5%  11% 
-2% 
-35% 
-16% 
-35% 
-35% 
-8% 
-35%  26% 
-35%  125% 
-35%  46% 
-35%  81% 
-2% 
-35% 
3% 
-35% 

-3% 
-67% 
-67%  35% 
3% 
-67% 
-6% 
-67% 
-6% 
-67% 
-67% 
-6% 
-67%   
1% 
-67% 
-4% 
-67% 
-5% 
-67% 
-2% 
-67% 
5% 
-67% 
-6% 
-67% 
1% 
-67% 
-6% 
-67% 
-6% 
-67% 
-2% 
-67% 
-5% 
-67% 
-6% 
-67% 
-5% 
-67% 
-4% 
-67% 
-5% 
-67% 
0% 
-67% 
-2% 
-67% 
-67%  17% 
-5% 
-67% 
-2% 
-67% 

-2% 
-21% 
-27% 
-21% 
-6% 
-21% 
-26% 
-21% 
-12% 
-21% 
-21% 
-24% 
-21%  58% 
-10% 
-21% 
-10% 
-21% 
-11% 
-21% 
-2% 
-21% 
-21% 
-3% 
0% 
-21% 
-11%  109% 
-21% 
-21% 
-6% 
-21% 
5% 
-21% 
-23% 
-21% 
2% 
-21% 
-8% 
-21% 
-12% 
-21% 
-12% 
-8% 
-21%  110% 
-1% 
-21% 
-22% 
-21% 
-13% 
-21% 
-13% 
-21% 

-17% 
-17% 
-17% 
-17% 
-17% 
-17% 
-17% 
-17% 
-17% 
-17% 
-17% 
-17% 
-17% 
-17% 
-17% 
-17% 
-17% 
-17% 
-17% 
-17% 
-17% 
-17% 
-17% 
-17% 
-17% 
-17% 
-17% 

-39% 
-16% 
-34% 
0% 
-38% 
-35% 
-30% 
-48% 
-40% 
-25% 
-32% 
-41% 
-32% 
-23% 
-27% 
-16% 
-25% 
-27% 
-23% 
-38% 
-16% 
-33% 
-25% 
-33% 
-31% 
-37% 
-36% 

-15% 
-15% 
-15% 
-15% 
-15% 
-15% 
-15% 
-15% 
-15% 
-15% 
-15% 
-15% 
-15% 
-15% 
-15% 
-15% 
-15% 
-15% 
-15% 
-15% 
-15% 
-15% 
-15% 
-15% 
-15% 
-15% 
-8% 

-48% 
-48% 
-48% 
-48% 
-48% 
-48% 
-48% 
-48% 
-48% 
-48% 
-48% 
-48% 
-48% 
-48% 
-48% 
-48% 
-48% 
-48% 
-48% 
-48% 
-48% 
-48% 
-48% 
-48% 
-48% 
-48% 
-48% 

-19% 
-19% 
-19% 
-19% 
-19% 
-19% 
-19% 
-19% 
-19% 
-19% 
-19% 
-19% 
-19% 
-19% 
-19% 
-19% 
-19% 
-19% 
-19% 
-19% 
-19% 
-19% 
-19% 
-19% 
-19% 
-16% 
0% 

-70% 
-70% 
-70% 
-70% 
-70% 
-70% 
-70% 
-70% 
-70% 
-70% 
-70% 
-70% 
-70% 
-70% 
-70% 
-70% 
-70% 
-70% 
-70% 
-70% 
-70% 
-70% 
-70% 
-70% 
-70% 
-70% 
-70% 

-23% 
-23% 
-23% 
-23% 
-23% 
-23% 
-23% 
-23% 
-23% 
-23% 
-23% 
-21% 
-23% 
-23% 
-23% 
-23% 
-23% 
-23% 
0% 
-23% 
-23% 
-23% 
-23% 
-23% 
-23% 
-23% 
-23% 

-1% 
-1% 
-1% 
-1% 
-1% 
-1% 
-1% 
-1% 
-1% 
-1% 
-1% 
-1% 
-1% 
-1% 
-1% 
-1% 
-1% 
-1% 
-1% 
-1% 
-1% 
-1% 
-1% 
-1% 
-1% 
-1% 
-1% 

-12%  27% 
-4%  128% 

-11% 
-15% 
-19% 
0% 

-3% 
-6% 
-6% 
-1% 

-16% 
-19% 

-11% 
-16% 

-13%  19% 

-3% 
0% 

-21% 

8% 

-- 

74%  46% 
-8% 
17% 
-8%   
-3% 
-4% 
95% 
-6% 
127% 
-8% 
-12% 
-3% 
7% 
-8% 
-30% 
-3% 
53% 
41%   
-34% 

-7% 

-19% 
-19% 

-19% 
-19% 
-19% 
-19% 
-6% 
-19% 

-19% 

-70% 
0% 
-70% 
-70% 
-70% 
-70% 
-70% 
-70% 
-70% 
-70% 
-70% 

0% 

37% 
-21% 

0%   
0% 
0%   
-5%  30% 
0% 
5% 
0%  40% 
0%  23% 
2% 
0%  96%  11% 
0% 
-16% 
-21% 
-3% 
0%  85% 
0%  100%  50%   
2% 
0% 

-10% 

0% 
0% 

-17% 
-17% 
-17% 
-17% 
-17% 
-17% 

-17% 

-8% 
38% 

582%  179% 
-4%   
5% 
-4%   
0%   
-4%   
493%  152% 
-4%   
1247%  107% 
-4%   
-4%  35% 
7% 
-4%  294%  199%  67% 
-4% 
4%   
-18%  201% 
19%  30% 
-4%   
-4%  109%  118%  82% 
-9% 
-4% 

-10% 

-14% 

-1% 

0% 
-48% 

0% 
-22% 

-9%   
-15% 

-10% 

-7% 
0% 

-6%   
-13% 

-26% 

-48% 
-48% 
-48% 
-35% 

-22% 
-22% 
-22% 
-22% 

-6% 
-14% 
-13% 
-10% 

-48% 
-48%   
-48% 

-22% 

-22% 

-4% 
4%   
-14% 

-10%  31% 
-8% 
-10% 
-9% 
-10% 
-8% 
-10% 
-10% 
-4% 
-10%  12% 

-13% 
-18% 
-13% 
-18% 

-5%  20% 
4% 
-23% 
-6% 
2% 
6%  23% 
79% 
-28% 

3%  146% 

-10% 

48 

-6% 

-67%   
0% 
-67%   
-67% 
-2% 
-67%  12% 
-6% 
-67% 
-1% 
-67% 
-6% 
-67% 
-5% 
-67% 
-67%   
-67% 

-6% 

-15% 
-15% 
-8% 
-15% 
0% 
-15% 
0% 
-15% 
-15% 
-15% 
-15% 

-13%  10% 

37%  74% 
-35% 
0%   
0% 
-35%  411% 
2% 
-1%   
-35%  102% 
2% 
9%  84% 
-35% 1413% 
7%  359% 
-6% 
-35% 
0% 
6% 
-23% 
-35%  151% 
7%  13% 
-1% 
-35%  340% 
-35% 
6% 
1% 
-19% 
-18% 
-35%  565%  11%  99%  10% 
58% 
-35%  804% 
-8%   
6% 
-35%  32% 

-20%  10% 

-- 

-1% 
-1% 
-1%   
-1% 
-1% 
-1% 
-1% 
-1% 
-1% 
-1% 
-1% 

-26% 
-12% 

-23%   
-23% 

2% 
-19%   
-22% 
-32% 
-37% 
-28% 
4%   
-39% 

-23%   

-2% 
-23% 
-18% 
-23%   

-23% 

-21% 

13% 
-2% 
27% 
37% 
-- 
-1% 
-19% 
0%  33% 
8% 
10% 
-10% 

-21% 
-16% 

-21% 

 
 
 
 
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
 
 
 
Annex A.2: Box-plot method  

Number of outliers identified by country by indicator Box-plot method vs. EIS current method (z-score +/- 2 cut-off values) 

Green highlighted cells show the number of more identified and treated outlier using the Box-plot method vs. the current EIS outlier method (z-score +/- 2 cut-
off values), red highlighted cells show the number of less identified and treated outliers using the Box-plot method vs. the current EIS outlier method (z-score 
+/- 2 cut-off values). 

Country/Ind. 
Belgium 
Bulgaria 
Czechia 
Denmark 
Germany 
Estonia 
Ireland 
Greece 
Spain 
France 
Croatia 
Italy 
Cyprus 
Latvia 
Lithuania 
Luxembourg 
Hungary 
Malta 
Netherlands 
Austria 
Poland 
Portugal 
Romania 
Slovenia 
Slovakia 
Finland 
Sweden 
Bosnia and 
Herzegovina 
Iceland 
Israel 
North Macedonia 
Montenegro 
Norway 
Serbia 
Switzerland 
Turkey 
Ukraine 
United Kingdom 
Total 

1.1.1  1.1.2  1.1.3  1.2.1  1.2.2  1.2.3  1.3.1  1.3.2  2.1.1  2.1.2  2.1.3  2.2.1  2.2.2  2.2.3  2.3.1  2.3.2  3.1.1  3.1.2  3.2.1  3.2.2  3.2.3  3.3.1  3.3.2  3.3.3  4.1.1  4.1.2  4.2.1  4.2.2  4.2.3  4.3.1  4.3.2  4.3.3 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
-3 
0 
0 
0 

0 
0 
0 
0 
0 
0 
0 
-4 
0 
0 
0 
-7 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
-2 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 

-1 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
-3 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
-2 
-8 

0 
0 
0 
0 
0 
0 
0 
-8 
0 
0 
0 
-18 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 

0 
0 
0 
0 
0 
0 
0 
3 
0 
0 
0 
3 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 

0 
0 
0 
0 
0 
0 
0 
-1 
0 
0 
0 
-1 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 

0 
-7 
0 
0 
0 
0 
0 
0 
0 
0 
0 
-7 

0 
0 
0 
-3 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
-3 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
-1 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
-1 

0 
0 
0 
0 
0 
0 
-2 
0 
0 
0 
0 
0 
0 
0 
0 
0 
-3 
0 
0 
-1 
0 
0 
0 
-3 
0 
0 
0 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
-3 
-12 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 

0 
0 
0 
0 
0 
3 
0 
0 
0 
0 
0 
0 
0 
0 
4 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 

0 
0 
0 
0 
0 
0 
0 
0 
3 
0 
0 
10 

-3 
0 
0 
-2 
-5 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
-6 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
-16 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
-1 
0 
0 
-2 
0 

0 
-2 
0 
0 
0 
-6 
0 
0 
0 
0 
0 
-11 

0 
0 
0 
0 
0 
-1 
0 
0 
0 
0 
0 
0 
-1 
0 
0 
0 
0 
0 
0 
0 
0 
0 
-7 
0 
0 
0 
0 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
-9 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
-4 
-8 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
-12 

49 

-1 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
-1 

-1 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
-1 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 

0 
0 
0 
0 
0 
0 
0 
7 
0 
0 
0 
7 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
-1 
0 
0 
0 
0 

0 
-1 
0 
0 
0 
0 
0 
0 
0 
0 
0 
-2 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 

0 
0 
0 
0 
0 
3 
0 
0 
0 
0 
0 
0 
1 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 

0 
1 
0 
0 
0 
0 
0 
0 
0 
0 
0 
5 

0 
-1 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
-1 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
-1 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
-1 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
-6 
0 
0 
0 
0 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
-6 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 

0 
-8 
0 
0 
-4 
-6 
0 
0 
0 
0 
0 
-18 

0 
0 
0 
0 
0 
0 
-3 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
-3 

0 
0 
0 
0 
0 
0 
0 
1 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
1 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
-1 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 

0 
0 
0 
0 
0 
0 
0 
-2 
0 
0 
0 
-3 

0 
0 
0 
0 
0 
3 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 

0 
0 
0 
0 
0 
0 
6 
0 
0 
0 
0 
9 

0 
-1 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
-1 

 
 
 
 
 
Additional statistics Box-plot method vs. EIS current method (z-score +/- 2 cut-off values) 

Ind. 

EIS 

MIN 

MAX 

RANGE 

Box-plot 
method 

MIN 

MAX 

1.1.1  1.1.2  1.1.3  1.2.1  1.2.2  1.2.3  1.3.1  1.3.2  2.1.1  2.1.2  2.1.3  2.2.1  2.2.2  2.2.3  2.3.1  2.3.2  3.1.1  3.1.2  3.2.1  3.2.2  3.2.3  3.3.1  3.3.2  3.3.3  4.1.1  4.1.2  4.2.1  4.2.2  4.2.3  4.3.1  4.3.2  4.3.3 

0.03  22.52 

0.9 

9  0.012  0.202 

1.60  57.00 

28.1 

64  0.161 55.295 

0 

34 

8 

0.11 

0.00 

0.00 

0.05 

0.10 

237 

5 

1.10 

7.71  14.89 

1.20 

2.80 

2.00 

0.00 

0.38 

0.00 

5.70  29.26  16.60  14.50 

1.42 

0.62 

0.12 

3.66 

53 

1.07 

0.25 

0.30 

2.44 

1.61  9872 

38 

6.40  44.78  54.59  24.70  27.72  11.70 

2.77 

4.46 

9.64  24.50  80.15  70.28  93.19  23.81 

3.66 

0.85  21.50 

1.57  34.48  27.20  54.88 

0.15  55.09  34.00  45.00 

0.96 

0.25 

0.30 

2.39 

1.51  9635  33.00 

5.30  37.06  39.70  23.50  24.93 

9.70 

2.77 

4.07 

9.64  18.80  50.90  53.68  78.68  22.39 

3.04 

0.73  17.84 

0.03  21.50 

0.9 

76  0.012  0.202 

1.80  60.30 

34.3  3538  0.161 55.940 

0 

34 

8 

0.11 

0.00 

0.01 

0.05 

0.01 

237 

4 

1.10 

2.84  14.89 

1.20 

2.80 

1.60 

0.00 

0.15 

0.00 

5.70  14.44 

8.16  14.50 

1.42 

0.62 

0.01 

3.66 

58 

1.11 

0.26 

0.60 

2.44 

1.79  11955 

44 

7.00  48.87  56.82  26.38  24.78  12.40 

2.77  13.93  10.49  25.70  80.15  70.28  94.02  20.81 

3.84 

0.48  21.74 

RANGE 

1.77  38.80  33.40  3462 

0.15  55.74  34.00  50.00 

1.00 

0.26 

0.59 

2.39 

1.78  11718  40.00 

5.90  46.04  41.94  25.18  21.98  10.80 

2.77  13.78  10.49  20.00  65.71  62.13  79.52  19.38 

3.22 

0.47  18.08 

Change in 
RANGE 

Percentage 
change 

Change in MIN 
value 

Change in MAX 
value 

Skewness 
EIS 

Skewness 
Box-plot 
method 

Square root 
transformation 
EIS 

Square root 
transformation 
Box-plot 
method 

0.20 

4.33 

6.20  3407 

0.00 

0.65 

0.00 

5.00 

0.04 

0.01 

0.29 

0.00 

0.27  2083 

7.00 

0.60 

8.97 

2.24 

1.68 

-2.94 

1.10 

0.00 

9.71 

0.85 

1.20  14.81 

8.44 

0.83 

-3.01 

0.19 

-0.27 

0.23 

12.7%  12.6%  22.8% 6208%  0.0%  1.2%  0.0%  11.1%  4.2%  2.2%  96.9%  0.0%  18.0%  21.6%  21.2%  11.3%  24.2%  5.6%  7.1% -11.8%  11.3%  0.0% 

%  8.8%  6.4%  29.1%  15.7%  1.1% 

13.4%  6.1% 

36.5%  1.3% 

238.2

-

-

0.0%  -4.6%  0.0% 

%  0.0%  0.0% 

--  0.0%  0.0% 

-- 

%  0.0% 

89.7%  0.0% 

20.0%  0.0% 

63.2%  0.0%  0.0%  0.0% 

773.6

17992

-

-

-

-
20.0% 

-
61.6% 

-- 

212.4

--  0.0% -50.6% 

50.9%  0.0%  0.0%  0.0% 

88.2%  0.0% 

-

-

-

-

12.5%  5.8%  22.1% 5462%  0.0%  1.2%  0.0%  9.4%  3.7%  2.2%  98.7%  0.0%  11.1%  21.1%  15.8%  9.4%  9.1%  4.1%  6.8% -10.6%  6.0%  0.0% 

%  8.8%  4.9%  0.0%  0.0%  0.9% 

12.6%  5.1% 

43.7%  1.1% 

0.392  -0.044  0.810  1.166  0.073  0.809  0.826  0.146  0.188  0.881  0.852  0.695  1.651  0.536  0.098  0.227  0.033  -0.320  0.611  1.357  0.101  1.030  1.507  0.743  0.378  -0.299  -0.524  0.144  0.789  0.783  1.459  0.325 

0.477  -0.027  0.948  0.969  0.073  0.820  0.826  0.249  0.195  0.912  1.037  0.695  0.863  0.827  0.232  0.342  -0.028  -0.302  0.679  1.016  0.106  1.030  0.660  0.864  0.479  -0.559  -0.662  0.145  0.547  0.844  0.923  0.346 

No 

No 

No 

Yes 

No 

No 

No 

No 

No 

No 

No 

No 

Yes 

No 

No 

No 

No 

No 

No 

Yes 

No 

Yes 

Yes 

No 

No 

No 

No 

No 

No 

No 

Yes 

No 

No 

No 

No 

No 

No 

No 

No 

No 

No 

No 

Yes 

No 

No 

No 

No 

No 

No 

No 

No 

Yes 

No 

Yes 

No 

No 

No 

No 

No 

No 

No 

No 

No 

No 

50 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Change in normalised score Box-plot method vs. EIS current method (z-score +/- 2 cut-off values) (Box-plot - current method) 

Changes bigger than 25% are highlighted in green, changes smaller than 25% are highlighted in red. 

Belgium 
Bulgaria 
Czechia 
Denmark 
Germany 
Estonia 
Ireland 
Greece 
Spain 
France 
Croatia 
Italy 
Cyprus 
Latvia 
Lithuania 
Luxembourg 
Hungary 
Malta 
Netherlands 
Austria 

1.1.1  1.1.2  1.1.3  1.2.1  1.2.2  1.2.3  1.3.1  1.3.2  2.1.1  2.1.2  2.1.3  2.2.1  2.2.2  2.2.3  2.3.1  2.3.2  3.1.1  3.1.2  3.2.1  3.2.2  3.2.3  3.3.1  3.3.2  3.3.3  4.1.1  4.1.2  4.2.1  4.2.2  4.2.3  4.3.1  4.3.2  4.3.3 
-11.3%  -7.5%  2.6%  2.8%  0.0%  17.6%  9.7% -10.0%  -4.0%  9.0%  14.7%  8.1%  4.0%  0.0% -14.6% -10.2%  -0.7%  0.0%  0.0%  5.8%  -3.1%  -3.0% -14.2%  0.0%  0.0%  1.8%  0.0%  -1.0%  15.5%  7.6% -25.5%  3.2% 
-11.3%  -2.2%  2.6% -47.3%  0.0%  17.6%  9.7% -10.0%  -4.0%  9.0%  14.7%  8.1% -29.1%  7.8%  23.8% -10.2%  19.9%  -5.3%  0.0% -52.7%  34.7% -60.3%  -7.3%  0.0%  0.0%  41.7%  0.0%  -1.0%  15.5%  12.7%  25.4%  3.2% 
-11.3%  -2.1%  2.6% -17.8%  0.0%  17.6%  9.7% -10.0%  -4.0%  9.0%  14.7%  8.1%  13.9%  7.8% -13.4% -10.2%  2.9%  -5.3%  0.0% -24.3%  3.1% -53.0% -27.6%  0.0%  0.0%  11.3%  0.0%  -1.0%  15.5%  12.7% -39.3%  3.2% 
-11.3%  -7.2%  2.6%  15.9%  0.0%  17.6%  0.0% -10.0%  -4.0%  9.0%  14.7%  8.1% -16.6%  7.8% -14.2% -10.2%  -3.0%  -5.3%  0.0%  13.4%  -6.1%  14.7%  -0.5%  0.0%  0.0%  6.2%  0.0%  -1.0%  0.0%  12.7% -52.0%  3.2% 
-11.3%  -2.7%  2.6% -18.9%  0.0%  17.6%  9.7% -10.0%  -4.0%  9.0%  14.7%  8.1%  34.3%  0.0% -13.2% -10.2%  -6.8%  -5.3%  0.0%  -9.5%  -5.6%  11.6%  -8.4%  0.0%  0.0%  1.1%  0.0%  -1.0%  15.5%  12.7% -49.4%  3.2% 
-11.3%  -6.7%  2.6%  1.3%  0.0%  17.6%  9.7% -10.0%  -4.0%  9.0%  14.7%  8.1%  5.2%  7.8% -10.6% -10.2%  0.0%  -5.3%  0.0%  -7.2%  -6.0% -30.4%  2.7%  0.0%  0.0%  0.1%  0.0%  -1.0%  15.5%  12.7%  33.9%  3.2% 
-27.6% -30.4%  0.0%  0.0%  21.1%  0.0%  -0.7%  15.5%  12.7% -48.7%  3.2% 
-11.3%  -8.4%  2.6%  2.0%  0.0%  17.6%  9.7% -10.0%  -4.0%  9.0%  14.7%  8.1% -17.4%  0.0% -13.8% -10.2%  -0.1%  -5.3%  0.0%  1.1%   
-11.3%  -6.6%  2.6% -22.7%  0.0%  17.6%  9.7% -10.0%  -4.0%  9.0%  14.7%  8.1%  12.0%  7.8%  -5.7% -10.2%  -8.0%  -5.3%  0.0% -24.5%  0.7% -58.2% -22.4%  0.0%  0.0%  4.2%  0.0%  -1.0%  0.0%  12.7%  21.8%  3.2% 
-11.3%  -7.3%  2.6% -20.8%  0.0%  17.6%  0.0% -10.0%  -4.0%  9.0%  14.7%  8.1% -10.9%  7.8% -12.0% -10.2%  45.1%  -5.3%  0.0% -26.6%  -4.0% -36.9%  -7.3%  0.0%  0.0%  45.9%  0.0%  -1.0%  15.5%  12.7% -15.4%  3.2% 
-11.3%  -7.6%  2.6% -22.1%  0.0%  17.6%  9.7% -10.0%  -4.0%  3.9%  0.0%  8.1% -19.9%  7.8%  -9.3% -10.2%  -3.2%  -5.3%  0.0% -24.5%  -4.7%  5.0% -36.8%  0.0%  0.0%  7.5%  0.0%  -1.0%  15.5%  12.7% -27.5%  3.2% 
-11.3%  -4.1%  2.6% -24.1%  0.0%  17.6%  9.7% -10.0%  -4.0%  9.0%  14.7%  8.1%  12.2%  7.8% -12.9% -10.2%  -6.4%  -5.3%  0.0% -24.4%  -2.0% -62.8% -38.7%  0.0%  0.0%  8.5%  0.0%  -1.0%  15.5%  12.7%  10.5%  3.2% 
-11.3%  6.5%  2.6% -19.2%  0.0%  17.6%  9.7% -10.0%  -4.0%  9.0%  14.7%  8.1%  12.2%  7.8%  -9.3% -10.2%  -5.6%  -5.3%  0.0% -21.9%  4.8% -20.8%  -9.0%  0.0%  0.0%  2.9%  0.0%  -1.0%  15.5%  0.0% -26.9%  3.2% 
-11.3%  0.0%  2.6%  19.4%  0.0%  17.6%  9.7% -10.0%  -4.0%  0.0%  14.7%  8.1%  13.9%  7.8% -13.4% -10.2%  -1.4%  0.0%  0.0%  4.3%  -5.8% -59.5% -34.2%  0.0%  0.0%  2.1%  0.0%  -1.0%  15.5%  12.7%  27.8%  0.0% 
-11.3%  -6.9%  2.6% -30.7%  0.0%  17.6%  9.7% -10.0%  -4.0%  9.0%  14.7%  8.1% -19.6%  7.8% -10.6% -10.2%  24.7%  -5.3%  0.0% -38.9%  1.4% -51.1% -12.1%  0.0%  0.0%  34.0%  0.0%  -1.0%  15.5%  12.7%  0.0%  3.2% 
-11.3%  -8.4%  2.6% -25.0%  0.0%  17.6%  0.0% -10.0%  -4.0%  9.0%  14.7%  8.1%  35.4%  7.8%  -8.3% -10.2%  -2.2%  -5.3%  0.0% -39.4%  -6.0% -60.2%  -5.0%  0.0%  0.0%  6.9%  0.0%  -1.0%  15.5%  12.7% -37.5%  3.2% 
-11.3%  -8.4%  2.6%  22.2%  0.0%  0.0%  0.0% -10.0%  -4.0%  0.0%  14.7%  8.1% -37.3%  7.8% -12.3% -10.2%  -0.8%  -5.3%  0.0%  11.4%  -5.7% -28.3%  0.0%  0.0%  0.0%  14.4%  0.0%  -1.0%  15.5%  4.3% -10.8%  3.2% 
-11.3%  0.1%  2.6% -32.4%  0.0%  17.6%  9.7% -10.0%  -4.0%  9.0%  14.7%  8.1% -18.4%  7.8% -10.0% -10.2%  13.7%  -5.3%  0.0% -30.3%  -1.8% -41.3% -33.8%  0.0%  0.0%  87.8%  0.0%  -1.0%  15.5%  12.7% -16.9%  3.2% 
-11.3%  -6.0%  2.6% -16.0%  0.0%  17.6%  9.7% -10.0%  -4.0%  9.0%  14.7%  8.1%  2.0%  7.8% -13.9% -10.2%  -2.2%  -5.3%  0.0% -33.8%  -5.0% -38.1%  0.0%  0.0%  0.0%  9.3%  0.0%  -1.0%  15.5%  12.7% -43.3%  0.0% 
-11.3%  -7.7%  2.6%  12.8%  0.0%  17.6%  0.0% -10.0%  -4.0%  9.0%  14.7%  8.1% -47.8%  7.8% -13.2% -10.2%  1.1%  -5.3%  0.0%  15.7%  -5.6%  21.9%  -6.0%  0.0%  0.0%  9.2%  0.0%  -1.0%  15.5%  0.0% -32.6%  3.2% 
-11.3%  -6.4%  2.6%  7.3%  0.0%  17.6%  9.7% -10.0%  -4.0%  9.0%  14.7%  8.1% -28.8%  7.8% -11.2% -10.2%  -3.9%  -5.3%  0.0%  20.1%  -4.6%  20.8%  7.7%  0.0%  0.0%  3.0%  0.0%  -1.0%  15.5%  12.7% -45.7%  3.2% 

Poland 
Portugal 

-11.3%  -6.8%  2.6% -38.7%  0.0%  17.6%  9.7% -10.0%  -4.0%  9.0%  14.7%  8.1%  -6.8%  7.8% -11.2% -10.2%  68.4%  -5.3%  0.0% -45.4%  -3.9% -62.7% -21.8%  0.0%  0.0% 
%  0.0%  -1.0%  15.5%  12.7%  34.9%  3.2% 
-11.3%  -5.0%  2.6%  -5.0%  0.0%  17.6%  0.0% -10.0%  -4.0%  9.0%  14.7%  8.1% -29.3%  7.8% -12.9% -10.2%  0.7%  -5.3%  0.0% -23.9%  -5.1% -48.5% -10.0%  0.0%  0.0%  18.9%  0.0%  -1.0%  15.5%  12.7%  0.0%  3.2% 

210.1

Romania 
Slovenia 
Slovakia 
Finland 
Sweden 
Bosnia and 
Herzegovina 
Iceland 
Israel 
North Macedonia 
Montenegro 
Norway 
Serbia 
Switzerland 
Turkey 
Ukraine 
United Kingdom 

0.0%  -1.0%  15.5%  12.7%  16.7%  3.2% 
-11.3%  19.5%  2.6% -47.4%  0.0%  17.6%  9.7% -10.0%  -4.0%  9.0%  14.7%  8.1% -56.4%  7.8%  65.0% -10.2% 
-11.3%  -6.9%  2.6%  -0.3%  0.0%  17.6%  9.7% -10.0%  -4.0%  9.0%  14.7%  8.1% -51.5%  7.8% -13.6% -10.2%  -5.2%  -5.3%  0.0%  2.4%  -2.4% -13.7%  -3.1%  0.0%  0.0%  10.3%  0.0%  -1.0%  15.5%  12.7%  -1.6%  3.2% 
-11.3%  -5.7%  2.6% -26.6%  0.0%  17.6%  9.7% -10.0%  -4.0%  9.0%  14.7%  8.1%  3.3%  7.8% -10.0% -10.2%  43.0%  -5.3%  0.0% -40.3%  17.5% -59.2% -29.8%  0.0%  0.0%  49.8%  0.0%  -1.0%  15.5%  12.7% -26.2%  3.2% 
-11.3%  -6.5%  0.0%  16.5%  0.0%  17.6%  0.0% -10.0%  -4.0%  0.0%  14.7%  8.1% -12.6%  7.8% -15.0%  -3.4%  -5.2%  -5.3%  0.0%  14.2%  -4.7%  0.9%  -1.4%  0.0%  0.0%  2.8%  0.0%  -1.0%  15.5%  12.7% -17.3%  3.2% 
-11.3%  -7.6%  0.0%  21.9%  0.0%  17.6%  0.0% -10.0%  -4.0%  9.0%  14.7%  0.0% -13.6%  0.0% -14.4%  0.0%  -8.0%  -5.3%  0.0%  24.1%  -2.0%  0.0%  1.3%  0.0%  0.0%  3.0%  0.0%  -1.0%  15.5%  12.7% -28.4%  3.2% 

%  0.0%  0.0% -52.7%  0.0% -75.9% -41.7%  0.0%  0.0%  0.0% 

220.3

-12.6%  0.0%   

3.2% 
14.7%  8.1%  0.0%  0.0%  -9.3%   
9.7%  0.0%  -4.0%   
-11.3%  46.5%  2.6% -24.1%  0.0%   
14.7%  8.1% -12.8%  7.8% -14.6% -10.2%  0.1%  -5.3%  0.0%  0.0%  -5.7%  1.0% -34.2%  0.0%  0.0%  3.8%  0.0%  -1.0%  15.5%  12.7%  24.5%  3.2% 
-11.3%  -7.6%  2.6%  0.0%  0.0%  17.6%  9.7%  0.0%  -4.0%   
3.2% 
14.7%  0.0%   
-11.3%  -7.6%   
-4.0%   
3.2% 
14.7%  8.1%  13.9%  7.8%  -5.7% -10.2%  30.8%  -5.3%  0.0% -59.4%  -2.0% -71.4% -65.3%  0.0%  0.0%  32.6%  0.0%  -1.0%  15.5%  12.7%   
-11.3%  -4.3%  2.6% -50.1%  0.0%  17.6%  9.7% -10.0%  -4.0%   
0.0% 
14.7%  8.1% -50.7%  7.8% -13.6% -10.2%  -8.3%  -5.3%  0.0% -52.2%  12.3%  0.0% -65.9%  0.0%  0.0%  2.7%  0.0%  -1.0%  15.5%   
-11.3%  -5.7%  2.6% -29.2%  0.0%  17.6%  9.7% -10.0%  -4.0%   
-11.3%  -8.1%  2.6%  17.5%  0.0%  17.6%  9.7% -10.0%  -4.0%  9.0%  14.7%  8.1% -12.4%  7.8% -13.2% -10.2%  -8.9%  -5.3%  0.0%  -0.3%  -5.5%  -3.4% -41.7%  0.0%  0.0%  2.1%  0.0% 
-1.0%  15.5%  0.0%  1.5%  3.2% 
-11.3%  -2.8%  2.6% -35.3%  0.0%  17.6%  9.7% -10.0%  -4.0%  9.0%  14.7%  8.1%  0.0%  7.8% -10.0% -10.2%  -8.2%  -5.3%  0.0% -47.4%  -1.4%  0.0% -60.6%  0.0%  0.0%  2.7%  0.0%  -1.0%  15.5%  12.7%  0.0%  3.2% 
-10.2%  -4.3%  -5.3%  0.0%  0.0%  -5.7%  8.2%  5.4%  0.0%  0.0%  0.6%  0.0%  -1.0%  15.5%  0.0%  0.0%  3.2% 
3.2% 
-11.3%  -3.3%  2.6% -56.7%  0.0%  17.6%  9.7% -10.0%  -4.0%   
-11.3%   
3.2% 
-11.3%  -7.5%  2.6%  1.8%  0.0%  17.6%  9.7% -10.0%  -4.0%  0.0%  0.0%  8.1% -21.6%  7.8% -13.9% -10.2%  3.3%  -5.3%  0.0%  -4.7%  -6.3%  -5.5% -22.6%  0.0%  0.0%  3.9%  0.0%  -1.0%  15.5%  12.7%  -8.0%  3.2% 

14.7%  8.1% -29.0%  7.8%  -3.8% -10.2%  11.6%  -5.3%  0.0% -65.0%  -4.6% -54.0% -65.3%  0.0%  0.0%  38.4%  0.0%  -1.0%  15.5%  12.7%   

0.0%  -7.8%  0.0%  0.0%  0.0%  0.0%  0.0% -10.0%  -4.0%  9.0%  14.7%  8.1%   

14.3%  0.0%  -1.0%  15.5%  12.7%   

-4.0%  9.0%  14.7%  8.1%  -3.6%   

-65.6% -69.2%  0.0%  0.0%   

-59.6%  0.0%  17.6%  9.7%   

0.0% -43.8%  0.0%  0.0%   

-82.6% -38.7%  0.0%   

0.0%  -1.0%  15.5%   

-6.8%  -5.3%   

0.0%  -1.0%   

0.0% -63.3%   

-58.8%   

-21.2%   

4.2%   

51 

 
 
 
 
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
 
 
 
Annex A.3: MAD method  

Number of outliers identified by country by indicator MAD method vs. EIS current method (z-score +/- 2 cut-off values) 

Green highlighted cells show the number of more identified and treated outlier using the MAD method vs. the current EIS outlier method (z-score +/- 2 cut-off 
values), red highlighted cells show the number of less identified and treated outliers using the MAD method vs. the current EIS outlier method (z-score +/- 2 
cut-off values). 

Country/Ind. 
Belgium 
Bulgaria 
Czechia 
Denmark 
Germany 
Estonia 
Ireland 
Greece 
Spain 
France 
Croatia 
Italy 
Cyprus 
Latvia 
Lithuania 
Luxembourg 
Hungary 
Malta 
Netherlands 
Austria 
Poland 
Portugal 
Romania 
Slovenia 
Slovakia 
Finland 
Sweden 
Bosnia and 
Herzegovina 
Iceland 
Israel 
North Macedonia 
Montenegro 
Norway 
Serbia 
Switzerland 
Turkey 
Ukraine 
United Kingdom 
Total 

1.1.1  1.1.2  1.1.3  1.2.1  1.2.2  1.2.3  1.3.1  1.3.2  2.1.1  2.1.2  2.1.3  2.2.1  2.2.2  2.2.3  2.3.1  2.3.2  3.1.1  3.1.2  3.2.1  3.2.2  3.2.3  3.3.1  3.3.2  3.3.3  4.1.1  4.1.2  4.2.1  4.2.2  4.2.3  4.3.1  4.3.2  4.3.3 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
-3 
0 
0 
0 

0 
0 
0 
0 
0 
0 
0 
-4 
0 
0 
0 
-7 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
-2 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 

-1 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
-3 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 

0 
1 
0 
0 
0 
0 
0 
0 
0 
0 
0 
1 

0 
0 
0 
2 
0 
0 
0 
0 
0 
0 
0 
0 
1 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 

0 
0 
0 
0 
0 
1 
0 
8 
0 
0 
0 
12 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 

0 
0 
0 
0 
0 
0 
0 
7 
0 
0 
0 
7 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
1 
1 
0 
0 
0 
0 
0 
1 
0 
0 
0 
2 
1 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
6 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 

0 
-7 
0 
0 
0 
0 
0 
0 
0 
0 
0 
-7 

0 
0 
0 
-3 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
-3 

0 
0 
0 
0 
0 
0 
0 
0 
0 
1 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
1 
0 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
2 

0 
0 
0 
0 
0 
0 
2 
0 
0 
0 
0 
0 
0 
0 
0 
0 
1 
0 
0 
1 
0 
0 
0 
0 
0 
0 
0 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
1 
5 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
1 
3 

0 
0 
0 
0 
0 
0 
0 
6 
0 
0 
0 
10 

0 
0 
0 
0 
0 
6 
0 
0 
0 
0 
0 
0 
0 
0 
4 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 

0 
0 
0 
0 
0 
0 
2 
0 
3 
0 
0 
15 

0 
0 
0 
0 
0 
0 
1 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
1 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
-1 
0 
0 
-2 
0 

0 
-2 
0 
0 
0 
-6 
0 
0 
0 
0 
0 
-11 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
-4 
-8 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
-12 

52 

0 
0 
0 
0 
0 
-1 
0 
0 
0 
0 
0 
0 
-1 
0 
0 
0 
0 
0 
0 
0 
0 
0 
-7 
0 
0 
0 
0 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
-9 

-1 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
-1 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 

0 
0 
0 
6 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
1 

0 
0 
0 
0 
0 
0 
0 
8 
0 
0 
0 
15 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
-1 
0 
0 
0 
0 

0 
-1 
0 
0 
0 
0 
0 
0 
0 
0 
0 
-2 

0 
0 
0 
8 
8 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
7 
2 
0 
0 
0 
0 
0 
3 
0 

0 
0 
0 
0 
0 
0 
0 
8 
0 
0 
0 
36 

0 
0 
0 
0 
0 
3 
0 
0 
0 
0 
0 
0 
1 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 

0 
1 
0 
0 
0 
0 
0 
0 
0 
0 
0 
5 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
-3 
0 
0 
0 
0 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
-3 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 

0 
0 
0 
0 
0 
0 
-3 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
-3 

0 
0 
0 
0 
0 
0 
0 
1 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
1 

1 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
4 
0 
0 
0 
3 
0 
0 
3 
0 
0 
0 
0 
0 
0 
0 
0 

0 
0 
0 
0 
0 
0 
0 
3 
0 
0 
0 
14 

0 
0 
0 
0 
0 
4 
0 
0 
0 
0 
0 
0 
6 
0 
0 
0 
0 
0 
0 
0 
1 
0 
0 
0 
0 
0 
0 

0 
2 
0 
0 
0 
0 
6 
0 
0 
0 
0 
19 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 

0 
0 
0 
0 
0 
0 
3 
0 
0 
0 
0 
3 

 
 
 
 
 
Additional statistics MAD method vs. EIS current method (z-score +/- 2 cut-off values) 

Ind. 

EIS 

MIN 

MAX 

1.1.1  1.1.2  1.1.3  1.2.1  1.2.2  1.2.3  1.3.1  1.3.2  2.1.1  2.1.2  2.1.3  2.2.1  2.2.2  2.2.3  2.3.1  2.3.2  3.1.1  3.1.2  3.2.1  3.2.2  3.2.3  3.3.1  3.3.2  3.3.3  4.1.1  4.1.2  4.2.1  4.2.2  4.2.3  4.3.1  4.3.2  4.3.3 

0.03  22.52 

0.9 

9  0.012  0.202 

1.60  57.00 

28.1 

64  0.161 55.295 

0 

34 

8 

0.11 

0.00 

0.00 

0.05 

0.10 

237 

5 

1.10 

7.71  14.89 

1.20 

2.80 

2.00 

0.00 

0.38 

0.00 

5.70  29.26  16.60  14.50 

1.42 

0.62 

0.12 

3.66 

53 

1.07 

0.25 

0.30 

2.44 

1.61  9872 

38 

6.40  44.78  54.59  24.70  27.72  11.70 

2.77 

4.46 

9.64  24.50  80.15  70.28  93.19  23.81 

3.66 

0.85  21.50 

RANGE 

1.57  34.48  27.20  54.88 

0.15  55.09  34.00  45.00 

0.96 

0.25 

0.30 

2.39 

1.51  9635  33.00 

5.30  37.06  39.70  23.50  24.93 

9.70 

2.77 

4.07 

9.64  18.80  50.90  53.68  78.68  22.39 

3.04 

0.73  17.84 

MAD method 

MIN 

MAX 

0.03  21.50 

0.9 

76  0.012  0.202 

1.80  60.30 

27.4  2878  0.161 47.036 

0 

31 

8 

0.11 

0.00 

0.00 

0.05 

0.01 

237 

4 

1.10 

2.84  14.89 

1.20 

7.82 

1.60 

0.00 

0.15 

0.00 

5.70  23.44  16.60  14.50 

1.42 

0.62 

0.01 

3.66 

58 

1.11 

0.23 

0.26 

2.26 

1.48  9172 

44 

7.00  48.87  56.82  24.70 509.28  12.40 

4.97  13.93 

9.64  24.50  80.15  70.28  94.02  20.81 

3.32 

0.39  20.94 

RANGE 

1.77  38.80  26.50  2802 

0.15  46.83  31.00  50.00 

1.00 

0.23 

0.26 

2.21 

1.47  8934  40.00 

5.90  46.04  41.94  23.50 501.45  10.80 

4.97  13.78 

9.64  18.80  56.72  53.68  79.52  19.38 

2.70 

0.37  17.28 

Change in 
RANGE 

Percentage 
change 

Change in MIN 
value 

Change in MAX 
value 

Skewness 
EIS 

Skewness 
MAD method 

Square root 
transformation 
EIS 

Square root 
transformation 
MAD method 

0.20 

4.33 

-0.70  2747 

0.00 

-8.26 

-3.00 

5.00 

0.04 

-0.02 

-0.04 

-0.18 

-0.04 

-700 

7.00 

0.60 

8.97 

2.24 

0.00 476.53 

1.10 

2.20 

9.71 

0.00 

0.00 

5.82 

0.00 

0.83 

-3.01 

-0.34 

-0.36 

-0.56 

12.7%  12.6%  -2.6% 5005%  0.0% 

15.0%  -8.8%  11.1%  4.2%  -8.2% 

12.8%  -7.5%  -2.5%  -7.3%  21.2%  11.3%  24.2%  5.6%  0.0% 1912%  11.3%  79.3% 

%  0.0%  0.0%  11.4%  0.0%  1.1% 

13.4% -11.2% 

49.4%  -3.1% 

-

-

238.2

-

-

0.0%  -4.6%  0.0% 

%  0.0%  0.0% 

--  0.0%  0.0% 

--  0.0%  0.0% -89.7%  0.0% -20.0%  0.0% 

63.2%  0.0%  0.0% 

773.6

-

-

-

179.7
% 

-
20.0% 

-- -61.6% 

--  0.0% 

19.9%  0.0%  0.0%  0.0%  0.0% 

88.2%  0.0% 

-

-

212.4

-

-

12.5%  5.8%  -2.5% 4424%  0.0% 

14.9%  -8.8%  9.4%  3.7%  -8.2% 

12.8%  -7.4%  -8.1%  -7.1%  15.8%  9.4%  9.1%  4.1%  0.0% 1737%  6.0%  79.3% 

%  0.0%  0.0%  0.0%  0.0%  0.9% 

12.6%  -9.3% 

54.8%  -2.6% 

0.392  -0.044  0.810  1.166  0.073  0.809  0.826  0.146  0.188  0.881  0.852  0.695  1.651  0.536  0.098  0.227  0.033  -0.320  0.611  1.357  0.101  1.030  1.507  0.743  0.378  -0.299  -0.524  0.144  0.789  0.783  1.459  0.325 

0.477  -0.027  0.781  0.688  0.073  0.636  0.669  0.249  0.195  0.765  0.693  0.641  0.493  0.428  0.232  0.342  -0.028  -0.302  0.611  0.750  0.106  0.597  0.660  0.743  0.378  -0.404  -0.524  0.145  0.547  0.630  0.657  0.267 

No 

No 

No 

Yes 

No 

No 

No 

No 

No 

No 

No 

No 

Yes 

No 

No 

No 

No 

No 

No 

Yes 

No 

Yes 

Yes 

No 

No 

No 

No 

No 

No 

No 

Yes 

No 

No 

No 

No 

No 

No 

No 

No 

No 

No 

No 

No 

No 

No 

No 

No 

No 

No 

No 

No 

No 

No 

No 

No 

No 

No 

No 

No 

No 

No 

No 

No 

No 

53 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Change in normalised score MAD method vs. EIS current method (z-score +/- 2 cut-off values) (MAD - current method) 

Changes bigger than 25% are highlighted in green, changes smaller than 25% are highlighted in red. 

Belgium 

Bulgaria 

Czechia 

Denmark 

Germany 

Estonia 

Ireland 

Greece 

Spain 

France 

Croatia 

Italy 

Cyprus 

Latvia 

1.1.1 

1.1.2 

1.1.3 

1.2.1 

1.2.2 

1.2.3 

1.3.1 

1.3.2 

2.1.1 

2.1.2 

2.1.3 

2.2.1 

2.2.2 

2.2.3 

2.3.1 

2.3.2 

3.1.1 

3.1.2 

3.2.1 

3.2.2 

3.2.3 

3.3.1 

3.3.2 

3.3.3 

4.1.1 

4.1.2 

4.2.1 

4.2.2 

4.2.3 

4.3.1 

4.3.2 

4.3.3 

-11.3% 

-7.5%  -18.6%  -16.8% 

0.0% 

-1.2% 

0.0%  -10.0% 

-4.0% 

-2.2% 

2.5% 

0.0%  -14.1% 

-1.3%  -14.6%  -10.2% 

-0.7% 

0.0% 

0.0%  13.4% 

-3.1% 

0.0%  -14.2% 

-8.1% 

-6.0% 

3.9% 

7.8% 

-1.0%  15.5% 

-5.7%  -40.7% 

-1.3% 

-11.3% 

-2.2%  -18.6%  -57.4% 

0.0% 

-1.2% 

0.0%  -10.0% 

-4.0% 

-2.2%  388% 

0.0%  -41.4%  -17.8%  23.8%  -10.2%  19.9% 

-5.3% 

-6.7%  13.4%  34.7% 

0.0% 

-7.3% 

-8.1% 

-6.0%  91.7%  24.6% 

-1.0%  15.5% 

-5.7% 

-0.2% 

-1.3% 

-11.3% 

-2.1%  -18.6%  -33.5% 

0.0% 

-1.2% 

0.0%  -10.0% 

-4.0% 

-2.2%  37.7% 

0.0% 

-5.8%  -17.8%  -13.4%  -10.2% 

2.9% 

-5.3% 

-6.7%  13.4% 

3.1% 

0.0%  -27.6% 

-8.1% 

-6.0%  24.9% 

0.6% 

-1.0%  15.5% 

-5.7%  -51.7% 

-1.3% 

-11.3% 

-7.2%  -18.6% 

2.8% 

0.0% 

-1.2% 

0.0%  -10.0% 

-4.0% 

-2.2%  95.6% 

0.0%  -31.1%  -17.8%  -14.2%  -10.2% 

-3.0% 

-5.3% 

-6.7%  13.4% 

-6.1% 

0.0% 

-0.5% 

-8.1% 

-6.0%  13.6% 

7.2% 

-1.0% 

0.0% 

-5.7%  -61.8% 

-1.3% 

-11.3% 

-2.7%  -18.6%  -34.4% 

0.0% 

-1.2% 

0.0%  -10.0% 

-4.0% 

-2.2%  92.4% 

0.0%  11.0% 

-1.2%  -13.2%  -10.2% 

-6.8% 

-5.3% 

-6.7%  13.4% 

-5.6% 

0.0% 

-8.4% 

-8.1% 

-6.0% 

2.3% 

0.7% 

-1.0%  15.5% 

-5.7%  -59.7% 

-1.3% 

-11.3% 

-6.7%  -18.6%  -18.1% 

0.0% 

-1.2% 

0.0%  -10.0% 

-4.0% 

-2.2%  167% 

0.0% 

5.2%  -17.8%  -10.6%  -10.2% 

0.0% 

-5.3% 

0.0%  13.4% 

-6.0% 

0.0% 

2.7% 

-8.1% 

-6.0% 

0.1%  15.9% 

-1.0%  15.5% 

-5.7%  23.6% 

-1.3% 

-11.3% 

-8.4%  -18.6%  -17.5% 

0.0% 

-1.2% 

0.0%  -10.0% 

-4.0% 

-2.2%  28.3% 

0.0%  -31.7%  -17.8%  -13.8%  -10.2% 

-0.1% 

-5.3% 

-6.7%  13.4%    

0.0%  -30.4% 

-8.1% 

-6.0%  46.4% 

2.9% 

-0.7%  15.5% 

-5.7%  -59.1% 

-1.3% 

-11.3% 

-6.6%  -18.6%  -37.5% 

0.0% 

-1.2% 

0.0%  -10.0% 

-4.0% 

-2.2%  141% 

0.0% 

-7.4%  -17.8% 

-5.7%  -10.2% 

-8.0% 

-5.3% 

-6.7%  13.4% 

0.7% 

0.0%  -22.4% 

-8.1% 

-6.0% 

9.2%  44.5% 

-1.0% 

0.0% 

-5.7% 

-3.0% 

-1.3% 

-11.3% 

-7.3%  -18.6%  -35.9% 

0.0% 

-1.2% 

0.0%  -10.0% 

-4.0% 

-2.2%  73.9% 

0.0%  -26.3%  -17.8%  -12.0%  -10.2%  45.1% 

-5.3% 

-6.7%  13.4% 

-4.0% 

0.0% 

-7.3% 

-8.1% 

-6.0%  101%  11.0% 

-1.0%  15.5% 

-5.7%  -32.6% 

-1.3% 

-11.3% 

-7.6%  -18.6%  -37.0% 

0.0% 

-1.2% 

0.0%  -10.0% 

-4.0% 

-2.2% 

0.0% 

0.0%  -33.8%  -17.8% 

-9.3%  -10.2% 

-3.2% 

-5.3% 

-6.7%  13.4% 

-4.7% 

0.0%  -36.8% 

-8.1% 

-6.0%  16.4% 

4.4% 

-1.0%  15.5% 

-5.7%  -42.3% 

-1.3% 

-11.3% 

-4.1%  -18.6%  -38.6% 

0.0% 

-1.2% 

0.0%  -10.0% 

-4.0% 

-2.2%  512% 

0.0% 

-7.2%  -17.8%  -12.9%  -10.2% 

-6.4% 

-5.3% 

-6.7%  13.4% 

-2.0% 

0.0%  -38.7% 

-8.1% 

-6.0%  18.7%  16.8% 

-1.0%  15.5% 

-5.7%  -12.0% 

-1.3% 

-11.3% 

6.5%  -18.6%  -34.6% 

0.0% 

-1.2% 

0.0%  -10.0% 

-4.0% 

-2.2% 

4.8% 

0.0% 

-7.2%  -17.8% 

-9.3%  -10.2% 

-5.6% 

-5.3% 

-6.7%  13.4% 

4.8% 

0.0% 

-9.0% 

-8.1% 

-6.0% 

6.3% 

7.5% 

-1.0%  15.5% 

-4.0%  -41.8% 

-1.3% 

-11.3% 

0.0%  -18.6% 

0.6% 

0.0% 

-1.2% 

0.0%  -10.0% 

-4.0% 

0.0%  656% 

0.0% 

-5.9%  -17.8%  -13.4%  -10.2% 

-1.4% 

0.0% 

0.0%  13.4% 

-5.8% 

0.0%  -34.2% 

-8.1% 

-6.0% 

4.6% 

5.1% 

-1.0%  15.5% 

-5.7%  27.8% 

0.0% 

-11.3% 

-6.9%  -18.6%  -43.9% 

0.0% 

-1.2% 

0.0%  -10.0% 

-4.0% 

-2.2%  429% 

0.0%  -33.6%  -17.8%  -10.6%  -10.2%  24.7% 

-5.3% 

-6.7%  13.4% 

1.4% 

0.0%  -12.1% 

-8.1% 

-6.0%  74.7%  29.2% 

-1.0%  15.5% 

-5.7% 

0.0% 

-1.3% 

Lithuania 

-11.3% 

-8.4%  -18.6%  -39.3% 

0.0% 

-1.2% 

0.0%  -10.0% 

-4.0% 

-2.2%  200% 

0.0%  11.9%  -17.8% 

-8.3%  -10.2% 

-2.2% 

-5.3% 

-6.7%  13.4% 

-6.0% 

0.0% 

-5.0% 

-8.1% 

-6.0%  15.1%  17.2% 

-1.0%  15.5% 

-5.7%  -50.2% 

-1.3% 

Luxembourg 

-11.3% 

-8.4%  -18.6% 

-1.1% 

0.0% 

0.0% 

0.0%  -10.0% 

-4.0% 

0.0%  130% 

0.0%  -48.2%  -17.8%  -12.3%  -10.2% 

-0.8% 

-5.3% 

-6.7%  13.4% 

-5.7% 

0.0% 

0.0% 

-8.1% 

0.0%  31.7% 

9.5% 

-1.0%  15.5% 

-5.7%  -29.0% 

-1.3% 

Hungary 

Malta 

-11.3% 

0.1%  -18.6%  -45.3% 

0.0% 

-1.2% 

0.0%  -10.0% 

-4.0% 

-2.2% 

1.0% 

0.0%  -32.6%  -17.8%  -10.0%  -10.2%  13.7% 

-5.3% 

-6.7%  13.4% 

-1.8% 

0.0%  -33.8% 

-8.1% 

-6.0%  193% 

0.2% 

-1.0%  15.5% 

-5.7%  -33.8% 

-1.3% 

-11.3% 

-6.0%  -18.6%  -32.0% 

0.0% 

-1.2% 

0.0%  -10.0% 

-4.0% 

-2.2%  128% 

0.0%  -15.7%  -17.8%  -13.9%  -10.2% 

-2.2% 

-5.3% 

-6.7%  13.4% 

-5.0% 

0.0% 

0.0% 

-8.1% 

-6.0%  20.4% 

6.2% 

-1.0%  15.5% 

-5.7%  -54.9% 

0.0% 

Netherlands 

-11.3% 

-7.7%  -18.6% 

-8.7% 

0.0% 

-1.2% 

0.0%  -10.0% 

-4.0% 

-2.2%  24.2% 

0.0%  -56.8%  -17.8%  -13.2%  -10.2% 

1.1% 

-5.3% 

-6.7%  13.4% 

-5.6% 

0.0% 

-6.0% 

-8.1% 

-6.0%  20.1% 

7.7% 

-1.0%  15.5% 

0.0%  -46.4% 

-1.3% 

Austria 

Poland 

Portugal 

Romania 

Slovenia 

Slovakia 

Finland 

Sweden 

Bosnia and 
Herzegovina 

Iceland 

Israel 

-11.3% 

-6.4%  -18.6%  -13.2% 

0.0% 

-1.2% 

0.0%  -10.0% 

-4.0% 

-2.2% 

-2.0% 

0.0%  -41.1%  -17.8%  -11.2%  -10.2% 

-3.9% 

-5.3% 

-6.7%  13.4% 

-4.6% 

0.0% 

7.7% 

-8.1% 

-6.0% 

6.6% 

3.9% 

-1.0%  15.5% 

-5.7%  -56.8% 

-1.3% 

-11.3% 

-6.8%  -18.6%  -50.4% 

0.0% 

-1.2% 

0.0%  -10.0% 

-4.0% 

-2.2%  42.6% 

0.0%  -22.9%  -17.8%  -11.2%  -10.2%  68.4% 

-5.3% 

-6.7%  13.4% 

-3.9% 

0.0%  -21.8% 

-8.1% 

-6.0%  462% 

8.6% 

-1.0%  15.5% 

-5.7% 

7.4% 

-1.3% 

-11.3% 

-5.0%  -18.6%  -23.1% 

0.0% 

-1.2% 

0.0%  -10.0% 

-4.0% 

-2.2%  10.2% 

0.0%  -41.6%  -17.8%  -12.9%  -10.2% 

0.7% 

-5.3% 

-6.7%  13.4% 

-5.1% 

0.0%  -10.0% 

-8.1% 

-6.0%  41.6%  14.5% 

-1.0%  15.5% 

-5.7% 

0.0% 

-1.3% 

-11.3%  19.5%  -18.6%  -57.4% 

0.0% 

-1.2% 

0.0%  -10.0% 

-4.0% 

-2.2%  242% 

0.0%  -63.9%  -17.8%  65.0%  -10.2%  220% 

0.0% 

-6.7%  13.4% 

0.0% 

0.0%  -41.7% 

-8.1% 

-6.0% 

-- 

4.3% 

-1.0%  15.5% 

-5.7% 

-7.1% 

-1.3% 

-11.3% 

-6.9%  -18.6%  -19.3% 

0.0% 

-1.2% 

0.0%  -10.0% 

-4.0% 

-2.2%  15.5% 

0.0%  -59.9%  -17.8%  -13.6%  -10.2% 

-5.2% 

-5.3% 

-6.7%  13.4% 

-2.4% 

0.0% 

-3.1% 

-8.1% 

-6.0%  22.6% 

3.4% 

-1.0%  15.5% 

-5.7%  -21.6% 

-1.3% 

-11.3% 

-5.7%  -18.6%  -40.6% 

0.0% 

-1.2% 

0.0%  -10.0% 

-4.0% 

-2.2%  149% 

0.0%  -14.6%  -17.8%  -10.0%  -10.2%  43.0% 

-5.3% 

-6.7%  13.4%  17.5% 

0.0%  -29.8% 

-8.1% 

-6.0%  109% 

0.3% 

-1.0%  15.5% 

-5.7%  -41.2% 

-1.3% 

-11.3% 

-6.5%  -15.9% 

-5.8% 

0.0% 

-1.2% 

0.0%  -10.0% 

-4.0% 

-2.2%  106% 

0.0%  -27.8%  -17.8%  -15.0% 

-3.4% 

-5.2% 

-5.3% 

0.0%  13.4% 

-4.7% 

0.0% 

-1.4% 

-8.1% 

-6.0% 

6.1%  10.8% 

-1.0%  15.5% 

-5.7%  -34.2% 

-1.3% 

-11.3% 

-7.6% 

0.0% 

-1.3% 

0.0% 

-1.2% 

0.0%  -10.0% 

-4.0% 

-2.2%  41.1% 

0.0%  -28.6% 

0.0%  -14.4% 

0.0% 

-8.0% 

-5.3% 

-6.7%  13.4% 

-2.0% 

0.0% 

1.3% 

-8.1% 

-6.0% 

6.6% 

5.0% 

-1.0%  15.5% 

-5.7%  -43.0% 

-1.3% 

-11.3%  46.5%  -18.6%  -38.6% 

0.0%    

0.0% 

0.0% 

-4.0%    

628% 

0.0% 

0.0% 

0.0% 

-9.3%    

-6.8% 

-5.3%    

13.4%    

0.0%  -38.7% 

0.0%    

31.4%  74.3% 

-1.0%  15.5% 

-5.7%    

-1.3% 

-11.3% 

-7.6%  -18.6% 

0.0% 

0.0% 

-1.2% 

0.0% 

0.0% 

-4.0%    

-1.2% 

0.0%  -27.9%  -17.8%  -14.6%  -10.2% 

0.1% 

-5.3% 

-6.7% 

0.0% 

-5.7% 

0.0%  -34.2% 

-8.1% 

-6.0% 

8.3% 

0.0% 

-1.0%  15.5% 

-5.7% 

-0.9% 

-1.3% 

-11.3% 

-7.6%    

-29.3% 

0.0%    

-4.0%    

47.7% 

0.0%    

13.4%    

0.0%  -43.8% 

-8.1% 

0.0%    

2.3% 

-1.0%    

North Macedonia 

-11.3% 

-4.3%  -18.6%  -59.6% 

0.0% 

-1.2% 

0.0%  -10.0% 

-4.0%    

533% 

0.0% 

-5.9%  -17.8% 

-5.7%  -10.2%  30.8% 

-5.3% 

-6.7%  13.4% 

-2.0% 

0.0%  -65.3% 

-8.1% 

-6.0%  71.5% 

1.5% 

-1.0%  15.5% 

-5.7%    

Montenegro 

-11.3% 

-5.7%  -18.6%  -42.7% 

0.0% 

-1.2% 

0.0%  -10.0% 

-4.0%    

1339% 

0.0%  -59.3%  -17.8%  -13.6%  -10.2% 

-8.3% 

-5.3% 

-6.7%  13.4%  12.3% 

0.0%  -65.9% 

0.0% 

-6.0% 

6.0%  359% 

-1.0%  15.5%    

Norway 

Serbia 

-11.3% 

-8.1%  -18.6% 

1.8% 

0.0% 

-1.2% 

0.0%  -10.0% 

-4.0% 

-2.2% 

6.2% 

0.0%  -27.6%  -17.8%  -13.2%  -10.2% 

-8.9% 

-5.3% 

0.0%  13.4% 

-5.5% 

0.0%  -41.7% 

-8.1% 

-6.0% 

4.7% 

-- 

-1.0%  15.5% 

0.0%  -19.2% 

-1.3% 

-11.3% 

-2.8%  -18.6%  -47.7% 

0.0% 

-1.2% 

0.0%  -10.0% 

-4.0% 

-2.2%  219% 

0.0% 

0.0%  -17.8%  -10.0%  -10.2% 

-8.2% 

-5.3% 

-6.7%  13.4% 

-1.4% 

0.0%  -60.6% 

-8.1% 

-6.0% 

5.9%  12.6% 

-1.0%  15.5% 

-5.7% 

0.0% 

-1.3% 

Switzerland 

0.0% 

-7.8% 

-6.0% 

0.0% 

0.0% 

0.0% 

0.0%  -10.0% 

-4.0% 

-2.2%  221% 

0.0%    

-10.2% 

-4.3% 

-5.3% 

-6.7% 

0.0% 

-5.7% 

0.0% 

5.4% 

-8.1% 

-6.0% 

1.2% 

6.1% 

-1.0%  15.5% 

0.0% 

0.0% 

-1.3% 

Turkey 

Ukraine 

-11.3% 

-3.3%  -18.6%  -64.9% 

0.0% 

-1.2% 

0.0%  -10.0% 

-4.0%    

27.3% 

0.0%  -41.3%  -17.8% 

-3.8%  -10.2%  11.6% 

-5.3% 

-6.7%  13.4% 

-4.6% 

0.0%  -65.3% 

-8.1% 

-6.0%  84.3% 

9.7% 

-1.0%  15.5% 

-5.7%    

-11.3%    

-67.3% 

0.0% 

-1.2% 

0.0%    

-4.0% 

-2.2%  132% 

0.0%  -20.3%    

4.2%    

-6.7%  13.4%    

0.0%  -69.2% 

-8.1% 

-6.0%    

58.5% 

-1.0%  15.5%    

-1.3% 

-1.3% 

United Kingdom 

-11.3% 

-7.5%  -18.6%  -17.6% 

0.0% 

-1.2% 

0.0%  -10.0% 

-4.0% 

0.0% 

-4.3% 

0.0%  -35.2%  -17.8%  -13.9%  -10.2% 

3.3% 

-5.3% 

-6.7%  13.4% 

-6.3% 

0.0%  -22.6% 

-8.1% 

-6.0% 

8.6% 

6.2% 

-1.0%  15.5% 

-5.7%  -26.7% 

-1.3% 

54 

-1.3% 

-1.3% 

0.0% 

 
 
 
 
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
 
 
 
Annex A.4: Z-score  

Number of outliers identified by country by indicator Z-score (+/- 3 cut-off values) method vs. EIS current method (z-score +/- 2 cut-off values) 

Green highlighted cells show the number of more identified and treated outlier using the Z-score method vs. the current EIS outlier method (z-score +/- 2 cut-
off values), red highlighted cells show the number of less identified and treated outliers using the Z-score (+/- 3 cut-off values) method vs. the current EIS 
outlier method (z-score +/- 2 cut-off values). 

Country/Ind. 
Belgium 
Bulgaria 
Czechia 
Denmark 
Germany 
Estonia 
Ireland 
Greece 
Spain 
France 
Croatia 
Italy 
Cyprus 
Latvia 
Lithuania 
Luxembourg 
Hungary 
Malta 
Netherlands 
Austria 
Poland 
Portugal 
Romania 
Slovenia 
Slovakia 
Finland 
Sweden 
Bosnia and 
Herzegovina 
Iceland 
Israel 
North Macedonia 
Montenegro 
Norway 
Serbia 
Switzerland 
Turkey 
Ukraine 
United Kingdom 
Total 

1.1.1  1.1.2  1.1.3  1.2.1  1.2.2  1.2.3  1.3.1  1.3.2  2.1.1  2.1.2  2.1.3  2.2.1  2.2.2  2.2.3  2.3.1  2.3.2  3.1.1  3.1.2  3.2.1  3.2.2  3.2.3  3.3.1  3.3.2  3.3.3  4.1.1  4.1.2  4.2.1  4.2.2  4.2.3  4.3.1  4.3.2  4.3.3 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
-3 
0 
0 
0 

0 
0 
0 
0 
0 
0 
0 
-4 
0 
0 
0 
-7 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
-2 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 

-1 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
-3 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
-2 
-8 

0 
0 
0 
0 
0 
0 
0 
-8 
0 
0 
0 
-18 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 

0 
0 
0 
0 
0 
0 
0 
-1 
0 
0 
0 
-1 

0 
0 
0 
-1 
0 
0 
0 
0 
-1 
0 
0 
0 
0 
0 
0 
0 
0 
0 
-1 
0 
0 
-2 
0 
0 
0 
-1 
-1 

0 
0 
0 
0 
0 
0 
0 
-3 
0 
0 
0 
-10 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 

0 
-8 
0 
0 
0 
0 
0 
0 
0 
0 
0 
-8 

0 
0 
0 
-3 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
-3 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
-1 
0 
-3 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
-2 
-6 

0 
0 
0 
0 
0 
0 
-2 
0 
0 
-7 
0 
0 
0 
0 
0 
0 
-3 
0 
0 
-1 
0 
0 
0 
-3 
0 
0 
0 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
-3 
-19 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 

0 
0 
0 
0 
0 
0 
-3 
0 
-4 
0 
0 
-7 

-3 
0 
0 
-2 
-5 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
-8 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
-18 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
-1 
0 
0 
-2 
0 

0 
-2 
0 
0 
0 
-6 
0 
0 
0 
0 
0 
-11 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
-4 
-8 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
-12 

55 

0 
0 
0 
0 
0 
-1 
0 
0 
0 
0 
0 
0 
-1 
0 
0 
0 
0 
0 
0 
0 
0 
0 
-7 
0 
0 
0 
0 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
-9 

-1 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
-1 
0 
0 
0 
0 
0 
0 
0 
0 
0 
-5 
0 
0 
0 
0 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
-7 

-3 
0 
0 
0 
0 
-1 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
-1 
0 

0 
0 
0 
0 
0 
-1 
0 
0 
0 
0 
-2 
-8 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
-1 
0 
0 
0 
0 

0 
-1 
0 
0 
0 
0 
0 
0 
0 
0 
0 
-2 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
-5 
-8 

0 
0 
-8 
0 
0 
0 
0 
0 
0 
0 
0 
-21 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
-6 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
-6 

0 
-1 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
-1 
0 
-2 
0 
0 
0 
0 
0 
0 
0 
0 
0 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
-4 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
-1 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
-1 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
-6 
0 
0 
0 
0 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
-6 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 

0 
-8 
0 
0 
-4 
-6 
0 
0 
0 
0 
0 
-18 

0 
0 
0 
0 
0 
0 
-3 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
-3 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 

0 
0 
0 
0 
0 
0 
0 
0 
-7 
0 
0 
-7 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
-1 
0 
0 
0 
0 
0 
0 
-3 
0 
0 
0 
0 
0 
0 
0 
0 

0 
0 
0 
0 
0 
-8 
0 
-2 
0 
0 
0 
-14 

0 
0 
0 
0 
0 
-4 
0 
0 
0 
0 
0 
0 
0 
-6 
0 
0 
0 
0 
0 
0 
0 
-4 
0 
0 
0 
0 
0 

0 
0 
0 
0 
0 
0 
-2 
0 
0 
0 
0 
-16 

0 
-1 
0 
-4 
0 
-4 
0 
0 
0 
0 
0 
0 
-1 
0 
0 
0 
0 
-1 
0 
0 
0 
0 
0 
0 
0 
0 
0 

-1 
0 
0 
-1 
-4 
0 
0 
0 
0 
0 
0 
-17 

 
 
 
 
 
Additional statistics Z-score (+/- 3 cut-off values) method vs. EIS current method (z-score +/- 2 cut-off values) 

Ind. 

1.1.1  1.1.2  1.1.3  1.2.1  1.2.2  1.2.3  1.3.1  1.3.2  2.1.1  2.1.2  2.1.3  2.2.1  2.2.2  2.2.3  2.3.1  2.3.2  3.1.1  3.1.2  3.2.1  3.2.2  3.2.3  3.3.1  3.3.2  3.3.3  4.1.1  4.1.2  4.2.1  4.2.2  4.2.3  4.3.1  4.3.2  4.3.3 

EIS (+/- 2) 

MIN 

MAX 

0.03  22.52 

0.9 

9  0.012  0.202 

1.60  57.00 

28.1 

64  0.161  55.295 

0 

34 

8 

0.11 

0.00 

0.00 

0.05 

0.10 

237 

5 

1.10 

7.71  14.89 

1.20 

2.80 

2.00 

0.00 

0.38 

0.00 

5.70  29.26  16.60  14.50 

1.42 

0.62 

0.12 

3.66 

53 

1.07 

0.25 

0.30 

2.44 

1.61  9872 

38 

6.40  44.78  54.59  24.70  27.72  11.70 

2.77 

4.46 

9.64  24.50  80.15  70.28  93.19  23.81 

3.66 

0.85  21.50 

RANGE 

1.57  34.48  27.20  54.88 

0.15  55.09  34.00  45.00 

0.96 

0.25 

0.30 

2.39 

1.51 

964  33.00 

5.30  37.06  39.70  23.50  24.93 

9.70 

2.77 

4.07 

9.64  18.80  50.90  53.68  78.68  22.39 

3.04 

0.73  17.84 

Z-score 
(+/- 3 cut-
off values) 
method 

MIN 

MAX 

0.03  21.50 

0.9 

9  0.012  0.202 

0 

8 

0.11 

0.00 

0.01 

0.05 

0.10 

237 

4 

1.10 

2.84 

7.57 

1.20 

2.80 

1.60 

0.00 

0.38 

0.00 

5.70  14.44 

8.16  14.50 

1.19 

0.62 

0.12 

0.00 

1.80  60.30 

34.3 

64  0.161  55.940 

6.60 

62 

1.11 

0.56 

0.63 

2.44 

1.68  12524 

44 

7.00  48.87  65.62  30.56  27.72  12.40 

3.26 

5.16 

3.48  25.70  80.15  70.28  94.02 

5.79 

4.55 

0.98  26.37 

RANGE 

1.77  38.80  33.40  54.88 

0.15  55.74 

6.60  54.00 

1.00 

0.56 

0.63 

2.39 

1.58  12287  40.00 

5.90  46.04  58.05  29.36  24.93  10.80 

3.26 

4.78 

3.48  20.00  65.71  62.13  79.52 

4.60 

3.93 

0.87  26.37 

Change in 
RANGE 

Percentage 
change 

Change in 
MIN value 

0.20 

4.33 

6.20 

0.00 

0.00 

0.65  -27.40 

9.00 

0.04 

0.31 

0.33 

0.00 

0.07  2653 

7.00 

0.60 

8.97  18.34 

5.86 

0.00 

1.10 

0.49 

0.70 

-6.16 

1.20  14.81 

8.44 

0.83  -17.79 

0.89 

0.13 

8.53 

13% 

13% 

23% 

0% 

0% 

1% 

-81% 

20% 

4%  121%  108% 

0% 

5% 

28% 

21% 

11% 

24% 

46% 

25% 

0% 

11% 

18% 

17% 

-64% 

6% 

29% 

16% 

1% 

-79% 

29% 

18% 

48% 

0.0%  -4.6%  0.0%  0.0%  0.0%  0.0% 

--  0.0%  0.0% 

-- 17992%  0.0%  0.0%  0.0% -20.0%  0.0% -63.2% -49.1%  0.0%  0.0% -20.0% 

--  0.0% 

--  0.0% -50.6% -50.9%  0.0% -16.1%  0.0%  0.0%  -100% 

Change in 
MAX value  12.5%  5.8%  22.1%  0.0%  0.0%  1.2% -80.6%  17.0%  3.7% 

121.4
% 

109.6

%  0.0%  4.4%  26.9%  15.8%  9.4%  9.1%  20.2%  23.7%  0.0%  6.0%  17.6%  15.8% -63.9%  4.9%  0.0%  0.0%  0.9% -75.7%  24.4%  15.5%  22.6% 

Skewness 
EIS (+/- 2) 

Skewness 
Z-score (+/- 
3) method 

Square root 
trans-
formation 
EIS (+/- 2) 

Square root 
trans-
formation 
Z-score (+/- 
3) method 

0.392  -0.044  0.810  1.166  0.073  0.809  0.826  0.146  0.188  0.881  0.852  0.695  1.651  0.536  0.098  0.227  0.033  -0.320  0.611  1.357  0.101  1.030  1.507  0.743  0.378  -0.299  -0.524  0.144  0.789  0.783  1.459  0.325 

0.477  -0.027  0.948  1.166  0.073  0.820  1.252  0.264  0.195  1.219  1.120  0.695  1.790  0.844  0.232  0.342  -0.028  -0.359  0.808  1.357  0.106  1.314  2.187  1.100  0.479  -0.559  -0.662  0.145  1.800  0.983  1.786  0.392 

No 

No 

No 

Yes 

No 

No 

No 

No 

No 

No 

No 

No 

Yes 

No 

No 

No 

No 

No 

No 

Yes 

No 

Yes 

Yes 

No 

No 

No 

No 

No 

No 

No 

Yes 

No 

No 

No 

No 

Yes 

No 

No 

Yes 

No 

No 

Yes 

Yes 

No 

Yes 

No 

No 

No 

No 

No 

No 

Yes 

No 

Yes 

Yes 

Yes 

No 

No 

No 

No 

Yes 

No 

Yes 

No 

56 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Change  in  normalised  score  Z-score  (+/-  3  cut-off  values)  method  vs.  EIS  current  method  (z-score  +/-  2  cut-off  values)  (Z-score  (+/-  3  -  current 
method) 

Changes bigger than 25% are highlighted in green, changes smaller than 25% are highlighted in red. 

Belgium 

Bulgaria 

Czechia 

Denmark 

Germany 

Estonia 

Ireland 

Greece 

Spain 

France 

Croatia 

Italy 

Cyprus 

Latvia 

1.1.1 

1.1.2 

1.1.3 

1.2.1 

1.2.2 

1.2.3 

1.3.1 

1.3.2 

2.1.1 

2.1.2 

2.1.3 

2.2.1 

2.2.2 

2.2.3 

2.3.1 

2.3.2 

3.1.1 

3.1.2 

3.2.1 

3.2.2 

3.2.3 

3.3.1 

3.3.2 

3.3.3 

4.1.1 

4.1.2 

4.2.1 

4.2.2 

4.2.3 

4.3.1 

4.3.2 

4.3.3 

-11.3% 

-7.5%  -18.6%  0.0%  0.0% 

-1.2% 

-7.5%  -16.7% 

-4.0%  12.2% 

-2.9%  0.0% 

-4.5% 

-5.8%  -14.6%  -10.2% 

-0.7%  -15.1%  -14.3%  0.0% 

-3.1%  -14.9%  -14.7%  73.8% 

-6.0%  3.9%  7.8% 

-1.0% 

-5.5%  -22.7%  -15.3%  10.7% 

-11.3% 

-2.2%  -18.6%  0.0%  0.0% 

-1.2%  33.0%  -16.7% 

-4.0% 238.1% 362.6%  0.0% 

-4.5%  -21.6%  23.8%  -10.2%  19.9%  83.8%  -20.0%  0.0%  34.7%  -14.9%  -14.7%  32.2% 

-6.0%  91.7%  24.6% 

-1.0%  31.2%  -22.7%  -15.3%  -18.0% 

-11.3% 

-2.1%  -18.6%  0.0%  0.0% 

-1.2%  33.0%  -16.7% 

-4.0% 155.0%  30.4%  0.0% 

-4.5%  -21.6%  -13.4%  -10.2%  2.9%  -10.8%  -20.0%  0.0%  3.1%  -14.9%  -14.7%  82.1% 

-6.0%  24.9%  0.6% 

-1.0%  2.1%  -22.7%  -15.3%  5.9% 

-11.3% 

-7.2%  -18.6%  0.0%  0.0% 

-1.2%  0.0%  -16.7% 

-4.0%  7.3%  85.3%  0.0% 

-4.5%  -21.6%  -14.2%  -10.2% 

-3.0%  -15.6%  -20.0%  0.0% 

-6.1%  -14.9%  -14.7% 

-0.5% 

-6.0%  13.6%  7.2% 

-1.0%  0.0%  -22.7%  -15.3%  -17.3% 

-11.3% 

-2.7%  -18.6%  0.0%  0.0% 

-1.2%  12.4%  -16.7% 

-4.0%  33.8%  82.3%  0.0% 

-4.5% 

-5.7%  -13.2%  -10.2% 

-6.8%  -18.9%  -20.0%  0.0% 

-5.6%  -14.9%  -14.7%  21.9% 

-6.0%  2.3%  0.7% 

-1.0% 

-3.5%  -22.7%  -15.3% 

-1.2% 

-11.3% 

-6.7%  -18.6%  0.0%  0.0% 

-1.2%  24.9%  -16.7% 

-4.0% 

-1.9% 153.0%  0.0% 

-4.5%  -21.6%  -10.6%  -10.2%  0.0%  -18.3%  -10.0%  0.0% 

-6.0%  -14.9%  -14.7%  30.7% 

-6.0%  0.1%  15.9% 

-1.0%  1.0%  -22.7%  -15.3%  -14.4% 

-11.3% 

-8.4%  -18.6%  0.0%  0.0% 

-1.2%  9.8%  -16.7% 

-4.0%  9.5%  21.5%  0.0% 

-4.5%  -21.6%  -13.8%  -10.2% 

-0.1% 

-9.9%  -20.0%  0.0%   

-14.9%  -14.7% 115.3% 

-6.0%  46.4%  2.9% 

-0.7%  9.6%  -22.7%  -15.3%  78.2% 

-11.3% 

-6.6%  -18.6%  0.0%  0.0% 

-1.2%  82.1%  -16.7% 

-4.0% 123.5% 128.6%  0.0% 

-4.5%  -21.6% 

-5.7%  -10.2% 

-8.0%  -19.0%  -20.0%  0.0%  0.7%  -14.9%  -14.7% 218.3% 

-6.0%  9.2%  44.5% 

-1.0%  -19.8%  -22.7%  -15.3%  1.1% 

-11.3% 

-7.3%  -18.6%  0.0%  0.0% 

-1.2% 

-5.4%  -16.7% 

-4.0%  19.0%  64.7%  0.0% 

-4.5%  -21.6%  -12.0%  -10.2%  45.1%  30.7%  -20.0%  0.0% 

-4.0%  -14.9%  -14.7%  78.9% 

-6.0% 100.8%  11.0% 

-1.0% 

-6.5%  -22.7%  -15.3%  1.1% 

-11.3% 

-7.6%  -18.6%  0.0%  0.0% 

-1.2%  42.8%  -16.7% 

-4.0% 

-8.7%  0.0%  0.0% 

-4.5%  -21.6% 

-9.3%  -10.2% 

-3.2%  -10.9%  -20.0%  0.0% 

-4.7%  -14.9%  -14.7%  81.6% 

-6.0%  16.4%  4.4% 

-1.0%  16.9%  -22.7%  -15.3%  0.6% 

-11.3% 

-4.1%  -18.6%  0.0%  0.0% 

-1.2%  48.7%  -16.7% 

-4.0%  41.5% 479.8%  0.0% 

-4.5%  -21.6%  -12.9%  -10.2% 

-6.4%  -15.2%  -20.0%  0.0% 

-2.0%  -14.9%  -14.7% 244.7% 

-6.0%  18.7%  16.8% 

-1.0%  6.3%  -22.7%  -15.3%  10.4% 

-11.3%  6.5%  -18.6%  0.0%  0.0% 

-1.2%  42.8%  -16.7% 

-4.0%  68.8% 

-0.7%  0.0% 

-4.5%  -21.6% 

-9.3%  -10.2% 

-5.6%  -18.6%  -20.0%  0.0%  4.8%  -14.9%  -14.7%  20.6% 

-6.0%  6.3%  7.5% 

-1.0% 

-8.1%  -21.2%  -15.3%  9.5% 

-11.3%  0.0%  -18.6%  0.0%  0.0% 

-1.2%  24.9%  -16.7% 

-4.0%  0.0% 616.3%  0.0% 

-4.5%  -21.6%  -13.4%  -10.2% 

-1.4%  0.0%  0.0%  0.0% 

-5.8%  -14.9%  -14.7%  78.7% 

-6.0%  4.6%  5.1% 

-1.0%  6.0%  -22.7%  -15.3%  0.0% 

-11.3% 

-6.9%  -18.6%  0.0%  0.0% 

-1.2%  24.9%  -16.7% 

-4.0% 227.8% 400.8%  0.0% 

-4.5%  -21.6%  -10.6%  -10.2%  24.7%  17.7%  -20.0%  0.0%  1.4%  -14.9%  -14.7%  99.4% 

-6.0%  74.7%  29.2% 

-1.0%  19.1%  -22.7% 

-5.3% 135.0% 

Lithuania 

-11.3% 

-8.4%  -18.6%  0.0%  0.0% 

-1.2%  -11.7%  -16.7% 

-4.0%  21.6% 183.8%  0.0% 

-4.5%  -21.6% 

-8.3%  -10.2% 

-2.2%  -13.9%  -20.0%  0.0% 

-6.0%  -14.9%  -14.7% 131.7% 

-6.0%  15.1%  17.2% 

-1.0%  13.9%  -22.7%  -15.3%  -11.2% 

Luxembourg 

-11.3% 

-8.4%  -18.6%  0.0%  0.0%  0.0%  -11.7%  -16.7% 

-4.0% 

-9.4% 118.0%  0.0% 

-4.5%  -21.6%  -12.3%  -10.2% 

-0.8%  -11.1%  -20.0%  0.0% 

-5.7%  -14.9%  -12.1%  14.7%  0.0%  31.7%  9.5% 

-1.0%  31.1%  -22.7%  -15.3%  5.7% 

Hungary 

Malta 

-11.3%  0.1%  -18.6%  0.0%  0.0% 

-1.2%  12.4%  -16.7% 

-4.0%  40.2% 

-4.3%  0.0% 

-4.5%  -21.6%  -10.0%  -10.2%  13.7%  90.1%  -20.0%  0.0% 

-1.8%  -14.9%  -14.7% 187.9% 

-6.0% 192.9%  0.2% 

-1.0%  16.9%  -22.7%  -15.3%  18.6% 

-11.3% 

-6.0%  -18.6%  0.0%  0.0% 

-1.2% 

-4.4%  -16.7% 

-4.0% 456.9% 116.2%  0.0% 

-4.5%  -21.6%  -13.9%  -10.2% 

-2.2%  -11.4%  -20.0%  0.0% 

-5.0%  -14.9% 

0.0%  46.7% 

-6.0%  20.4%  6.2% 

-1.0%  14.1%  -22.7%  -15.3%  -13.5% 

Netherlands 

-11.3% 

-7.7%  -18.6%  0.0%  0.0% 

-1.2%  -10.4%  -16.7% 

-4.0%  2.8%  17.6%  0.0% 

-4.5%  -21.6%  -13.2%  -10.2%  1.1%  -11.2%  -20.0%  0.0% 

-5.6%  -14.9%  -14.7%  29.0% 

-6.0%  20.1%  7.7% 

-1.0%  20.0%  0.0%  -15.3%  14.5% 

Austria 

Poland 

Portugal 

Romania 

Slovenia 

Slovakia 

Finland 

Sweden 

Bosnia and 
Herzegovina 

Iceland 

Israel 

-11.3% 

-6.4%  -18.6%  0.0%  0.0% 

-1.2%  24.9%  -16.7% 

-4.0% 104.1% 

-7.2%  0.0% 

-4.5%  -21.6%  -11.2%  -10.2% 

-3.9%  -18.8%  -20.0%  0.0% 

-4.6%  -14.9%  -14.7%  10.7% 

-6.0%  6.6%  3.9% 

-1.0% 

-3.7%  -22.7%  -15.3%  4.0% 

-11.3% 

-6.8%  -18.6%  0.0%  0.0% 

-1.2%  5.1%  -16.7% 

-4.0%  98.5%  35.1%  0.0% 

-4.5%  -21.6%  -11.2%  -10.2%  68.4% 179.4%  -20.0%  0.0% 

-3.9%  -14.9%  -14.7%  22.0% 

-6.0% 461.8%  8.6% 

-1.0%  30.6%  -22.7%  -15.3% 

-1.1% 

-11.3% 

-5.0%  -18.6%  0.0%  0.0% 

-1.2% 

-3.0%  -16.7% 

-4.0%  40.3%  4.4%  0.0% 

-4.5%  -21.6%  -12.9%  -10.2%  0.7% 

-1.7%  -20.0%  0.0% 

-5.1%  -14.9%  -14.7%  66.2% 

-6.0%  41.6%  14.5% 

-1.0%  3.8%  -22.7% 

-6.3%  3.9% 

-11.3%  19.5%  -18.6%  0.0%  0.0% 

-1.2%  7.4%  -16.7% 

-4.0%  41.4% 223.6%  0.0% 

-4.5%  -21.6%  65.0%  -10.2% 220.3%  0.0%  -20.0%  0.0%  0.0%  -14.9%  -14.7% 197.6% 

-6.0% 

--  4.3% 

-1.0%  17.1%  -22.7%  -15.3% 136.1% 

-11.3% 

-6.9%  -18.6%  0.0%  0.0% 

-1.2%  9.8%  -16.7% 

-4.0% 471.2%  9.4%  0.0% 

-4.5%  -21.6%  -13.6%  -10.2% 

-5.2% 

-7.8%  -20.0%  0.0% 

-2.4%  -14.9%  -14.7%  93.0% 

-6.0%  22.6%  3.4% 

-1.0%  3.6%  -22.7%  -15.3%  11.8% 

-11.3% 

-5.7%  -18.6%  0.0%  0.0% 

-1.2%  33.0%  -16.7% 

-4.0% 212.3% 135.8%  0.0% 

-4.5%  -21.6%  -10.0%  -10.2%  43.0%  49.3%  -20.0%  0.0%  17.5%  -14.9%  -14.7% 139.4% 

-6.0% 109.4%  0.3% 

-1.0%  7.2%  -22.7%  -15.3%  -12.6% 

-11.3% 

-6.5%  -15.9%  0.0%  0.0% 

-1.2% 

-5.4%  -16.7% 

-4.0%  -10.4%  95.0%  0.0% 

-4.5%  -21.6%  -15.0% 

-3.4% 

-5.2%  -15.8%  -10.2%  0.0% 

-4.7%  -14.9%  -14.7%  29.4% 

-6.0%  6.1%  10.8% 

-1.0% 

-2.1%  -22.7%  -15.3% 

-1.6% 

-11.3% 

-7.6%  0.0%  0.0%  0.0% 

-1.2%  0.0%  -16.7% 

-4.0%  5.4%  33.6%  0.0% 

-4.5% 

-4.6%  -14.4%  0.0% 

-8.0%  -16.2%  -20.0%  0.0% 

-2.0% 

-8.4%  -14.7%  36.7% 

-6.0%  6.6%  5.0% 

-1.0% 

-0.5%  -22.7%  -15.3% 

-2.6% 

-11.3%  46.5%  -18.6%  0.0%  0.0%   

62.9%  0.0% 

-4.0%   

589.7%  0.0%  0.0%  0.0% 

-9.3%   

-6.8% 

-6.5%   

0.0%   

-14.9%  -14.7%  0.0%   

31.4%  74.3% 

-1.0%  15.8%  -22.7%   

27.3% 

-11.3% 

-7.6%  -18.6%  0.0%  0.0% 

-1.2% 

-6.0%  0.0% 

-4.0%   

-6.5%  0.0% 

-4.5%  -21.6%  -14.6%  -10.2%  0.1%  -13.1%  -20.0%  0.0% 

-5.7%  -14.9%  -14.7% 576.1% 

-6.0%  8.3%  0.0% 

-1.0%  37.1%  -22.7%  -15.3%  10.5% 

-11.3% 

-7.6%   

0.0%  0.0%   

-4.0%   

39.9%  0.0%   

0.0%   

-7.9%  -14.7% 167.6%  0.0%   

2.3% 

-1.0%   

North Macedonia 

-11.3% 

-4.3%  -18.6%  0.0%  0.0% 

-1.2%  55.3%  -16.7% 

-4.0%   

499.7%  0.0% 

-4.5%  -21.6% 

-5.7%  -10.2%  30.8% 

-4.7%  -20.0%  0.0% 

-2.0%  -14.9%  -14.7%  1900% 

-6.0%  71.5%  1.5% 

-1.0%  58.6%  -22.7%   

Montenegro 

-11.3% 

-5.7%  -18.6%  0.0%  0.0% 

-1.2%  24.9%  -16.7% 

-4.0%   

1263%  0.0% 

-4.5%  -21.6%  -13.6%  -10.2% 

-8.3%  -12.7%  -20.0%  0.0%  12.3%  0.0%  -14.7%  0.0% 

-6.0%  6.0% 358.9% 

-1.0%  25.8%   

Norway 

Serbia 

-11.3% 

-8.1%  -18.6%  0.0%  0.0% 

-1.2%  21.4%  -16.7% 

-4.0%  57.0%  0.6%  0.0% 

-4.5%  -21.6%  -13.2%  -10.2% 

-8.9%  -18.0% 

-8.7%  0.0% 

-5.5%  -14.9%  -14.7% 231.2% 

-6.0%  4.7% 

-- 

-1.0%  20.8% 

-2.3%  -15.3%  11.3% 

-11.3% 

-2.8%  -18.6%  0.0%  0.0% 

-1.2%  33.0%  -16.7% 

-4.0% 357.0% 202.3%  0.0%  0.0%  -21.6%  -10.0%  -10.2% 

-8.2%  -13.4%  -20.0%  0.0% 

-1.4%  0.0%  -14.7% 481.9% 

-6.0%  5.9%  12.6% 

-1.0%  5.1%  -22.7%  -10.3% 

-8.5% 

Switzerland 

0.0% 

-7.8% 

-6.0%  0.0%  0.0%  0.0%  0.0%  -16.7% 

-4.0% 

-4.9% 204.2%  0.0%   

-10.2% 

-4.3%  -17.6%  -20.0%  0.0% 

-5.7%  -14.9%  -14.7%  8.9% 

-6.0%  1.2%  6.1% 

-1.0% 

-2.2%  -18.0%  0.0%  49.3% 

Turkey 

Ukraine 

-11.3% 

-3.3%  -18.6%  0.0%  0.0% 

-1.2%  15.2%  -16.7% 

-4.0%   

20.6%  0.0% 

-4.5%  -21.6% 

-3.8%  -10.2%  11.6%  5.6%  -20.0%  0.0% 

-4.6%  -14.9%  -14.7% 779.6% 

-6.0%  84.3%  9.7% 

-1.0%  11.8%  -22.7%   

-11.3%   

0.0%  0.0% 

-1.2%  78.8%   

-4.0% 143.2% 120.1%  0.0% 

-4.5%   

4.2%   

-20.0%  0.0%   

-14.9%  -14.7%  1095% 

-6.0%   

58.5% 

-1.0%  61.8%   

21.8% 

23.4% 

United Kingdom 

-11.3% 

-7.5%  -18.6%  0.0%  0.0% 

-1.2%  21.4%  -16.7% 

-4.0% 

-0.1% 

-9.4%  0.0% 

-4.5%  -21.6%  -13.9%  -10.2%  3.3% 146.0%  -20.0%  0.0% 

-6.3%  -14.9%  -14.7%  75.1% 

-6.0%  8.6%  6.2% 

-1.0% 

-5.2%  -22.7%  -15.3%  1.3% 

57 

43.1% 

54.1% 

-- 

 
 
 
 
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
 
 
  ANNEX B. Outliers treatment results for EU Member States only 

Results compared to outliers’ treatment of both EU Member States and non-EU countries.  

•  Box-plot:  Netherlands  and  Luxembourg  switch  places,  with  Netherlands  now  being  placed 
above  Luxembourg.  Similarly,  Czechia  and  Lithuania  switch  places  with  Czechia  now  being 
placed above Lithuania.  

•  MAD:  Greece  and  Lithuania  switch  places,  with  Greece  now  being  placed  above  Lithuania. 

No other changes. 

•  Z-Score (+/- 3 cut-off values): Belgium and Denmark switch places, with Belgium now being 

place above Denmark. No other changes. 

Table  30  EIS  2021  results  using  the  current  method  vs.  the  Box-plot  method  –  EU  Member 
States only 

Country 
Sweden 
Finland 
Denmark 
Belgium 
Netherlands 
Luxembourg 
Estonia 
Germany 
Austria 
France 
Ireland 
Cyprus 
Italy 
Slovenia 
Portugal 
Malta 
Spain 
Czechia 
Lithuania 
Greece 
Croatia 
Hungary 
Poland 
Slovakia 
Latvia 
Bulgaria 
Romania 

SII 
144.5 
138.7 
134.3 
130 
126.3 
125.4 
122.6 
121.3 
117.8 
106.6 
105.6 
103.1 
93.9 
91.3 
90 
89.3 
88.4 
81.1 
80.9 
78.7 
68.7 
66.7 
62.2 
61.5 
56 
46.5 
29.6 

Box-plot 

 Performance group 

Innovation Leader 
Innovation Leader 
Innovation Leader 
Innovation Leader 
Innovation Leader 
Innovation Leader 

Rank 
1 
2 
3 
4 
5 
6 
7  Strong Innovator 
8  Strong Innovator 
9  Strong Innovator 
10  Strong Innovator 
11  Strong Innovator 
12  Strong Innovator 
13  Moderate Innovator 
14  Moderate Innovator 
15  Moderate Innovator 
16  Moderate Innovator 
17  Moderate Innovator 
18  Moderate Innovator 
19  Moderate Innovator 
20  Moderate Innovator 
21  Emerging Innovator 
22  Emerging Innovator 
23  Emerging Innovator 
24  Emerging Innovator 
25  Emerging Innovator 
26  Emerging Innovator 
27  Emerging Innovator 

EIS 2021 results 
Rank   Performance group 

Innovation Leader 
1 
Innovation Leader 
2 
Innovation Leader 
3 
Innovation Leader 
4 
Strong Innovator 
6 
Strong Innovator 
5 
Strong Innovator 
7 
Strong Innovator 
8 
9 
Strong Innovator 
10  Strong Innovator 
11  Strong Innovator 
12  Strong Innovator 
13  Moderate Innovator 
14  Moderate Innovator 
15  Moderate Innovator 
16  Moderate Innovator 
17  Moderate Innovator 
18  Moderate Innovator 
20  Moderate Innovator 
19  Moderate Innovator 
21  Moderate Innovator 
22  Emerging Innovator 
23  Emerging Innovator 
24  Emerging Innovator 
25  Emerging Innovator 
26  Emerging Innovator 
27  Emerging Innovator 

SII 
137.5 
134.1 
128.4 
127.9 
123.2 
123.8 
119.5 
119 
116.5 
107.8 
106.2 
100.5 
95.3 
91.4 
90 
89.4 
86.5 
81.6 
80.9 
81.1 
70.9 
68.2 
62.6 
61.6 
58.2 
47.1 
33.6 

Differences 
SII  Outliers  Rank 
22 
6.9 
8 
4.6 
5 
5.9 
5 
2.1 
0 
3 
2 
1.7 
-8 
3 
5 
2.3 
1 
1.3 
0 
-1.2 
5 
-0.6 
2 
2.6 
1 
-1.4 
6 
-0.1 
0 
0 
0 
-0.1 
0 
1.9 
0 
-0.5 
-4 
0 
-1 
-2.4 
0 
-2.2 
3 
-1.5 
0 
-0.4 
0 
-0.1 
0 
-2.2 
2 
-0.6 
15 
-4 

0 
0 
0 
0 
1 
-1 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
1 
-1 
0 
0 
0 
0 
0 
0 
0 

58 

 
 
 
  
 
 
 
Table 31 EIS 2021 results using the current method vs. the MAD method  – EU Member States 
only 

Country 
Sweden 
Finland 
Denmark 
Belgium 
Netherlands 
Luxembourg 
Estonia 
Germany 
Austria 
France 
Ireland 
Cyprus 
Italy 
Portugal 
Slovenia 
Malta 
Spain 
Czechia 
Greece 
Lithuania 
Croatia 
Hungary 
Poland 
Slovakia 
Latvia 
Bulgaria 
Romania 

SII 
142.9 
135.8 
132.3 
129.2 
125.9 
123.4 
121.8 
121.3 
119 
106.9 
106.3 
105.4 
93.9 
91.2 
90.7 
87.3 
86.4 
82.2 
79.6 
78.8 
69.8 
68.8 
64.5 
61.6 
59.5 
47.4 
31.9 

MAD 

Rank   Performance group 

Innovation Leader 
1 
Innovation Leader 
2 
Innovation Leader 
3 
Innovation Leader 
4 
Innovation Leader 
5 
Strong Innovator 
6 
Strong Innovator 
7 
Strong Innovator 
8 
9 
Strong Innovator 
10  Strong Innovator 
11  Strong Innovator 
12  Strong Innovator 
13  Moderate Innovator 
14  Moderate Innovator 
15  Moderate Innovator 
16  Moderate Innovator 
17  Moderate Innovator 
18  Moderate Innovator 
19  Moderate Innovator 
20  Moderate Innovator 
21  Emerging Innovator 
22  Emerging Innovator 
23  Emerging Innovator 
24  Emerging Innovator 
25  Emerging Innovator 
26  Emerging Innovator 
27  Emerging Innovator 

EIS 2021 results 

Rank   Performance group 

Innovation Leader 
1 
Innovation Leader 
2 
Innovation Leader 
3 
Innovation Leader 
4 
Strong Innovator 
6 
Strong Innovator 
5 
Strong Innovator 
7 
Strong Innovator 
8 
9 
Strong Innovator 
10  Strong Innovator 
11  Strong Innovator 
12  Strong Innovator 
13  Moderate Innovator 
15  Moderate Innovator 
14  Moderate Innovator 
16  Moderate Innovator 
17  Moderate Innovator 
18  Moderate Innovator 
19  Moderate Innovator 
20  Moderate Innovator 
21  Moderate Innovator 
22  Emerging Innovator 
23  Emerging Innovator 
24  Emerging Innovator 
25  Emerging Innovator 
26  Emerging Innovator 
27  Emerging Innovator 

SII 
137.5 
134.1 
128.4 
127.9 
123.2 
123.8 
119.5 
119 
116.5 
107.8 
106.2 
100.5 
95.3 
90 
91.4 
89.4 
86.5 
81.6 
81.1 
80.9 
70.9 
68.2 
62.6 
61.6 
58.2 
47.1 
33.6 

SII 
5.4 
1.8 
3.9 
1.2 
2.7 
-0.4 
2.3 
2.3 
2.6 
-0.9 
0.2 
4.9 
-1.4 
1.2 
-0.6 
-2 
-0.1 
0.6 
-1.5 
-2.1 
-1.1 
0.6 
2 
0 
1.3 
0.3 
-1.7 

Differences 

Outliers  Rank 

3 
-1 
-13 
0 
-10 
-4 
-12 
-8 
-3 
-1 
0 
-5 
-4 
-1 
3 
0 
0 
0 
-1 
-5 
0 
-1 
-1 
0 
0 
0 
12 

0 
0 
0 
0 
1 
-1 
0 
0 
0 
0 
0 
0 
0 
1 
-1 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 

Table 32 EIS 2021 results using the current method vs. the Z-score (+/-3 cut-off values) method 
– EU Member States only 

Z-score (+/-3 cut-off values) 

Country 
Sweden 
Finland 
Denmark 
Belgium 
Netherlands 
Luxembourg 
Germany 
Estonia 
Austria 
France 
Ireland 
Cyprus 
Italy 
Slovenia 
Portugal 
Malta 
Spain 
Czechia 
Lithuania 
Greece 
Croatia 
Hungary 
Poland 
Slovakia 
Latvia 
Bulgaria 
Romania 

SII 
142.6 
134.8 
132.1 
128.9 
125.5 
123.9 
120.8 
120.4 
117.4 
106.6 
106.6 
103.9 
93.7 
90.7 
90.2 
89.8 
86.8 
82.6 
79.3 
78.9 
68.8 
68 
62.7 
62.3 
57.9 
46.2 
30.6 

Rank   Performance group 

Innovation Leader 
1 
Innovation Leader 
2 
Innovation Leader 
3 
Innovation Leader 
4 
Innovation Leader 
5 
Strong Innovator 
6 
Strong Innovator 
7 
Strong Innovator 
8 
Strong Innovator 
9 
10  Strong Innovator 
11  Strong Innovator 
12  Strong Innovator 
13  Moderate Innovator 
14  Moderate Innovator 
15  Moderate Innovator 
16  Moderate Innovator 
17  Moderate Innovator 
18  Moderate Innovator 
19  Moderate Innovator 
20  Moderate Innovator 
21  Emerging Innovator 
22  Emerging Innovator 
23  Emerging Innovator 
24  Emerging Innovator 
25  Emerging Innovator 
26  Emerging Innovator 
27  Emerging Innovator 

SII 
137.5 
134.1 
128.4 
127.9 
123.2 
123.8 
119 
119.5 
116.5 
107.8 
106.2 
100.5 
95.3 
91.4 
90 
89.4 
86.5 
81.6 
80.9 
81.1 
70.9 
68.2 
62.6 
61.6 
58.2 
47.1 
33.6 

EIS 2021 results 

Rank   Performance group 

Innovation Leader 
1 
Innovation Leader 
2 
Innovation Leader 
3 
Innovation Leader 
4 
Strong Innovator 
6 
Strong Innovator 
5 
Strong Innovator 
8 
Strong Innovator 
7 
Strong Innovator 
9 
10  Strong Innovator 
11  Strong Innovator 
12  Strong Innovator 
13  Moderate Innovator 
14  Moderate Innovator 
15  Moderate Innovator 
16  Moderate Innovator 
17  Moderate Innovator 
18  Moderate Innovator 
20  Moderate Innovator 
19  Moderate Innovator 
21  Moderate Innovator 
22  Emerging Innovator 
23  Emerging Innovator 
24  Emerging Innovator 
25  Emerging Innovator 
26  Emerging Innovator 
27  Emerging Innovator 

Differences 

Outliers  Rank 

33 
15 
10 
7 
4 
11 
5 
10 
1 
7 
5 
5 
1 
6 
6 
3 
1 
0 
0 
0 
0 
3 
0 
0 
7 
2 
20 

0 
0 
0 
0 
1 
-1 
1 
-1 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
1 
-1 
0 
0 
0 
0 
0 
0 
0 

SII 
5 
0.8 
3.7 
0.9 
2.3 
0.1 
1.8 
0.9 
0.9 
-1.2 
0.4 
3.4 
-1.6 
-0.7 
0.2 
0.4 
0.3 
0.9 
-1.6 
-2.2 
-2.1 
-0.2 
0.1 
0.6 
-0.3 
-0.9 
-3 

59 

 
 
  
 
  
 
 
 
  ANNEX C. Box-plot overviews EIS 

A Box-plot is composed of several elements, including a box that spans the interquartile range (IQR), 
whiskers that extend from the box to the minimum and maximum values within 1.5 times the IQR, and 
individual points beyond the whiskers that are considered outliers. The Box-plot overview compares 4 
outlier treatment methods for each of the indicators included in the EIS and includes no treatment of 
outliers  as  baseline.  Any  dots  above  or  below  the  whiskers  for  the  4  outlier  treatment  methods 
indicate that there are still outliers remaining after treatment. The overview is grouped by the four EIS 
dimensions: Framework conditions, Investments, Innovation activities and Impacts. 

60 

 
 
 
 
Figure 5 provides a visual overview of the country dataset distribution for each of the indicators that 
are  part  of  the  Framework  conditions  dimension.  The  Framework  condition  dimension  includes  8 
indicators, of which 5 include outliers. 

Figure 5. Outlier treatment by method and indicator for the EIS 2021: Framework conditions  

Note: No treatment is original raw data distribution. Original approach is the Z-score method with cut-off values 
+/-2. Z-score refers to Z-score method with cut-off values +/-3. 

61 

 
 
 
 
Figure  6  provides  a  visual  overview  of  the  distribution  for  each  of  the  indicators  that  are  part  of  the 
Investments dimension. The Investments dimension includes 8 indicators, of which 5 include outliers.  

Figure 6. Outlier treatment by method and indicator for the EIS 2021: Investments 

Note: No treatment is original raw data distribution. Original approach is the Z-score method with cut-off values 
+/-2. Z-score refers to Z-score method with cut-off values +/-3. 

62 

 
 
 
 
Figure 7 provides a visual overview of the country dataset distribution for each of the indicators that 
are  part  of  the  Innovation  Activities  dimension.  The  Innovation  Activities  dimension  includes  8 
indicators, of which 6 include outliers.  

Figure 7. Outlier treatment by method and indicator for the EIS 2021: Innovation activities 

Note: No treatment is original raw data distribution. Original approach is the Z-score method with cut-off values 
+/-2. Z-score refers to Z-score method with cut-off values +/-3. 

63 

 
 
 
Figure 8 provides a visual overview of the country dataset distribution for each of the indicators that 
are  part  of  the  Impact  dimension.  The  Impact  dimension  includes  8  indicators,  of  which  7  include 
outliers.  

Figure 8. Outlier treatment by method and indicator for the EIS 2021: Impacts 

Note: No treatment is original raw data distribution. Original approach is the Z-score method with cut-off values 
+/-2. Z-score refers to Z-score method with cut-off values +/-3. 

64 

 
 
 
 
 
  ANNEX D. RIS 2021 results by region 

Table 33 RIS 2021 results using the current method (z-score +/- 2 cut-off values) vs. no outlier 
treatment 

Region 
BE 

Jihozápad 

Jihovýchod 

Belgium 
Région de Bruxelles-
Capitale / Brussels 
Hoofdstedelijk Gewest 
BE1 
Vlaams Gewest 
BE2 
Région Wallonne 
BE3 
BG 
Bulgaria 
BG31  Severozapaden 
BG32  Severen tsentralen 
BG33  Severoiztochen 
BG34  Yugoiztochen 
BG41  Yugozapaden 
BG42  Yuzhen tsentralen 
Czechia 
CZ 
CZ01  Praha 
CZ02  Strední Cechy 
CZ03 
CZ04  Severozápad 
CZ05  Severovýchod 
CZ06 
CZ07  Strední Morava 
CZ08  Moravskoslezsko 
Denmark 
DK 
DK01  Hovedstaden 
DK02  Sjælland 
DK03  Syddanmark 
DK04  Midtjylland 
DK05  Nordjylland 
DE 
Germany 
DE11  Stuttgart 
DE12  Karlsruhe 
DE13  Freiburg 
DE14  Tübingen 
DE21  Oberbayern 
DE22  Niederbayern 
DE23  Oberpfalz 
DE24  Oberfranken 
DE25  Mittelfranken 
DE26  Unterfranken 
DE27  Schwaben 
Berlin 
DE3 
Brandenburg 
DE4 
Bremen 
DE5 
DE6 
Hamburg 
DE71  Darmstadt 
DE72  Gießen 
DE73  Kassel 

Mecklenburg-
DE8 
Vorpommern 
DE91  Braunschweig 
DE92  Hannover 
DE93  Lüneburg 
DE94  Weser-Ems 

No outlier treatment 
Rank  Performance group  RII 

RII 

RIS 2021 results 

Rank  Performance group 

Dif. 
RII 

131.2 
125.5 
109.8 

18 Innovation Leader 
29 Innovation Leader 
69 Strong Innovator 

137.6 
131.9 
113.1 

14 Innovation Leader 
25 Innovation Leader 
69 Strong Innovator 

36.7 
44.3 
43.9 
36.3 
67.3 
45.3 

106.7 
86.8 
74.4 
52.4 
79.6 
87.9 
76.8 
81.0 

141.7 
96.6 
104.9 
122.7 
111.6 

129.3 
134.2 
118.9 
127.6 
142.0 
98.2 
113.7 
107.9 
122.0 
116.2 
105.8 
138.4 
97.1 
112.6 
127.0 
121.3 
118.2 
98.7 

93.6 
133.4 
114.2 
92.4 
87.9 

234 Emerging Innovator 
229 Emerging Innovator 
230 Emerging Innovator 
235 Emerging Innovator 
190 Emerging Innovator 
228 Emerging Innovator 

29.1 
36.7 
36.2 
27.8 
59.8 
37.9 

234 Emerging Innovator 
229 Emerging Innovator 
230 Emerging Innovator 
235 Emerging Innovator 
191 Emerging Innovator 
228 Emerging Innovator 

77 Strong Innovator 
145 Moderate Innovator 
172 Moderate Innovator 
222 Emerging Innovator 
162 Moderate Innovator 
144 Moderate Innovator 
168 Moderate Innovator 
157 Moderate Innovator 

5 Innovation Leader 
120 Moderate Innovator 
87 Strong Innovator 
34 Strong Innovator 
65 Strong Innovator 

21 Innovation Leader 
11 Innovation Leader 
43 Strong Innovator 
22 Innovation Leader 
4 Innovation Leader 
114 Moderate Innovator 
57 Strong Innovator 
73 Strong Innovator 
35 Strong Innovator 
50 Strong Innovator 
83 Strong Innovator 

7 Innovation Leader 
119 Moderate Innovator 
61 Strong Innovator 
24 Innovation Leader 
37 Strong Innovator 
45 Strong Innovator 
109 Moderate Innovator 

126 Moderate Innovator 
12 Innovation Leader 
54 Strong Innovator 
129 Moderate Innovator 
142 Moderate Innovator 

65 

108.3 
89.2 
72.4 
48.1 
80.4 
90.3 
76.6 
81.0 

149.2 
98.8 
107.6 
131.5 
115.0 

130.4 
145.0 
125.3 
134.9 
154.1 
99.3 
117.4 
111.7 
130.0 
120.1 
107.9 
149.3 
97.9 
114.0 
136.1 
129.5 
125.0 
98.7 

94.1 
135.5 
118.4 
90.9 
86.1 

80 Strong Innovator 
134 Moderate Innovator 
169 Moderate Innovator 
218 Emerging Innovator 
154 Moderate Innovator 
131 Moderate Innovator 
161 Moderate Innovator 
151 Moderate Innovator 

5 Innovation Leader 
106 Moderate Innovator 
86 Strong Innovator 
26 Innovation Leader 
63 Strong Innovator 

32 Innovation Leader 
7 Innovation Leader 
40 Innovation Leader 
19 Innovation Leader 
3 Innovation Leader 
105 Moderate Innovator 
53 Strong Innovator 
73 Strong Innovator 
34 Innovation Leader 
48 Strong Innovator 
83 Strong Innovator 

4 Innovation Leader 
114 Moderate Innovator 
67 Strong Innovator 
15 Innovation Leader 
35 Innovation Leader 
41 Innovation Leader 
110 Moderate Innovator 

122 Moderate Innovator 
17 Innovation Leader 
51 Strong Innovator 
130 Moderate Innovator 
142 Moderate Innovator 

-6.4 
-6.4 
-3.4 

7.6 
7.6 
7.7 
8.5 
7.5 
7.5 

-1.6 
-2.3 
2.0 
4.2 
-0.8 
-2.4 
0.2 
0.0 

-7.5 
-2.2 
-2.7 
-8.9 
-3.4 

-1.1 
-10.8 
-6.4 
-7.3 
-12.0 
-1.1 
-3.6 
-3.8 
-8.0 
-4.0 
-2.1 
-10.9 
-0.8 
-1.4 
-9.2 
-8.2 
-6.8 
0.1 

-0.5 
-2.1 
-4.2 
1.5 
1.8 

 
 
 
 
   
  
 
 
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
Region 
DEA1  Düsseldorf 
DEA2  Köln 
DEA3  Münster 
DEA4  Detmold 
DEA5  Arnsberg 
DEB1  Koblenz 
DEB2  Trier 
DEB3  Rheinhessen-Pfalz 
DEC 
Saarland 
DED2  Dresden 
DED4  Chemnitz 
DED5  Leipzig 
Sachsen-Anhalt 
DEE 
Schleswig-Holstein 
DEF 
Thüringen 
DEG 
Estonia 
EE 
Ireland 
IE 
Northern and Western 
IE04 
Southern 
IE05 
Eastern and Midland 
IE06 
Greece 
EL 
EL3 
Attiki 
EL41  Voreio Aigaio 
EL42  Notio Aigaio 
EL43  Kriti 

Anatoliki Makedonia, 
Thraki 

EL51 
EL52  Kentriki Makedonia 
EL53  Dytiki Makedonia 
Ipeiros 
EL54 
EL61  Thessalia 
EL62 
Ionia Nisia 
EL63  Dytiki Ellada 
EL64  Sterea Ellada 
EL65  Peloponnisos 
ES 
Spain 
ES11  Galicia 
ES12  Principado de Asturias 
ES13  Cantabria 
ES21  País Vasco 

Comunidad de Madrid 

Comunidad Foral de 
ES22 
Navarra 
ES23  La Rioja 
ES24  Aragón 
ES3 
ES41  Castilla y León 
ES42  Castilla-la Mancha 
ES43  Extremadura 
ES51  Cataluña 
ES52  Comunidad Valenciana 
ES53 
ES61  Andalucía 
ES62  Región de Murcia 

Illes Balears 

Ciudad Autónoma de 
Ceuta 
Ciudad Autónoma de 
Melilla 
Canarias 
France 
Île de France 

ES63 

ES64 
ES7 
FR 
FR1 

No outlier treatment 
Rank  Performance group  RII 

82 Strong Innovator 
33 Strong Innovator 
84 Strong Innovator 
59 Strong Innovator 
74 Strong Innovator 
113 Moderate Innovator 
97 Strong Innovator 
28 Innovation Leader 
81 Strong Innovator 
36 Strong Innovator 

108.9 
130.6 
107.8 
117.9 
110.6 
97.5 
98.7 
134.8 
106.3 
127.7 
107 Moderate Innovator  101.3 
120.2 
94.2 
108.2 
107.6 

52 Strong Innovator 
127 Moderate Innovator 
80 Strong Innovator 
89 Strong Innovator 

RIS 2021 results 

Rank  Performance group 
76 Strong Innovator 
29 Innovation Leader 
84 Strong Innovator 
52 Strong Innovator 
74 Strong Innovator 
115 Moderate Innovator 
109 Moderate Innovator 
20 Innovation Leader 
93 Strong Innovator 
38 Innovation Leader 

101 Strong Innovator 
47 Strong Innovator 
121 Moderate Innovator 
81 Strong Innovator 
85 Strong Innovator 

RII 
106.0 
123.7 
105.6 
113.3 
107.5 
98.2 
100.8 
125.7 
106.1 
122.0 
99.4 
115.5 
93.6 
106.2 
104.4 

94.0 
100.6 
110.3 

125 Moderate Innovator 
98 Strong Innovator 
68 Strong Innovator 

95.1 
103.6 
115.6 

120 Moderate Innovator 
97 Strong Innovator 
60 Strong Innovator 

90.4 
69.4 
51.5 
80.8 

64.7 
85.7 
57.6 
72.3 
75.3 
59.7 
74.5 
62.7 
60.0 

81.5 
79.8 
79.5 
104.8 

97.8 
80.8 
82.9 
100.4 
78.2 
68.5 
63.5 
100.0 
89.8 
69.2 
72.6 
77.5 

134 Moderate Innovator 
184 Emerging Innovator 
224 Emerging Innovator 
158 Moderate Innovator 

196 Emerging Innovator 
148 Moderate Innovator 
217 Emerging Innovator 
175 Moderate Innovator 
170 Moderate Innovator 
213 Emerging Innovator 
171 Moderate Innovator 
203 Emerging Innovator 
209 Emerging Innovator 

89.6 
64.3 
46.4 
82.5 

61.1 
84.7 
52.1 
68.8 
74.2 
58.9 
69.2 
59.8 
55.6 

133 Moderate Innovator 
181 Emerging Innovator 
221 Emerging Innovator 
148 Moderate Innovator 

186 Emerging Innovator 
144 Moderate Innovator 
212 Emerging Innovator 
172 Emerging Innovator 
167 Moderate Innovator 
193 Emerging Innovator 
171 Emerging Innovator 
190 Emerging Innovator 
205 Emerging Innovator 

156 Moderate Innovator 
161 Moderate Innovator 
163 Moderate Innovator 
88 Strong Innovator 

78.1 
75.7 
75.0 
106.7 

158 Moderate Innovator 
163 Moderate Innovator 
164 Moderate Innovator 
91 Strong Innovator 

98.8 
116 Moderate Innovator 
80.8 
159 Moderate Innovator 
79.9 
150 Moderate Innovator 
101.8 
102 Strong Innovator 
74.5 
165 Moderate Innovator 
62.8 
188 Emerging Innovator 
56.6 
200 Emerging Innovator 
104 Moderate Innovator  102.3 
91.5 
137 Moderate Innovator 
65.5 
186 Emerging Innovator 
67.1 
174 Moderate Innovator 
76.2 
166 Moderate Innovator 

107 Moderate Innovator 
152 Moderate Innovator 
156 Moderate Innovator 
100 Strong Innovator 
165 Moderate Innovator 
184 Emerging Innovator 
199 Emerging Innovator 
98 Strong Innovator 
128 Moderate Innovator 
180 Emerging Innovator 
177 Emerging Innovator 
162 Moderate Innovator 

39.3 

231 Emerging Innovator 

31.0 

232 Emerging Innovator 

49.1 
59.9 

226 Emerging Innovator 
211 Emerging Innovator 

41.8 
50.9 

226 Emerging Innovator 
213 Emerging Innovator 

Dif. 
RII 
-2.9 
-7.0 
-2.1 
-4.6 
-3.1 
0.7 
2.1 
-9.1 
-0.2 
-5.8 
-1.9 
-4.7 
-0.6 
-2.0 
-3.2 

-1.1 
-3.0 
-5.4 

0.8 
5.1 
5.1 
-1.7 

3.6 
1.0 
5.5 
3.5 
1.1 
0.7 
5.3 
2.8 
4.5 

3.4 
4.1 
4.5 
-1.8 

-1.0 
-0.1 
3.0 
-1.4 
3.8 
5.7 
6.9 
-2.3 
-1.7 
3.6 
5.6 
1.4 

8.3 

7.3 
8.9 

129.9 

20 Innovation Leader 

134.7 

21 Innovation Leader 

-4.8 

66 

 
 
 
   
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
Region 
FRB 

Centre - Val de Loire 
Bourgogne - Franche-
Comté 

Hauts-de-France 

FRC 
FRD  Normandie 
FRE 
FRF  Grand Est 
FRG 
FRH 
FRI 
FRJ 

Pays de la Loire 
Bretagne 
Nouvelle-Aquitaine 
Occitanie 
Auvergne - Rhône-
Alpes 
Provence-Alpes-Côte 
FRL 
d'Azur 
FRM  Corse 

FRK 

Régions ultra-
périphériques 
françaises 
Croatia 

ITC2 
ITC3 
ITC4 

ITH1 

FRY 
HR 
HR02  Panonska Hrvatska 
HR03  Jadranska Hrvatska 
HR05  Grad Zagreb 
HR06  Sjeverna Hrvatska 
IT 
ITC1 

Italy 
Piemonte 
Valle d'Aosta/Vallée 
d'Aoste 
Liguria 
Lombardia 
Provincia Autonoma 
Bolzano/Bozen 
Provincia Autonoma 
Trento 
Veneto 
Friuli-Venezia Giulia 
Emilia-Romagna 
Toscana 
Umbria 
Marche 
Lazio 
Abruzzo 

ITH2 
ITH3 
ITH4 
ITH5 
ITI1 
ITI2 
ITI3 
ITI4 
ITF1 
ITF2  Molise 
ITF3 
ITF4 
ITF5 
ITF6 
ITG1  Sicilia 
ITG2  Sardegna 
Cyprus 
CY 
Latvia 
LV 
LT 
Lithuania 
LT01  Sostinės regionas 

Campania 
Puglia 
Basilicata 
Calabria 

Vidurio ir vakarų 
Lietuvos regionas  
LT02 
Luxembourg 
LU 
HU 
Hungary 
HU11  Budapest 
HU12  Pest 
HU21  Közép-Dunántúl 
HU22  Nyugat-Dunántúl 

No outlier treatment 
Rank  Performance group  RII 

RII 

90.4 

136 Moderate Innovator 

87.2 

RIS 2021 results 

Rank  Performance group 
139 Moderate Innovator 

90.7 
81.5 
89.4 
96.6 
100.5 
106.3 
93.6 
112.2 

132 Moderate Innovator 
154 Moderate Innovator 
138 Moderate Innovator 
122 Moderate Innovator 
101 Strong Innovator 
79 Strong Innovator 
128 Moderate Innovator 
62 Strong Innovator 

88.2 
76.7 
85.8 
95.4 
98.7 
108.5 
91.8 
116.9 

136 Moderate Innovator 
160 Moderate Innovator 
143 Moderate Innovator 
119 Moderate Innovator 
111 Moderate Innovator 
79 Strong Innovator 
127 Moderate Innovator 
54 Strong Innovator 

113.7 

58 Strong Innovator 

116.7 

55 Strong Innovator 

105.0 
52.9 

86 Strong Innovator 
221 Emerging Innovator 

106.8 
44.7 

88 Strong Innovator 
223 Emerging Innovator 

Dif. 
RII 
3.2 

2.6 
4.9 
3.7 
1.2 
1.8 
-2.2 
1.8 
-4.7 

-3.1 

-1.8 
8.2 

82.0 

153 Moderate Innovator 

69.5 

170 Emerging Innovator 

12.4 

85.9 
65.0 
90.7 
88.3 

147 Moderate Innovator 
194 Emerging Innovator 
133 Moderate Innovator 
141 Moderate Innovator 

82.9 
61.4 
89.0 
86.2 

146 Moderate Innovator 
185 Emerging Innovator 
135 Moderate Innovator 
141 Moderate Innovator 

98.6 

110 Moderate Innovator  102.1 

99 Strong Innovator 

65.5 
86.5 
103.6 

192 Emerging Innovator 
146 Moderate Innovator 
91 Strong Innovator 

63.0 
86.3 
107.9 

183 Emerging Innovator 
140 Moderate Innovator 
82 Strong Innovator 

3.0 
3.6 
1.7 
2.1 

-3.6 

2.5 
0.1 
-4.3 

91.1 

130 Moderate Innovator 

93.2 

126 Moderate Innovator 

-2.1 

103.3 
103.6 
102.5 
106.8 
98.5 
97.9 
87.9 
98.3 
84.9 
81.5 
82.8 
75.4 
78.6 
69.2 
71.1 
71.2 

94 Strong Innovator 
92 Strong Innovator 
95 Strong Innovator 
76 Strong Innovator 

106.7 
108.7 
107.0 
112.0 
111 Moderate Innovator  100.9 
98.6 
115 Moderate Innovator 
143 Moderate Innovator 
89.7 
112 Moderate Innovator  100.8 
83.7 
149 Moderate Innovator 
80.5 
155 Moderate Innovator 
82.6 
151 Moderate Innovator 
74.2 
169 Moderate Innovator 
77.4 
164 Moderate Innovator 
66.1 
185 Emerging Innovator 
68.6 
178 Moderate Innovator 
67.2 
177 Moderate Innovator 

89 Strong Innovator 
78 Strong Innovator 
87 Strong Innovator 
71 Strong Innovator 
102 Strong Innovator 
112 Moderate Innovator 
132 Moderate Innovator 
103 Strong Innovator 
145 Moderate Innovator 
153 Moderate Innovator 
147 Moderate Innovator 
166 Moderate Innovator 
159 Moderate Innovator 
178 Emerging Innovator 
173 Emerging Innovator 
176 Emerging Innovator 

-3.4 
-5.1 
-4.5 
-5.3 
-2.4 
-0.7 
-1.8 
-2.5 
1.2 
1.0 
0.2 
1.2 
1.2 
3.1 
2.5 
4.0 

99.2 

108 Moderate Innovator  103.9 

95 Strong Innovator 

-4.7 

69.6 

182 Emerging Innovator 

67.6 

174 Emerging Innovator 

1.9 

99.8 
71.8 
63.7 
61.3 

105 Moderate Innovator 
176 Moderate Innovator 
199 Emerging Innovator 
206 Emerging Innovator 

99.7 
67.5 
57.9 
54.9 

104 Moderate Innovator 
175 Emerging Innovator 
195 Emerging Innovator 
206 Emerging Innovator 

0.1 
4.3 
5.8 
6.4 

67 

 
 
 
   
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
Malta 
Netherlands 

Region 
HU23  Dél-Dunántúl 
HU31  Észak-Magyarország 
HU32  Észak-Alföld 
HU33  Dél-Alföld 
MT 
NL 
NL11  Groningen 
NL12  Friesland 
NL13  Drenthe 
NL21  Overijssel 
NL22  Gelderland 
NL23  Flevoland 
NL31  Utrecht 
NL32  Noord-Holland 
NL33  Zuid-Holland 
NL34  Zeeland 
NL41  Noord-Brabant 
Limburg 
NL42 
Austria 
AT 
Ostösterreich 
AT1 
Südösterreich 
AT2 
AT3  Westösterreich 
PL 
Poland 
PL21  Malopolskie 
PL22  Slaskie 
PL41  Wielkopolskie 
PL42  Zachodniopomorskie 
PL43 
PL51  Dolnoslaskie 
PL52  Opolskie 
PL61  Kujawsko-Pomorskie 
PL62  Warminsko-Mazurskie 
PL63  Pomorskie 
PL71 
Lódzkie 
PL72  Swietokrzyskie 
PL81 
PL82  Podkarpackie 
PL84  Podlaskie 
PL91  Warszawski stoleczny 
PL92  Mazowiecki regionalny 
PT 
PT11  Norte 
PT15  Algarve 
PT16  Centro 
PT17 
Lisboa 
PT18  Alentejo 

Lubelskie 

Lubuskie 

Portugal 

PT2 

Região Autónoma dos 
Açores 
Região Autónoma da 
Madeira 
PT3 
RO 
Romania 
RO11  Nord-Vest 
RO12  Centru 
RO21  Nord-Est 
RO22  Sud-Est 
RO31  Sud - Muntenia 
RO32  Bucuresti - Ilfov 
RO41  Sud-Vest Oltenia 
RO42  Vest 
SI 

Slovenia 

No outlier treatment 
Rank  Performance group  RII 

RII 

57.2 
58.1 
59.3 
64.0 

218 Emerging Innovator 
215 Emerging Innovator 
214 Emerging Innovator 
198 Emerging Innovator 

50.1 
52.2 
52.2 
58.5 

RIS 2021 results 

Rank  Performance group 
216 Emerging Innovator 
211 Emerging Innovator 
210 Emerging Innovator 
194 Emerging Innovator 

120.8 
99.5 
96.6 
111.9 
119.7 
107.1 
131.9 
126.6 
121.3 
97.4 
126.3 
118.7 

114.5 
109.6 
106.6 

80.1 
64.8 
62.4 
55.6 
54.9 
73.6 
59.9 
59.8 
51.7 
70.1 
64.5 
51.2 
62.3 
65.2 
57.8 
96.1 
46.7 

39 Strong Innovator 
106 Moderate Innovator 
121 Moderate Innovator 
63 Strong Innovator 
42 Strong Innovator 
75 Strong Innovator 
17 Innovation Leader 
25 Innovation Leader 
38 Strong Innovator 
118 Moderate Innovator 
26 Innovation Leader 
44 Strong Innovator 

53 Strong Innovator 
70 Strong Innovator 
78 Strong Innovator 

160 Moderate Innovator 
195 Emerging Innovator 
204 Emerging Innovator 
219 Emerging Innovator 
220 Emerging Innovator 
173 Moderate Innovator 
210 Emerging Innovator 
212 Emerging Innovator 
223 Emerging Innovator 
181 Moderate Innovator 
197 Emerging Innovator 
225 Emerging Innovator 
205 Emerging Innovator 
193 Emerging Innovator 
216 Emerging Innovator 
123 Moderate Innovator 
227 Emerging Innovator 

90.8 
69.4 
90.4 
103.5 
77.0 

131 Moderate Innovator 
183 Emerging Innovator 
135 Moderate Innovator 
93 Strong Innovator 
167 Moderate Innovator 

115.0 
96.0 
93.4 
112.1 
122.2 
108.9 
130.4 
130.3 
123.4 
94.1 
128.7 
121.6 

123.7 
116.4 
113.6 

73.3 
54.1 
53.4 
45.2 
46.6 
63.9 
50.8 
50.7 
42.9 
60.5 
55.7 
42.1 
52.7 
57.9 
47.5 
87.8 
38.2 

81.5 
55.6 
79.1 
94.1 
65.6 

64 Strong Innovator 
118 Moderate Innovator 
125 Moderate Innovator 
70 Strong Innovator 
45 Strong Innovator 
77 Strong Innovator 
31 Innovation Leader 
33 Innovation Leader 
43 Strong Innovator 
124 Moderate Innovator 
36 Innovation Leader 
46 Strong Innovator 

42 Strong Innovator 
57 Strong Innovator 
68 Strong Innovator 

168 Moderate Innovator 
207 Emerging Innovator 
208 Emerging Innovator 
222 Emerging Innovator 
220 Emerging Innovator 
182 Emerging Innovator 
214 Emerging Innovator 
215 Emerging Innovator 
224 Emerging Innovator 
187 Emerging Innovator 
203 Emerging Innovator 
225 Emerging Innovator 
209 Emerging Innovator 
196 Emerging Innovator 
219 Emerging Innovator 
138 Moderate Innovator 
227 Emerging Innovator 

150 Moderate Innovator 
204 Emerging Innovator 
157 Moderate Innovator 
123 Moderate Innovator 
179 Emerging Innovator 

Dif. 
RII 
7.1 
5.9 
7.1 
5.5 

5.8 
3.5 
3.2 
-0.2 
-2.5 
-1.8 
1.5 
-3.7 
-2.1 
3.3 
-2.4 
-2.8 

-9.2 
-6.8 
-6.9 

6.8 
10.8 
9.0 
10.4 
8.3 
9.7 
9.1 
9.1 
8.8 
9.6 
8.8 
9.1 
9.6 
7.3 
10.3 
8.3 
8.5 

9.3 
13.8 
11.3 
9.3 
11.4 

63.4 

201 Emerging Innovator 

49.9 

217 Emerging Innovator 

13.5 

70.9 

179 Moderate Innovator 

56.5 

200 Emerging Innovator 

14.4 

39.2 
33.3 
30.2 
23.1 
26.1 
66.8 
25.1 
37.9 

232 Emerging Innovator 
236 Emerging Innovator 
237 Emerging Innovator 
240 Emerging Innovator 
238 Emerging Innovator 
191 Emerging Innovator 
239 Emerging Innovator 
233 Emerging Innovator 

31.4 
26.5 
22.7 
17.9 
19.6 
59.5 
19.6 
30.9 

231 Emerging Innovator 
236 Emerging Innovator 
237 Emerging Innovator 
240 Emerging Innovator 
238 Emerging Innovator 
192 Emerging Innovator 
239 Emerging Innovator 
233 Emerging Innovator 

7.8 
6.8 
7.5 
5.2 
6.5 
7.3 
5.6 
7.0 

68 

 
 
 
   
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
Etelä-Suomi 
Länsi-Suomi 
Pohjois- ja Itä-Suomi 
Åland 
Sweden 

Region 
Vzhodna Slovenija 
SI03 
Zahodna Slovenija 
SI04 
Slovakia 
SK 
SK01  Bratislavský kraj 
SK02  Západné Slovensko 
SK03  Stredné Slovensko 
SK04  Východné Slovensko 
Finland 
FI 
FI1B  Helsinki-Uusimaa 
FI1C 
FI19 
FI1D 
FI2 
SE 
SE11  Stockholm 
SE12  Östra Mellansverige 
SE21  Småland med öarna 
SE22  Sydsverige 
SE23  Västsverige 
SE31  Norra Mellansverige 
SE32  Mellersta Norrland 
SE33  Övre Norrland 
NO 
Norway 
NO01  Oslo og Akershus 
NO02  Hedmark og Oppland 
NO03  Sør-Østlandet 
NO04  Agder og Rogaland 
NO05  Vestlandet 
NO06  Trøndelag 
NO07  Nord-Norge 
CH 
Switzerland 
CH01  Région lémanique 
CH02  Espace Mittelland 
CH03  Nordwestschweiz 
CH04  Zürich 
CH05  Ostschweiz 
CH06  Zentralschweiz 
CH07  Ticino 
Serbia 
RS 
RS11  Belgrade 
RS12  Vojvodina 

No outlier treatment 
Rank  Performance group  RII 

RII 

82.1 
97.6 

152 Moderate Innovator 
117 Moderate Innovator 

80.2 
98.0 

RIS 2021 results 

Rank  Performance group 
155 Moderate Innovator 
113 Moderate Innovator 

88.7 
60.2 
62.7 
60.7 

140 Moderate Innovator 
208 Emerging Innovator 
202 Emerging Innovator 
207 Emerging Innovator 

88.2 
56.7 
59.9 
57.3 

137 Moderate Innovator 
198 Emerging Innovator 
189 Emerging Innovator 
197 Emerging Innovator 

145.6 
114.0 
124.7 
112.8 
101.3 

153.9 
130.1 
110.4 
140.2 
135.8 
100.5 
100.5 
117.9 

137.8 
95.7 
104.4 
108.4 
116.1 
136.8 
100.1 

127.5 
120.0 
133.3 
146.5 
124.8 
124.5 
132.1 

3 Innovation Leader 

55 Strong Innovator 
31 Strong Innovator 
60 Strong Innovator 
96 Strong Innovator 

1 Innovation Leader 
19 Innovation Leader 
67 Strong Innovator 

6 Innovation Leader 
10 Innovation Leader 
99 Strong Innovator 
100 Strong Innovator 
46 Strong Innovator 

8 Innovation Leader 
124 Moderate Innovator 
90 Strong Innovator 
72 Strong Innovator 
51 Strong Innovator 

9 Innovation Leader 

103 Strong Innovator 

23 Innovation Leader 
40 Strong Innovator 
13 Innovation Leader 
2 Innovation Leader 

30 Strong Innovator 
32 Strong Innovator 
16 Innovation Leader 

154.3 
115.8 
130.9 
116.6 
105.2 

155.2 
131.2 
109.5 
144.5 
138.4 
96.8 
96.9 
116.3 

139.1 
91.0 
103.8 
106.7 
115.5 
130.5 
98.8 

133.1 
128.7 
139.2 
147.9 
133.3 
135.4 
144.5 

2 Innovation Leader 

59 Strong Innovator 
28 Innovation Leader 
56 Strong Innovator 
94 Strong Innovator 

1 Innovation Leader 
27 Innovation Leader 
75 Strong Innovator 

9 Innovation Leader 
13 Innovation Leader 
117 Moderate Innovator 
116 Moderate Innovator 
58 Strong Innovator 

12 Innovation Leader 
129 Moderate Innovator 
96 Strong Innovator 
90 Strong Innovator 
61 Strong Innovator 
30 Innovation Leader 
108 Moderate Innovator 

24 Innovation Leader 
37 Innovation Leader 
11 Innovation Leader 
6 Innovation Leader 
22 Innovation Leader 
18 Innovation Leader 
8 Innovation Leader 

89.4 
67.7 

139 Moderate Innovator 
189 Emerging Innovator 

82.1 
60.1 

149 Moderate Innovator 
188 Emerging Innovator 

Dif. 
RII 
2.0 
-0.5 

0.6 
3.5 
2.8 
3.4 

-8.7 
-1.8 
-6.2 
-3.8 
-3.9 

-1.3 
-1.2 
1.0 
-4.3 
-2.6 
3.7 
3.6 
1.6 

-1.3 
4.7 
0.6 
1.7 
0.6 
6.3 
1.3 

-5.6 
-8.6 
-5.9 
-1.5 
-8.5 
-11.0 
-12.4 

7.4 
7.7 

RS21 

Šumadija and Western 
Serbia 
Southern and Eastern 
Serbia 
United Kingdom 

RS22 
UK 
UKC  North East 
UKD  North West 

Yorkshire and The 
Humber 
UKE 
UKF 
East Midlands 
UKG  West Midlands 
East of England 
UKH 
London 
UKI 
South East 
UKJ 
UKK 
South West  
UKL  Wales 
UKM  Scotland 
UKN  Northern Ireland 

68.6 

187 Emerging Innovator 

55.8 

202 Emerging Innovator 

12.8 

70.7 

180 Moderate Innovator 

56.4 

201 Emerging Innovator 

14.3 

108.8 
111.8 

71 Strong Innovator 
64 Strong Innovator 

112.0 
114.8 

72 Strong Innovator 
65 Strong Innovator 

111.2 
113.9 
117.4 
126.1 
132.4 
132.6 
119.8 
116.9 
116.5 
105.2 

66 Strong Innovator 
56 Strong Innovator 
47 Strong Innovator 
27 Innovation Leader 
15 Innovation Leader 
14 Innovation Leader 
41 Strong Innovator 
48 Strong Innovator 
49 Strong Innovator 
85 Strong Innovator 

114.7 
120.0 
122.8 
133.2 
135.6 
140.3 
126.3 
115.2 
119.9 
106.4 

66 Strong Innovator 
49 Strong Innovator 
44 Strong Innovator 
23 Innovation Leader 
16 Innovation Leader 
10 Innovation Leader 
39 Innovation Leader 
62 Strong Innovator 
50 Strong Innovator 
92 Strong Innovator 

-3.2 
-3.0 

-3.5 
-6.0 
-5.4 
-7.2 
-3.1 
-7.7 
-6.5 
1.7 
-3.4 
-1.2 

Note: Performance groups or rank increases highlighted in green, decreases in red. 

69 

 
 
 
   
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
Table 34 RIS 2021 results using the current method (z-score +/- 2 cut-off values) vs.  Box-plot 
method 

Region 
BE  Belgium 

Box-plot results 

RIS 2021 results 

Differences 

RII  Rank  Performance group  RII  Rank  Performance group 

RII 

Out-
liers 

Région de Bruxelles-
Capitale / Brussels 
Hoofdstedelijk Gewest 

Strong Innovator 

Innovation Leader 

138.4  14 
133.7  21 
115.1  61 

Innovation Leader 
Innovation Leader 
Strong Innovator 

Strong Innovator 
Innovation Leader 
Strong Innovator 

147.4  6 
97.8  114  Moderate Innovator 
106.4  89 
131.5  27 
114.0  65 

32.9  234  Emerging Innovator 
41.0  229  Emerging Innovator 
40.8  230  Emerging Innovator 
31.8  235  Emerging Innovator 
66.2  185  Emerging Innovator 
42.8  228  Emerging Innovator 

104.8  94 
87.1  143  Moderate Innovator 
71.1  172  Moderate Innovator 
48.5  223  Emerging Innovator 
80.0  159  Moderate Innovator 
88.8  139  Moderate Innovator 
75.8  166  Moderate Innovator 
82.1  156  Moderate Innovator 

BE1 
BE2  Vlaams Gewest 
BE3  Région Wallonne 
BG  Bulgaria 
BG31  Severozapaden 
BG32  Severen tsentralen 
BG33  Severoiztochen 
BG34  Yugoiztochen 
BG41  Yugozapaden 
BG42  Yuzhen tsentralen 
CZ  Czech Republic 
CZ01  Praha 
CZ02  Strední Cechy 
CZ03  Jihozápad 
CZ04  Severozápad 
CZ05  Severovýchod 
CZ06  Jihovýchod 
CZ07  Strední Morava 
CZ08  Moravskoslezsko 
DK  Denmark 
DK01  Hovedstaden 
DK02  Sjælland 
DK03  Syddanmark 
DK04  Midtjylland 
DK05  Nordjylland 
DE  Germany 
130.6  31 
DE11  Stuttgart 
144.0  9 
DE12  Karlsruhe 
124.5  41 
DE13  Freiburg 
134.2  18 
DE14  Tübingen 
154.2  2 
DE21  Oberbayern 
98.0  112  Moderate Innovator 
DE22  Niederbayern 
116.1  58 
DE23  Oberpfalz 
110.2  73 
DE24  Oberfranken 
128.4  37 
DE25  Mittelfranken 
119.5  50 
DE26  Unterfranken 
106.9  84 
DE27  Schwaben 
152.1  4 
DE3  Berlin 
99.5  108  Moderate Innovator 
DE4  Brandenburg 
113.0  67 
DE5  Bremen 
134.0  20 
DE6  Hamburg 
128.2  39 
DE71  Darmstadt 
124.1  42 
DE72  Gießen 
DE73  Kassel 
97.4  115  Moderate Innovator 
DE8  Mecklenburg-Vorpommern  95.9  121  Moderate Innovator 
DE91  Braunschweig 
DE92  Hannover 
DE93  Lüneburg 
DE94  Weser-Ems 
DEA1  Düsseldorf 
DEA2  Köln 
DEA3  Münster 
DEA4  Detmold 

134.6  17 
117.5  54 
89.8  134  Moderate Innovator 
85.3  144  Moderate Innovator 
107.9  81 
129.7  34 
106.4  88 
117.2  57 

Strong Innovator 
Strong Innovator 
Innovation Leader 
Strong Innovator 
Strong Innovator 
Innovation Leader 

Innovation Leader 
Innovation Leader 
Strong Innovator 
Innovation Leader 
Innovation Leader 

Strong Innovator 
Innovation Leader 
Innovation Leader 
Strong Innovator 

Strong Innovator 
Innovation Leader 
Strong Innovator 
Strong Innovator 

Innovation Leader 
Strong Innovator 

70 

137.6 
131.9 
113.1 

14 Innovation Leader 
25 Innovation Leader 
69 Strong Innovator 

29.1  234 Emerging Innovator 
36.7  229 Emerging Innovator 
36.2  230 Emerging Innovator 
27.8  235 Emerging Innovator 
59.8  191 Emerging Innovator 
37.9  228 Emerging Innovator 

108.3 

80 Strong Innovator 

89.2  134 Moderate Innovator 
72.4  169 Moderate Innovator 
48.1  218 Emerging Innovator 
80.4  154 Moderate Innovator 
90.3  131 Moderate Innovator 
76.6  161 Moderate Innovator 
81.0  151 Moderate Innovator 

149.2 

5 Innovation Leader 

98.8  106 Moderate Innovator 

107.6 
131.5 
115.0 

130.4 
145.0 
125.3 
134.9 
154.1 

86 Strong Innovator 
26 Innovation Leader 
63 Strong Innovator 

32 Innovation Leader 
7 Innovation Leader 
40 Innovation Leader 
19 Innovation Leader 
3 Innovation Leader 

99.3  105 Moderate Innovator 

117.4 
111.7 
130.0 
120.1 
107.9 
149.3 

53 Strong Innovator 
73 Strong Innovator 
34 Innovation Leader 
48 Strong Innovator 
83 Strong Innovator 
4 Innovation Leader 

97.9  114 Moderate Innovator 

114.0 
136.1 
129.5 
125.0 

67 Strong Innovator 
15 Innovation Leader 
35 Innovation Leader 
41 Innovation Leader 

98.7  110 Moderate Innovator 
94.1  122 Moderate Innovator 

135.5 
118.4 

17 Innovation Leader 
51 Strong Innovator 

90.9  130 Moderate Innovator 
86.1  142 Moderate Innovator 

108.9 
130.6 
107.8 
117.9 

76 Strong Innovator 
29 Innovation Leader 
84 Strong Innovator 
52 Strong Innovator 

0.8 
1.8 
2.0 

3.8 
4.3 
4.6 
4.1 
6.4 
4.9 

-3.5 
-2.1 
-1.3 
0.3 
-0.4 
-1.5 
-0.8 
1.1 

-1.9 
-1.1 
-1.2 
0.0 
-1.0 

0.2 
-1.0 
-0.9 
-0.7 
0.1 
-1.3 
-1.2 
-1.5 
-1.6 
-0.6 
-1.0 
2.8 
1.6 
-0.9 
-2.2 
-1.3 
-0.9 
-1.3 
1.8 
-0.9 
-0.9 
-1.1 
-0.8 
-1.1 
-1.0 
-1.3 
-0.8 

-6 
-2 
-2 

7 
1 
2 
4 
1 
6 

3 
0 
-2 
-3 
0 
0 
0 
2 

15 
0 
0 
1 
2 

-2 
1 
1 
3 
11 
0 
0 
-1 
4 
-2 
0 
1 
0 
0 
0 
0 
0 
3 
0 
6 
1 
-2 
0 
0 
0 
0 
2 

 
 
 
 
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
Box-plot results 

RIS 2021 results 

Differences 

RII  Rank  Performance group  RII  Rank  Performance group 

Strong Innovator 

Strong Innovator 
Strong Innovator 

Innovation Leader 
Strong Innovator 
Innovation Leader 

94.2  125  Moderate Innovator 
101.0  102  Strong Innovator 
Strong Innovator 
112.8  68 

109.5  77 
97.1  116  Moderate Innovator 
99.5  109  Moderate Innovator 
133.5  22 
105.3  93 
129.4  35 
102.4  100  Strong Innovator 
122.0  47 
Strong Innovator 
96.2  119  Moderate Innovator 
107.0  83 
109.8  76 

Region 
DEA5  Arnsberg 
DEB1  Koblenz 
DEB2  Trier 
DEB3  Rheinhessen-Pfalz 
DEC  Saarland 
DED2 Dresden 
DED4 Chemnitz 
DED5 Leipzig 
DEE  Sachsen-Anhalt 
DEF  Schleswig-Holstein 
DEG  Thüringen 
EE  Estonia 
IE 
Ireland 
IE04  Northern and Western 
IE05  Southern 
IE06  Eastern and Midland 
EL  Greece 
92.6  130  Moderate Innovator 
EL3  Attiki 
66.5  182  Emerging Innovator 
EL41  Voreio Aigaio 
49.1  222  Emerging Innovator 
EL42  Notio Aigaio 
EL43  Kriti 
88.7  140  Moderate Innovator 
EL51  Anatoliki Makedonia, Thraki  63.4  194  Emerging Innovator 
89.0  137  Moderate Innovator 
EL52  Kentriki Makedonia 
54.9  213  Emerging Innovator 
EL53  Dytiki Makedonia 
70.7  173  Moderate Innovator 
EL54  Ipeiros 
77.1  164  Moderate Innovator 
EL61  Thessalia 
64.5  188  Emerging Innovator 
EL62  Ionia Nisia 
74.1  169  Moderate Innovator 
EL63  Dytiki Ellada 
64.4  189  Emerging Innovator 
EL64  Sterea Ellada 
EL65  Peloponnisos 
56.5  209  Emerging Innovator 
ES  Spain 
ES11  Galicia 
ES12  Principado de Asturias 
ES13  Cantabria 
ES21  País Vasco 

80.2  158  Moderate Innovator 
78.0  163  Moderate Innovator 
76.6  165  Moderate Innovator 
109.5  78 

Strong Innovator 

Strong Innovator 

Comunidad Foral de 
101.4  101  Strong Innovator 
ES22 
Navarra 
83.8  149  Moderate Innovator 
ES23  La Rioja 
82.7  152  Moderate Innovator 
ES24  Aragón 
104.2  97 
ES3  Comunidad de Madrid 
78.2  162  Moderate Innovator 
ES41  Castilla y León 
65.3  186  Emerging Innovator 
ES42  Castilla-la Mancha 
59.6  202  Emerging Innovator 
ES43  Extremadura 
104.6  96 
ES51  Cataluña 
94.4  124  Moderate Innovator 
ES52  Comunidad Valenciana 
68.8  177  Emerging Innovator 
ES53  Illes Balears 
69.5  176  Emerging Innovator 
ES61  Andalucía 
ES62  Región de Murcia 
79.5  160  Moderate Innovator 
ES63  Ciudad Autónoma de Ceuta 35.0  232  Emerging Innovator 

Strong Innovator 

110.6 

74 Strong Innovator 

97.5  115 Moderate Innovator 
98.7  109 Moderate Innovator 

20 Innovation Leader 
93 Strong Innovator 
38 Innovation Leader 

134.8 
106.3 
127.7 
101.3  101 Strong Innovator 
47 Strong Innovator 
120.2 

94.2  121 Moderate Innovator 

108.2 
107.6 

81 Strong Innovator 
85 Strong Innovator 

95.1  120 Moderate Innovator 

103.6 
115.6 

97 Strong Innovator 
60 Strong Innovator 

89.6  133 Moderate Innovator 
64.3  181 Emerging Innovator 
46.4  221 Emerging Innovator 
82.5  148 Moderate Innovator 
61.1  186 Emerging Innovator 
84.7  144 Moderate Innovator 
52.1  212 Emerging Innovator 
68.8  172 Emerging Innovator 
74.2  167 Moderate Innovator 
58.9  193 Emerging Innovator 
69.2  171 Emerging Innovator 
59.8  190 Emerging Innovator 
55.6  205 Emerging Innovator 

78.1  158 Moderate Innovator 
75.7  163 Moderate Innovator 
75.0  164 Moderate Innovator 

106.7 

91 Strong Innovator 

98.8  107 Moderate Innovator 
80.8  152 Moderate Innovator 
79.9  156 Moderate Innovator 

101.8  100 Strong Innovator 

74.5  165 Moderate Innovator 
62.8  184 Emerging Innovator 
56.6  199 Emerging Innovator 

102.3 

98 Strong Innovator 

91.5  128 Moderate Innovator 
65.5  180 Emerging Innovator 
67.1  177 Emerging Innovator 
76.2  162 Moderate Innovator 
31.0  232 Emerging Innovator 

Ciudad Autónoma de 
Melilla 

ES64 
ES7  Canarias 
France 
FR 
FR1 
Île de France 
FRB  Centre - Val de Loire 
Bourgogne - Franche-
Comté 

FRC 
FRD  Normandie 
FRE  Hauts-de-France 

44.6  226  Emerging Innovator 
53.4  214  Emerging Innovator 

41.8  226 Emerging Innovator 
50.9  213 Emerging Innovator 

131.8  26 
82.7  153  Moderate Innovator 

Innovation Leader 

134.7 

21 Innovation Leader 

87.2  139 Moderate Innovator 

84.7  145  Moderate Innovator 
73.4  170  Moderate Innovator 
82.4  155  Moderate Innovator 

88.2  136 Moderate Innovator 
76.7  160 Moderate Innovator 
85.8  143 Moderate Innovator 

71 

RII 

-1.2 
-0.4 
0.7 
-1.3 
-1.0 
1.6 
1.2 
1.8 
2.1 
-1.2 
2.2 

-0.9 
-2.6 
-2.9 

3.0 
2.2 
2.8 
6.2 
2.3 
4.3 
2.8 
1.8 
3.0 
5.5 
4.9 
4.6 
1.0 

2.1 
2.3 
1.6 
2.8 

2.5 
3.0 
2.8 
2.5 
3.7 
2.5 
3.0 
2.3 
2.8 
3.3 
2.4 
3.3 
4.0 

2.8 
2.5 

-2.9 
-4.5 

-3.5 
-3.3 
-3.3 

Out-
liers 
-3 
1 
0 
2 
-2 
0 
0 
0 
-1 
0 
0 

0 
-1 
-2 

-1 
1 
-3 
-3 
-2 
1 
6 
2 
1 
-2 
0 
-3 
0 

0 
0 
0 
2 

0 
-3 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
15 

8 
0 

3 
0 

0 
0 
0 

 
 
 
 
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
Box-plot results 

RIS 2021 results 

Differences 

Region 
FRF  Grand Est 
FRG  Pays de la Loire 
FRH  Bretagne 
FRI  Nouvelle-Aquitaine 
FRJ  Occitanie 
FRK  Auvergne - Rhône-Alpes 

Provence-Alpes-Côte 
FRL 
d'Azur 
FRM  Corse 

RII  Rank  Performance group  RII  Rank  Performance group 

RII 

92.7  128  Moderate Innovator 
93.0  127  Moderate Innovator 
106.5  87 
88.9  138  Moderate Innovator 
112.3  69 
110.9  71 

Strong Innovator 
Strong Innovator 

Strong Innovator 

95.4  119 Moderate Innovator 
98.7  111 Moderate Innovator 

108.5 

79 Strong Innovator 

91.8  127 Moderate Innovator 

116.9 
116.7 

54 Strong Innovator 
55 Strong Innovator 

104.6  95 
45.6  225  Emerging Innovator 

Strong Innovator 

106.8 

88 Strong Innovator 

44.7  223 Emerging Innovator 

-2.7 
-5.7 
-1.9 
-2.9 
-4.6 
-5.9 

-2.2 
0.9 

69.5  170 Emerging Innovator 

1.8 

Régions ultrapériphériques 
françaises 

71.3  171  Moderate Innovator 

FRY 
HR  Croatia 
HR02  Panonska Hrvatska 
HR03  Jadranska Hrvatska 
HR05  Grad Zagreb 
HR06  Sjeverna Hrvatska 
IT 
Italy 
ITC1  Piemonte 
ITC2  Valle d'Aosta/Vallée d'Aoste  63.3  196  Emerging Innovator 
84.4  147  Moderate Innovator 
ITC3  Liguria 
106.3  90 
ITC4  Lombardia 

84.5  146  Moderate Innovator 
63.4  195  Emerging Innovator 
89.6  135  Moderate Innovator 
87.5  141  Moderate Innovator 

100.5  103  Strong Innovator 

Strong Innovator 

82.9  146 Moderate Innovator 
61.4  185 Emerging Innovator 
89.0  135 Moderate Innovator 
86.2  141 Moderate Innovator 

102.1 

99 Strong Innovator 

63.0  183 Emerging Innovator 
86.3  140 Moderate Innovator 

107.9 

82 Strong Innovator 

Provincia Autonoma 
Bolzano/Bozen 

93.2  126 Moderate Innovator 

92.6  131  Moderate Innovator 

98.6  112 Moderate Innovator 
89.7  132 Moderate Innovator 

89 Strong Innovator 
106.7 
78 Strong Innovator 
108.7 
87 Strong Innovator 
107.0 
112.0 
71 Strong Innovator 
100.9  102 Strong Innovator 

ITH1 
Strong Innovator 
ITH2  Provincia Autonoma Trento  106.6  86 
Strong Innovator 
107.7  82 
ITH3  Veneto 
Strong Innovator 
108.1  80 
ITH4  Friuli-Venezia Giulia 
110.1  74 
ITH5  Emilia-Romagna 
Strong Innovator 
100.0  105  Strong Innovator 
ITI1  Toscana 
99.5  107  Moderate Innovator 
ITI2  Umbria 
90.3  133  Moderate Innovator 
ITI3  Marche 
99.6  106  Moderate Innovator  100.8  103 Strong Innovator 
ITI4  Lazio 
84.0  148  Moderate Innovator 
ITF1  Abruzzo 
82.8  151  Moderate Innovator 
ITF2  Molise 
82.9  150  Moderate Innovator 
ITF3  Campania 
75.0  168  Moderate Innovator 
ITF4  Puglia 
79.1  161  Moderate Innovator 
ITF5  Basilicata 
66.3  184  Emerging Innovator 
ITF6  Calabria 
70.3  174  Moderate Innovator 
ITG1  Sicilia 
ITG2  Sardegna 
66.8  181  Emerging Innovator 
CY  Cyprus 
Latvia 
LV 
LT 
Lithuania 
LT01  Sostinės regionas 

83.7  145 Moderate Innovator 
80.5  153 Moderate Innovator 
82.6  147 Moderate Innovator 
74.2  166 Moderate Innovator 
77.4  159 Moderate Innovator 
66.1  178 Emerging Innovator 
68.6  173 Emerging Innovator 
67.2  176 Emerging Innovator 

95 Strong Innovator 

Strong Innovator 

105.8  91 

103.9 

Vidurio ir vakarų Lietuvos 
regionas  
Luxembourg 

LT02 
LU 
HU  Hungary 
HU11  Budapest 
HU12  Pest 
HU21  Közép-Dunántúl 
HU22  Nyugat-Dunántúl 
HU23  Dél-Dunántúl 
HU31  Észak-Magyarország 
HU32  Észak-Alföld 
HU33  Dél-Alföld 
MT  Malta 
NL  Netherlands 
NL11  Groningen 
NL12  Friesland 

68.6  178  Emerging Innovator 

67.6  174 Emerging Innovator 

98.8  110  Moderate Innovator 
67.4  179  Emerging Innovator 
58.5  206  Emerging Innovator 
55.3  212  Emerging Innovator 
50.8  219  Emerging Innovator 
52.8  217  Emerging Innovator 
53.4  215  Emerging Innovator 
60.0  201  Emerging Innovator 

99.7  104 Moderate Innovator 
67.5  175 Emerging Innovator 
57.9  195 Emerging Innovator 
54.9  206 Emerging Innovator 
50.1  216 Emerging Innovator 
52.2  211 Emerging Innovator 
52.2  210 Emerging Innovator 
58.5  194 Emerging Innovator 

115.2  60 
96.2  120  Moderate Innovator 

Strong Innovator 

115.0 

64 Strong Innovator 

96.0  118 Moderate Innovator 

72 

Out-
liers 
-3 
0 
-5 
0 
6 
0 

-3 
4 

0 

0 
-3 
0 
-4 

0 
0 
0 
7 

-2 
0 
2 
-2 
-3 
0 
0 
0 
-1 
0 
-1 
0 
0 
-1 
0 
0 
0 

-1 

0 

-3 
-2 
0 
-2 
0 
1 
0 
-3 

0 
2 

1.5 
2.0 
0.6 
1.3 

-1.6 
0.3 
-1.9 
-1.7 

-0.7 
-0.1 
-1.0 
1.1 
-2.0 
-0.9 
0.9 
0.6 
-1.2 
0.2 
2.3 
0.3 
0.7 
1.6 
0.2 
1.7 
-0.3 

1.9 

1.0 

-0.9 
0.0 
0.6 
0.4 
0.6 
0.6 
1.1 
1.5 

0.2 
0.1 

 
 
 
 
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
Region 
NL13  Drenthe 
NL21  Overijssel 
NL22  Gelderland 
NL23  Flevoland 
NL31  Utrecht 
NL32  Noord-Holland 
NL33  Zuid-Holland 
NL34  Zeeland 
NL41  Noord-Brabant 
NL42  Limburg 
Austria 
AT 
AT1  Ostösterreich 
AT2  Südösterreich 
AT3  Westösterreich 
PL 
Poland 
PL21  Malopolskie 
PL22  Slaskie 
PL41  Wielkopolskie 
PL42  Zachodniopomorskie 
PL43  Lubuskie 
PL51  Dolnoslaskie 
PL52  Opolskie 
PL61  Kujawsko-Pomorskie 
PL62  Warminsko-Mazurskie 
PL63  Pomorskie 
PL71  Lódzkie 
PL72  Swietokrzyskie 
PL81  Lubelskie 
PL82  Podkarpackie 
PL84  Podlaskie 
PL91  Warszawski stoleczny 
PL92  Mazowiecki regionalny 
PT 
PT11  Norte 
PT15  Algarve 
PT16  Centro 
PT17  Lisboa 
PT18  Alentejo 

Portugal 

PT2 

Região Autónoma dos 
Açores 
Região Autónoma da 
PT3 
Madeira 
RO  Romania 
RO11  Nord-Vest 
RO12  Centru 
RO21  Nord-Est 
RO22  Sud-Est 
RO31  Sud - Muntenia 
RO32  Bucuresti - Ilfov 
RO41  Sud-Vest Oltenia 
RO42  Vest 
SI 
SI03  Vzhodna Slovenija 
SI04  Zahodna Slovenija 
SK  Slovakia 
SK01  Bratislavský kraj 
SK02  Západné Slovensko 
SK03  Stredné Slovensko 
SK04  Východné Slovensko 

Slovenia 

Box-plot results 

RIS 2021 results 

Differences 

RII  Rank  Performance group  RII  Rank  Performance group 

Strong Innovator 
Strong Innovator 
Strong Innovator 
Innovation Leader 
Innovation Leader 
Strong Innovator 

94.8  122  Moderate Innovator 
113.2  66 
123.2  44 
110.3  72 
131.4  28 
130.9  29 
124.1  43 
94.7  123  Moderate Innovator 
130.3  32 
123.0  45 

Innovation Leader 
Strong Innovator 

93.4  125 Moderate Innovator 

112.1 
122.2 
108.9 
130.4 
130.3 
123.4 

70 Strong Innovator 
45 Strong Innovator 
77 Strong Innovator 
31 Innovation Leader 
33 Innovation Leader 
43 Strong Innovator 

94.1  124 Moderate Innovator 

128.7 
121.6 

36 Innovation Leader 
46 Strong Innovator 

121.1  49 
114.2  64 
111.4  70 

Strong Innovator 
Strong Innovator 
Strong Innovator 

123.7 
116.4 
113.6 

42 Strong Innovator 
57 Strong Innovator 
68 Strong Innovator 

81.8  157  Moderate Innovator 
60.6  200  Emerging Innovator 
59.5  203  Emerging Innovator 
50.6  220  Emerging Innovator 
52.5  218  Emerging Innovator 
70.0  175  Emerging Innovator 
56.3  211  Emerging Innovator 
57.1  208  Emerging Innovator 
49.3  221  Emerging Innovator 
66.4  183  Emerging Innovator 
61.9  198  Emerging Innovator 
47.7  224  Emerging Innovator 
58.6  205  Emerging Innovator 
64.0  192  Emerging Innovator 
53.0  216  Emerging Innovator 
96.4  118  Moderate Innovator 
43.0  227  Emerging Innovator 

73.3  168 Moderate Innovator 
54.1  207 Emerging Innovator 
53.4  208 Emerging Innovator 
45.2  222 Emerging Innovator 
46.6  220 Emerging Innovator 
63.9  182 Emerging Innovator 
50.8  214 Emerging Innovator 
50.7  215 Emerging Innovator 
42.9  224 Emerging Innovator 
60.5  187 Emerging Innovator 
55.7  203 Emerging Innovator 
42.1  225 Emerging Innovator 
52.7  209 Emerging Innovator 
57.9  196 Emerging Innovator 
47.5  219 Emerging Innovator 
87.8  138 Moderate Innovator 
38.2  227 Emerging Innovator 

93.1  126  Moderate Innovator 
64.2  190  Emerging Innovator 
89.2  136  Moderate Innovator 
105.6  92 
75.5  167  Moderate Innovator 

Strong Innovator 

81.5  150 Moderate Innovator 
55.6  204 Emerging Innovator 
79.1  157 Moderate Innovator 
94.1  123 Moderate Innovator 
65.6  179 Emerging Innovator 

59.4  204  Emerging Innovator 

49.9  217 Emerging Innovator 

65.3  187  Emerging Innovator 

56.5  200 Emerging Innovator 

35.4  231  Emerging Innovator 
29.6  236  Emerging Innovator 
26.6  237  Emerging Innovator 
20.4  240  Emerging Innovator 
22.7  238  Emerging Innovator 
63.9  193  Emerging Innovator 
21.9  239  Emerging Innovator 
33.5  233  Emerging Innovator 

31.4  231 Emerging Innovator 
26.5  236 Emerging Innovator 
22.7  237 Emerging Innovator 
17.9  240 Emerging Innovator 
19.6  238 Emerging Innovator 
59.5  192 Emerging Innovator 
19.6  239 Emerging Innovator 
30.9  233 Emerging Innovator 

82.4  154  Moderate Innovator 
100.3  104  Strong Innovator 

80.2  155 Moderate Innovator 
98.0  113 Moderate Innovator 

87.2  142  Moderate Innovator 
56.3  210  Emerging Innovator 
60.7  199  Emerging Innovator 
57.6  207  Emerging Innovator 

88.2  137 Moderate Innovator 
56.7  198 Emerging Innovator 
59.9  189 Emerging Innovator 
57.3  197 Emerging Innovator 

73 

RII 

1.4 
1.0 
1.1 
1.4 
1.0 
0.6 
0.7 
0.6 
1.6 
1.4 

-2.5 
-2.2 
-2.1 

8.5 
6.5 
6.1 
5.4 
5.9 
6.1 
5.5 
6.4 
6.4 
6.0 
6.2 
5.6 
5.9 
6.1 
5.5 
8.6 
4.7 

11.6 
8.5 
10.1 
11.5 
9.9 

9.5 

8.7 

4.0 
3.1 
4.0 
2.5 
3.1 
4.3 
2.4 
2.6 

2.3 
2.2 

-1.0 
-0.4 
0.9 
0.3 

Out-
liers 
0 
1 
1 
1 
8 
1 
0 
0 
-1 
0 

-1 
0 
2 

8 
0 
0 
0 
0 
0 
3 
0 
2 
0 
6 
2 
0 
0 
0 
5 
0 

-1 
2 
2 
2 
0 

0 

0 

11 
11 
11 
13 
11 
6 
18 
18 

0 
0 

5 
0 
0 
-2 

 
 
 
 
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
Box-plot results 

RIS 2021 results 

Differences 

RII  Rank  Performance group  RII  Rank  Performance group 

RII 

Out-
liers 

Finland 

Region 
FI 
FI1B  Helsinki-Uusimaa 
FI1C  Etelä-Suomi 
FI19  Länsi-Suomi 
FI1D  Pohjois- ja Itä-Suomi 
FI2  Åland 
SE  Sweden 
SE11  Stockholm 
SE12  Östra Mellansverige 
SE21  Småland med öarna 
SE22  Sydsverige 
SE23  Västsverige 
SE31  Norra Mellansverige 
SE32  Mellersta Norrland 
SE33  Övre Norrland 
NO  Norway 
NO01  Oslo og Akershus 
NO02  Hedmark og Oppland 
NO03  Sør-Østlandet 
NO04  Agder og Rogaland 
NO05  Vestlandet 
NO06  Trøndelag 
NO07  Nord-Norge 
CH  Switzerland 
CH01  Région lémanique 
CH02  Espace Mittelland 
CH03  Nordwestschweiz 
CH04  Zürich 
CH05  Ostschweiz 
CH06  Zentralschweiz 
CH07  Ticino 
RS  Serbia 
RS11  Belgrade 
RS12  Vojvodina 

152.7  3 
114.4  63 
129.8  33 
117.4  55 
103.9  98 

Innovation Leader 
Strong Innovator 
Innovation Leader 
Strong Innovator 
Strong Innovator 

Innovation Leader 
Innovation Leader 
Strong Innovator 
Innovation Leader 
Innovation Leader 

158.3  1 
130.9  30 
109.8  75 
146.5  7 
139.0  12 
96.4  117  Moderate Innovator 
98.0  113  Moderate Innovator 
118.7  51 

Strong Innovator 

Innovation Leader 

138.9  13 
92.1  132  Moderate Innovator 
103.8  99 
106.8  85 
114.9  62 
132.6  24 
98.3  111  Moderate Innovator 

Strong Innovator 
Strong Innovator 
Strong Innovator 
Innovation Leader 

154.3 
115.8 
130.9 
116.6 
105.2 

155.2 
131.2 
109.5 
144.5 
138.4 

2 Innovation Leader 
59 Strong Innovator 
28 Innovation Leader 
56 Strong Innovator 
94 Strong Innovator 

1 Innovation Leader 
27 Innovation Leader 
75 Strong Innovator 
9 Innovation Leader 
13 Innovation Leader 

96.8  117 Moderate Innovator 
96.9  116 Moderate Innovator 

116.3 

58 Strong Innovator 

139.1 

12 Innovation Leader 

91.0  129 Moderate Innovator 

103.8 
106.7 
115.5 
130.5 

96 Strong Innovator 
90 Strong Innovator 
61 Strong Innovator 
30 Innovation Leader 

98.8  108 Moderate Innovator 

134.0  19 
128.3  38 
140.2  11 
148.1  5 
132.4  25 
135.1  16 
144.4  8 

Innovation Leader 
Innovation Leader 
Innovation Leader 
Innovation Leader 
Innovation Leader 
Innovation Leader 
Innovation Leader 

133.1 
128.7 
139.2 
147.9 
133.3 
135.4 
144.5 

24 Innovation Leader 
37 Innovation Leader 
11 Innovation Leader 
6 Innovation Leader 
22 Innovation Leader 
18 Innovation Leader 
8 Innovation Leader 

-1.5 
-1.4 
-1.1 
0.8 
-1.3 

3.1 
-0.3 
0.3 
2.0 
0.6 
-0.4 
1.1 
2.3 

-0.1 
1.2 
0.0 
0.1 
-0.6 
2.1 
-0.5 

0.8 
-0.3 
1.0 
0.1 
-0.9 
-0.3 
-0.1 

92.7  129  Moderate Innovator 
67.1  180  Emerging Innovator 

82.1  149 Moderate Innovator 
60.1  188 Emerging Innovator 

10.6 
7.1 

RS21 

64.2  191  Emerging Innovator 

63.2  197  Emerging Innovator 

Šumadija and Western 
Serbia 
Southern and Eastern 
Serbia 
RS22 
UK  United Kingdom 
115.7  59 
UKC  North East 
UKD  North West 
117.3  56 
UKE  Yorkshire and The Humber  117.5  53 
122.9  46 
UKF  East Midlands 
127.0  40 
UKG  West Midlands 
133.3  23 
UKH  East of England 
136.4  15 
UKI  London 
142.5  10 
UKJ  South East 
128.7  36 
UKK  South West  
117.9  52 
UKL  Wales 
121.6  48 
UKM  Scotland 
108.9  79 
UKN  Northern Ireland 

112.0 
114.8 
114.7 
120.0 
122.8 
133.2 
135.6 
140.3 
126.3 
115.2 
119.9 
106.4 
Note: Performance groups or rank increases highlighted in green, decreases in red. 

Strong Innovator 
Strong Innovator 
Strong Innovator 
Strong Innovator 
Innovation Leader 
Innovation Leader 
Innovation Leader 
Innovation Leader 
Innovation Leader 
Strong Innovator 
Strong Innovator 
Strong Innovator 

55.8  202 Emerging Innovator 

56.4  201 Emerging Innovator 

72 Strong Innovator 
65 Strong Innovator 
66 Strong Innovator 
49 Strong Innovator 
44 Strong Innovator 
23 Innovation Leader 
16 Innovation Leader 
10 Innovation Leader 
39 Innovation Leader 
62 Strong Innovator 
50 Strong Innovator 
92 Strong Innovator 

7.4 

7.7 

3.7 
2.5 
2.8 
2.9 
4.2 
0.0 
0.8 
2.2 
2.4 
2.7 
1.6 
2.5 

74 

3 
1 
1 
2 
6 

7 
8 
7 
-5 
2 
8 
1 
8 

12 
4 
2 
1 
0 
5 
3 

6 
4 
0 
19 
12 
2 
8 

5 
-7 

2 

7 

4 
5 
-5 
-1 
3 
2 
8 
6 
1 
2 
0 
4 

 
 
 
 
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
 
 
Table  35  RIS  2021  results  using  the  current  method  (z-score  +/-  2  cut-off  values)  vs.  MAD 
method 

MAD results 

RIS 2021 results 

Differences 

Region 
BE  Belgium 

RII  Rank 

Performance  
group 

RII  Rank  Performance group 

RII 

Région de Bruxelles-
Capitale / Brussels 
Hoofdstedelijk Gewest 

108.3 

143.6 

80 Strong Innovator 

8 Innovation Leader 

139.0 
135.5 
112.8 

102.6 
131.9 
111.6 

89.6  134 Moderate Innovator 

12 Innovation Leader 
18 Innovation Leader 
67 Strong Innovator 

98 Strong Innovator 
26 Innovation Leader 
73 Strong Innovator 

29.9  234 Emerging Innovator 
38.3  229 Emerging Innovator 
38.6  228 Emerging Innovator 
27.7  235 Emerging Innovator 
63.9  184 Emerging Innovator 
39.7  226 Emerging Innovator 

86.1  142 Moderate Innovator 
68.2  172 Emerging Innovator 
44.2  225 Emerging Innovator 
77.7  158 Moderate Innovator 
88.1  136 Moderate Innovator 
73.5  166 Moderate Innovator 
79.1  156 Moderate Innovator 

BE1 
BE2  Vlaams Gewest 
BE3  Région Wallonne 
BG  Bulgaria 
BG31  Severozapaden 
BG32  Severen tsentralen 
BG33  Severoiztochen 
BG34  Yugoiztochen 
BG41  Yugozapaden 
BG42  Yuzhen tsentralen 
CZ  Czech Republic 
CZ01  Praha 
CZ02  Strední Cechy 
CZ03  Jihozápad 
CZ04  Severozápad 
CZ05  Severovýchod 
CZ06  Jihovýchod 
CZ07  Strední Morava 
CZ08  Moravskoslezsko 
DK  Denmark 
DK01  Hovedstaden 
DK02  Sjælland 
DK03  Syddanmark 
DK04  Midtjylland 
DK05  Nordjylland 
DE  Germany 
DE11  Stuttgart 
DE12  Karlsruhe 
DE13  Freiburg 
DE14  Tübingen 
DE21  Oberbayern 
DE22  Niederbayern 
DE23  Oberpfalz 
DE24  Oberfranken 
DE25  Mittelfranken 
DE26  Unterfranken 
DE27  Schwaben 
DE3  Berlin 
DE4  Brandenburg 
DE5  Bremen 
DE6  Hamburg 
DE71  Darmstadt 
DE72  Gießen 
DE73  Kassel 
91.5  125 Moderate Innovator 
DE8  Mecklenburg-Vorpommern  90.6  127 Moderate Innovator 
DE91  Braunschweig 
DE92  Hannover 
DE93  Lüneburg 
DE94  Weser-Ems 
DEA1  Düsseldorf 
DEA2  Köln 
DEA3  Münster 
DEA4  Detmold 

68 Strong Innovator 
77 Strong Innovator 
32 Innovation Leader 
50 Strong Innovator 
94 Strong Innovator 
3 Innovation Leader 

33 Innovation Leader 
7 Innovation Leader 
43 Innovation Leader 
17 Innovation Leader 
1 Innovation Leader 

72 Strong Innovator 
19 Innovation Leader 
35 Innovation Leader 
45 Strong Innovator 

88 Strong Innovator 
29 Innovation Leader 
97 Strong Innovator 
58 Strong Innovator 

84.3  144 Moderate Innovator 
78.8  157 Moderate Innovator 

21 Innovation Leader 
53 Strong Innovator 

112.8 
108.5 
128.8 
119.0 
104.7 
152.3 

128.8 
144.4 
125.1 
135.7 
156.4 

111.8 
135.0 
127.7 
124.3 

105.7 
129.6 
102.9 
115.9 

94.1  113 Moderate Innovator 

94.2  112 Moderate Innovator 

134.0 
116.7 

75 

137.6 
131.9 
113.1 

14 Innovation Leader 
25 Innovation Leader 
69 Strong Innovator 

29.1  234 Emerging Innovator 
36.7  229 Emerging Innovator 
36.2  230 Emerging Innovator 
27.8  235 Emerging Innovator 
59.8  191 Emerging Innovator 
37.9  228 Emerging Innovator 

108.3 

80 Strong Innovator 

89.2  134 Moderate Innovator 
72.4  169 Moderate Innovator 
48.1  218 Emerging Innovator 
80.4  154 Moderate Innovator 
90.3  131 Moderate Innovator 
76.6  161 Moderate Innovator 
81.0  151 Moderate Innovator 

149.2 

5 Innovation Leader 

98.8  106 Moderate Innovator 

107.6 
131.5 
115.0 

130.4 
145.0 
125.3 
134.9 
154.1 

86 Strong Innovator 
26 Innovation Leader 
63 Strong Innovator 

32 Innovation Leader 
7 Innovation Leader 
40 Innovation Leader 
19 Innovation Leader 
3 Innovation Leader 

99.3  105 Moderate Innovator 

117.4 
111.7 
130.0 
120.1 
107.9 
149.3 

53 Strong Innovator 
73 Strong Innovator 
34 Innovation Leader 
48 Strong Innovator 
83 Strong Innovator 
4 Innovation Leader 

97.9  114 Moderate Innovator 

114.0 
136.1 
129.5 
125.0 

67 Strong Innovator 
15 Innovation Leader 
35 Innovation Leader 
41 Innovation Leader 

98.7  110 Moderate Innovator 
94.1  122 Moderate Innovator 

135.5 
118.4 

17 Innovation Leader 
51 Strong Innovator 

90.9  130 Moderate Innovator 
86.1  142 Moderate Innovator 

108.9 
130.6 
107.8 
117.9 

76 Strong Innovator 
29 Innovation Leader 
84 Strong Innovator 
52 Strong Innovator 

1.4 
3.6 
-0.3 

0.8 
1.5 
2.4 
0.0 
4.1 
1.8 

0.0 
-3.1 
-4.2 
-4.0 
-2.8 
-2.2 
-3.1 
-1.9 

-5.6 
-9.2 
-5.0 
0.4 
-3.5 

-1.6 
-0.6 
-0.2 
0.8 
2.4 
-5.2 
-4.6 
-3.1 
-1.2 
-1.2 
-3.2 
3.0 
-3.7 
-2.1 
-1.1 
-1.7 
-0.7 
-7.2 
-3.5 
-1.5 
-1.7 
-6.7 
-7.4 
-3.2 
-1.0 
-4.9 
-2.1 

Out-
liers 

-15 
-3 
-2 

2 
1 
2 
4 
-1 
-3 

-6 
0 
-2 
-5 
-2 
-2 
-1 
-6 

-8 
-7 
-8 
-11 
-6 

-3 
-16 
-6 
-6 
-7 
-3 
0 
-10 
-9 
-8 
-7 
-13 
0 
-5 
-5 
-9 
0 
3 
0 
-3 
1 
-2 
0 
0 
-1 
-1 
2 

 
 
 
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
MAD results 

RIS 2021 results 

Differences 

RII  Rank  Performance group 

RII 

96.8  110 Moderate Innovator  101.3  101 Strong Innovator 
47 Strong Innovator 

49 Strong Innovator 

120.2 

91.8  121 Moderate Innovator 
99.6  106 Moderate Innovator  103.6 
115.6 

63 Strong Innovator 

95.1  120 Moderate Innovator 

97 Strong Innovator 
60 Strong Innovator 

RII  Rank 

Performance  
group 

119.9 

106.8 

103.2 
105.6 

81 Strong Innovator 

30 Innovation Leader 

23 Innovation Leader 

89.9  130 Moderate Innovator 

96 Strong Innovator 
90 Strong Innovator 

92.8  116 Moderate Innovator 
93.2  114 Moderate Innovator 

133.0 
101.6  101 Strong Innovator 
129.4 

Region 
DEA5  Arnsberg 
DEB1  Koblenz 
DEB2  Trier 
DEB3  Rheinhessen-Pfalz 
DEC  Saarland 
DED2 Dresden 
DED4 Chemnitz 
DED5 Leipzig 
DEE  Sachsen-Anhalt 
DEF  Schleswig-Holstein 
DEG  Thüringen 
EE  Estonia 
IE 
Ireland 
IE04  Northern and Western 
IE05  Southern 
IE06  Eastern and Midland 
EL  Greece 
91.5  124 Moderate Innovator 
EL3  Attiki 
64.5  181 Emerging Innovator 
EL41  Voreio Aigaio 
46.2  221 Emerging Innovator 
EL42  Notio Aigaio 
EL43  Kriti 
87.4  137 Moderate Innovator 
EL51  Anatoliki Makedonia, Thraki  60.2  192 Emerging Innovator 
86.1  143 Moderate Innovator 
EL52  Kentriki Makedonia 
52.2  213 Emerging Innovator 
EL53  Dytiki Makedonia 
69.8  171 Emerging Innovator 
EL54  Ipeiros 
74.4  163 Moderate Innovator 
EL61  Thessalia 
61.6  189 Emerging Innovator 
EL62  Ionia Nisia 
71.4  168 Moderate Innovator 
EL63  Dytiki Ellada 
62.2  187 Emerging Innovator 
EL64  Sterea Ellada 
EL65  Peloponnisos 
54.2  208 Emerging Innovator 
ES  Spain 
ES11  Galicia 
ES12  Principado de Asturias 
ES13  Cantabria 
ES21  País Vasco 

76.8  159 Moderate Innovator 
74.0  164 Moderate Innovator 
73.6  165 Moderate Innovator 

71 Strong Innovator 

111.9 

114.8 

105.0 

102.2 

92 Strong Innovator 

99 Strong Innovator 

81.8  148 Moderate Innovator 
79.7  154 Moderate Innovator 

Comunidad Foral de 
ES22 
Navarra 
ES23  La Rioja 
ES24  Aragón 
ES3  Comunidad de Madrid 
ES41  Castilla y León 
ES42  Castilla-la Mancha 
ES43  Extremadura 
ES51  Cataluña 
95.3  111 Moderate Innovator 
ES52  Comunidad Valenciana 
64.4  182 Emerging Innovator 
ES53  Illes Balears 
64.6  180 Emerging Innovator 
ES61  Andalucía 
ES62  Región de Murcia 
76.4  160 Moderate Innovator 
ES63  Ciudad Autónoma de Ceuta  32.3  233 Emerging Innovator 

74.8  162 Moderate Innovator 
60.2  193 Emerging Innovator 
54.5  206 Emerging Innovator 

82 Strong Innovator 

106.3 

110.6 

74 Strong Innovator 

97.5  115 Moderate Innovator 
98.7  109 Moderate Innovator 

134.8 
106.3 
127.7 

20 Innovation Leader 
93 Strong Innovator 
38 Innovation Leader 

94.2  121 Moderate Innovator 

108.2 
107.6 

81 Strong Innovator 
85 Strong Innovator 

89.6  133 Moderate Innovator 
64.3  181 Emerging Innovator 
46.4  221 Emerging Innovator 
82.5  148 Moderate Innovator 
61.1  186 Emerging Innovator 
84.7  144 Moderate Innovator 
52.1  212 Emerging Innovator 
68.8  172 Emerging Innovator 
74.2  167 Moderate Innovator 
58.9  193 Emerging Innovator 
69.2  171 Emerging Innovator 
59.8  190 Emerging Innovator 
55.6  205 Emerging Innovator 

78.1  158 Moderate Innovator 
75.7  163 Moderate Innovator 
75.0  164 Moderate Innovator 

106.7 

91 Strong Innovator 

98.8  107 Moderate Innovator 
80.8  152 Moderate Innovator 
79.9  156 Moderate Innovator 

101.8  100 Strong Innovator 

74.5  165 Moderate Innovator 
62.8  184 Emerging Innovator 
56.6  199 Emerging Innovator 

102.3 

98 Strong Innovator 

91.5  128 Moderate Innovator 
65.5  180 Emerging Innovator 
67.1  177 Emerging Innovator 
76.2  162 Moderate Innovator 
31.0  232 Emerging Innovator 

Ciudad Autónoma de 
Melilla 

ES64 
ES7  Canarias 
France 
FR 
FR1 
Île de France 
FRB  Centre - Val de Loire 
Bourgogne - Franche-
Comté 

FRC 
FRD  Normandie 
FRE  Hauts-de-France 

45.0  224 Emerging Innovator 
47.9  218 Emerging Innovator 

41.8  226 Emerging Innovator 
50.9  213 Emerging Innovator 

137.5 

14 Innovation Leader 

134.7 

21 Innovation Leader 

81.0  151 Moderate Innovator 

87.2  139 Moderate Innovator 

82.3  146 Moderate Innovator 
70.2  170 Moderate Innovator 
79.4  155 Moderate Innovator 

88.2  136 Moderate Innovator 
76.7  160 Moderate Innovator 
85.8  143 Moderate Innovator 

76 

Out-
liers 
-6 
0 
0 
-3 
-2 
0 
0 
-2 
-1 
0 
-1 

0 
-1 
-5 

-1 
0 
-3 
-7 
-8 
1 
1 
2 
-2 
-4 
0 
-4 
0 

0 
0 
0 
2 

-2 
-6 
0 
0 
0 
0 
0 
0 
-8 
0 
0 
0 
14 

8 
0 

-3 
0 

0 
0 
0 

-3.9 
-4.7 
-5.5 
-1.8 
-4.6 
1.7 
-4.5 
-0.3 
-4.3 
-5.1 
-2.0 

-3.3 
-4.0 
-0.9 

1.9 
0.2 
-0.2 
4.9 
-0.9 
1.4 
0.1 
0.9 
0.2 
2.6 
2.2 
2.4 
-1.3 

-1.3 
-1.7 
-1.5 
5.2 

3.4 
0.9 
-0.2 
3.2 
0.3 
-2.7 
-2.1 
4.0 
3.8 
-1.1 
-2.4 
0.2 
1.3 

3.2 
-3.0 

2.8 
-6.2 

-5.9 
-6.5 
-6.3 

 
 
 
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
Region 
FRF  Grand Est 
FRG  Pays de la Loire 
FRH  Bretagne 
FRI  Nouvelle-Aquitaine 
FRJ  Occitanie 
FRK  Auvergne - Rhône-Alpes 

Provence-Alpes-Côte 
FRL 
d'Azur 
FRM  Corse 

MAD results 

RIS 2021 results 

Differences 

RII  Rank 

Performance  
group 
91.8  120 Moderate Innovator 
92.1  119 Moderate Innovator 

RII  Rank  Performance group 

RII 

95.4  119 Moderate Innovator 
98.7  111 Moderate Innovator 

108.4 

78 Strong Innovator 

108.5 

79 Strong Innovator 

87.1  138 Moderate Innovator 

91.8  127 Moderate Innovator 

115.1 
116.2 

62 Strong Innovator 
57 Strong Innovator 

116.9 
116.7 

54 Strong Innovator 
55 Strong Innovator 

104.7 

93 Strong Innovator 

106.8 

88 Strong Innovator 

37.9  230 Emerging Innovator 

44.7  223 Emerging Innovator 

69.5  170 Emerging Innovator 

-2.0 

82.9  146 Moderate Innovator 
61.4  185 Emerging Innovator 
89.0  135 Moderate Innovator 
86.2  141 Moderate Innovator 

102.1 

99 Strong Innovator 

63.0  183 Emerging Innovator 
86.3  140 Moderate Innovator 

107.9 

82 Strong Innovator 

93.2  126 Moderate Innovator 

89 Strong Innovator 
106.7 
78 Strong Innovator 
108.7 
87 Strong Innovator 
107.0 
112.0 
71 Strong Innovator 
100.9  102 Strong Innovator 

98.6  112 Moderate Innovator 
89.7  132 Moderate Innovator 

Régions ultrapériphériques 
françaises 

67.5  174 Emerging Innovator 

FRY 
HR  Croatia 
HR02  Panonska Hrvatska 
HR03  Jadranska Hrvatska 
HR05  Grad Zagreb 
HR06  Sjeverna Hrvatska 
IT 
Italy 
ITC1  Piemonte 
ITC2  Valle d'Aosta/Vallée d'Aoste  58.4  198 Emerging Innovator 
83.8  145 Moderate Innovator 
ITC3  Liguria 
ITC4  Lombardia 

86.3  141 Moderate Innovator 
59.8  194 Emerging Innovator 
92.1  118 Moderate Innovator 
89.8  132 Moderate Innovator 

101.1  103 Strong Innovator 

79 Strong Innovator 

108.3 

91.5  123 Moderate Innovator 

Provincia Autonoma 
Bolzano/Bozen 

ITH1 
76 Strong Innovator 
ITH2  Provincia Autonoma Trento  110.3 
75 Strong Innovator 
110.4 
ITH3  Veneto 
70 Strong Innovator 
112.0 
ITH4  Friuli-Venezia Giulia 
114.4 
ITH5  Emilia-Romagna 
65 Strong Innovator 
102.1  100 Strong Innovator 
ITI1  Toscana 
100.7  104 Strong Innovator 
ITI2  Umbria 
ITI3  Marche 
ITI4  Lazio 
ITF1  Abruzzo 
ITF2  Molise 
ITF3  Campania 
ITF4  Puglia 
ITF5  Basilicata 
ITF6  Calabria 
ITG1  Sicilia 
ITG2  Sardegna 
CY  Cyprus 
Latvia 
LV 
LT 
Lithuania 
LT01  Sostinės regionas 

87 Strong Innovator 

105.9 

89.6  133 Moderate Innovator 
99.2  108 Moderate Innovator  100.8  103 Strong Innovator 
81.7  149 Moderate Innovator 
81.3  150 Moderate Innovator 
80.3  153 Moderate Innovator 
71.2  169 Moderate Innovator 
75.0  161 Moderate Innovator 
62.0  188 Emerging Innovator 
66.6  177 Emerging Innovator 
62.8  186 Emerging Innovator 

83.7  145 Moderate Innovator 
80.5  153 Moderate Innovator 
82.6  147 Moderate Innovator 
74.2  166 Moderate Innovator 
77.4  159 Moderate Innovator 
66.1  178 Emerging Innovator 
68.6  173 Emerging Innovator 
67.2  176 Emerging Innovator 

103.9 

95 Strong Innovator 

2.0 

Vidurio ir vakarų Lietuvos 
regionas  
Luxembourg 

LT02 
LU 
HU  Hungary 
HU11  Budapest 
HU12  Pest 
HU21  Közép-Dunántúl 
HU22  Nyugat-Dunántúl 
HU23  Dél-Dunántúl 
HU31  Észak-Magyarország 
HU32  Észak-Alföld 
HU33  Dél-Alföld 
MT  Malta 
NL  Netherlands 
NL11  Groningen 
NL12  Friesland 

65.0  179 Emerging Innovator 

67.6  174 Emerging Innovator 

-2.6 

105.7 

89 Strong Innovator 

66.8  176 Emerging Innovator 
57.0  202 Emerging Innovator 
53.4  209 Emerging Innovator 
47.6  219 Emerging Innovator 
50.3  215 Emerging Innovator 
49.9  216 Emerging Innovator 
57.8  200 Emerging Innovator 

99.7  104 Moderate Innovator 
67.5  175 Emerging Innovator 
57.9  195 Emerging Innovator 
54.9  206 Emerging Innovator 
50.1  216 Emerging Innovator 
52.2  211 Emerging Innovator 
52.2  210 Emerging Innovator 
58.5  194 Emerging Innovator 

115.1 

61 Strong Innovator 

115.0 

64 Strong Innovator 

91.6  122 Moderate Innovator 

96.0  118 Moderate Innovator 

77 

6.0 
-0.7 
-0.9 
-1.5 
-2.6 
-1.9 
-2.4 
-0.7 

0.1 
-4.5 

-3.6 
-6.6 
-0.1 
-4.7 
-1.9 
-0.6 

-2.1 
-6.8 

3.4 
-1.6 
3.1 
3.5 

-1.0 
-4.5 
-2.5 
0.4 

-1.8 
3.6 
1.7 
5.0 
2.4 
1.2 
2.0 
-0.1 
-1.6 
-2.0 
0.8 
-2.3 
-3.0 
-2.4 
-4.1 
-2.0 
-4.3 

Out-
liers 
-3 
0 
-8 
0 
-1 
-7 

-3 
4 

0 

-3 
-3 
-3 
-7 

0 
0 
0 
0 

-2 
-5 
-4 
-2 
-8 
0 
-1 
0 
-5 
0 
-1 
0 
0 
-1 
0 
0 
0 

-1 

0 

-8 
-2 
0 
-2 
0 
1 
0 
-3 

-2 
2 

 
 
 
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
Region 
NL13  Drenthe 
NL21  Overijssel 
NL22  Gelderland 
NL23  Flevoland 
NL31  Utrecht 
NL32  Noord-Holland 
NL33  Zuid-Holland 
NL34  Zeeland 
NL41  Noord-Brabant 
NL42  Limburg 
Austria 
AT 
AT1  Ostösterreich 
AT2  Südösterreich 
AT3  Westösterreich 
PL 
Poland 
PL21  Malopolskie 
PL22  Slaskie 
PL41  Wielkopolskie 
PL42  Zachodniopomorskie 
PL43  Lubuskie 
PL51  Dolnoslaskie 
PL52  Opolskie 
PL61  Kujawsko-Pomorskie 
PL62  Warminsko-Mazurskie 
PL63  Pomorskie 
PL71  Lódzkie 
PL72  Swietokrzyskie 
PL81  Lubelskie 
PL82  Podkarpackie 
PL84  Podlaskie 
PL91  Warszawski stoleczny 
PL92  Mazowiecki regionalny 
PT 
PT11  Norte 
PT15  Algarve 
PT16  Centro 
PT17  Lisboa 
PT18  Alentejo 

Portugal 

PT2 

Região Autónoma dos 
Açores 
Região Autónoma da 
PT3 
Madeira 
RO  Romania 
RO11  Nord-Vest 
RO12  Centru 
RO21  Nord-Est 
RO22  Sud-Est 
RO31  Sud - Muntenia 
RO32  Bucuresti - Ilfov 
RO41  Sud-Vest Oltenia 
RO42  Vest 
SI 
SI03  Vzhodna Slovenija 
SI04  Zahodna Slovenija 
SK  Slovakia 
SK01  Bratislavský kraj 
SK02  Západné Slovensko 
SK03  Stredné Slovensko 
SK04  Východné Slovensko 

Slovenia 

MAD results 

RIS 2021 results 

Differences 

RII  Rank  Performance group 

RII 

RII  Rank 

Performance  
group 
89.2  135 Moderate Innovator 

110.8 
125.4 
105.9 
131.7 
132.0 
126.3 

74 Strong Innovator 
42 Innovation Leader 
86 Strong Innovator 
27 Innovation Leader 
25 Innovation Leader 
40 Innovation Leader 

93.4  125 Moderate Innovator 

112.1 
122.2 
108.9 
130.4 
130.3 
123.4 

70 Strong Innovator 
45 Strong Innovator 
77 Strong Innovator 
31 Innovation Leader 
33 Innovation Leader 
43 Strong Innovator 

90.3  128 Moderate Innovator 

94.1  124 Moderate Innovator 

132.0 
126.9 

24 Innovation Leader 
37 Innovation Leader 

128.7 
121.6 

36 Innovation Leader 
46 Strong Innovator 

124.5 
116.6 
114.4 

44 Strong Innovator 
54 Strong Innovator 
66 Strong Innovator 

123.7 
116.4 
113.6 

42 Strong Innovator 
57 Strong Innovator 
68 Strong Innovator 

82.2  147 Moderate Innovator 
56.1  203 Emerging Innovator 
58.8  197 Emerging Innovator 
46.3  220 Emerging Innovator 
50.3  214 Emerging Innovator 
67.6  173 Emerging Innovator 
53.2  210 Emerging Innovator 
54.3  207 Emerging Innovator 
45.7  222 Emerging Innovator 
64.1  183 Emerging Innovator 
59.0  196 Emerging Innovator 
45.1  223 Emerging Innovator 
55.0  204 Emerging Innovator 
63.2  185 Emerging Innovator 
48.7  217 Emerging Innovator 
99.5  107 Moderate Innovator 
39.3  227 Emerging Innovator 

73.3  168 Moderate Innovator 
54.1  207 Emerging Innovator 
53.4  208 Emerging Innovator 
45.2  222 Emerging Innovator 
46.6  220 Emerging Innovator 
63.9  182 Emerging Innovator 
50.8  214 Emerging Innovator 
50.7  215 Emerging Innovator 
42.9  224 Emerging Innovator 
60.5  187 Emerging Innovator 
55.7  203 Emerging Innovator 
42.1  225 Emerging Innovator 
52.7  209 Emerging Innovator 
57.9  196 Emerging Innovator 
47.5  219 Emerging Innovator 
87.8  138 Moderate Innovator 
38.2  227 Emerging Innovator 

92.9  115 Moderate Innovator 
57.8  201 Emerging Innovator 
86.5  140 Moderate Innovator 

106.2 

84 Strong Innovator 

71.8  167 Moderate Innovator 

81.5  150 Moderate Innovator 
55.6  204 Emerging Innovator 
79.1  157 Moderate Innovator 
94.1  123 Moderate Innovator 
65.6  179 Emerging Innovator 

52.7  212 Emerging Innovator 

49.9  217 Emerging Innovator 

59.7  195 Emerging Innovator 

56.5  200 Emerging Innovator 

32.6  232 Emerging Innovator 
27.2  236 Emerging Innovator 
23.5  237 Emerging Innovator 
18.1  240 Emerging Innovator 
22.5  238 Emerging Innovator 
67.4  175 Emerging Innovator 
20.9  239 Emerging Innovator 
33.0  231 Emerging Innovator 

31.4  231 Emerging Innovator 
26.5  236 Emerging Innovator 
22.7  237 Emerging Innovator 
17.9  240 Emerging Innovator 
19.6  238 Emerging Innovator 
59.5  192 Emerging Innovator 
19.6  239 Emerging Innovator 
30.9  233 Emerging Innovator 

80.9  152 Moderate Innovator 

103.9 

95 Strong Innovator 

80.2  155 Moderate Innovator 
98.0  113 Moderate Innovator 

90.0  129 Moderate Innovator 
52.9  211 Emerging Innovator 
57.9  199 Emerging Innovator 
54.7  205 Emerging Innovator 

88.2  137 Moderate Innovator 
56.7  198 Emerging Innovator 
59.9  189 Emerging Innovator 
57.3  197 Emerging Innovator 

78 

Out-
liers 
0 
1 
0 
-1 
5 
-9 
-7 
0 
-1 
-8 

-3 
-6 
0 

0 
0 
-2 
0 
0 
0 
3 
0 
-2 
0 
-1 
2 
0 
0 
0 
-1 
0 

-3 
0 
2 
0 
0 

0 

0 

7 
3 
5 
4 
2 
1 
7 
8 

-3 
0 

2 
0 
0 
-2 

-4.2 
-1.3 
3.3 
-3.0 
1.3 
1.7 
2.9 
-3.9 
3.3 
5.3 

0.8 
0.2 
0.8 

8.9 
2.0 
5.4 
1.1 
3.7 
3.7 
2.4 
3.6 
2.8 
3.7 
3.3 
2.9 
2.3 
5.3 
1.2 
11.7 
1.1 

11.4 
2.2 
7.4 
12.0 
6.2 

2.8 

3.2 

1.2 
0.7 
0.9 
0.2 
2.9 
7.9 
1.3 
2.0 

0.7 
5.8 

1.9 
-3.8 
-2.0 
-2.6 

 
 
 
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
MAD results 

RIS 2021 results 

Differences 

RII  Rank 

Performance  
group 

RII  Rank  Performance group 

RII 

Out-
liers 

Finland 

Region 
FI 
FI1B  Helsinki-Uusimaa 
FI1C  Etelä-Suomi 
FI19  Länsi-Suomi 
FI1D  Pohjois- ja Itä-Suomi 
FI2  Åland 
SE  Sweden 
SE11  Stockholm 
SE12  Östra Mellansverige 
SE21  Småland med öarna 
SE22  Sydsverige 
SE23  Västsverige 
SE31  Norra Mellansverige 
SE32  Mellersta Norrland 
SE33  Övre Norrland 
NO  Norway 
NO01  Oslo og Akershus 
NO02  Hedmark og Oppland 
NO03  Sør-Østlandet 
NO04  Agder og Rogaland 
NO05  Vestlandet 
NO06  Trøndelag 
NO07  Nord-Norge 
CH  Switzerland 
CH01  Région lémanique 
CH02  Espace Mittelland 
CH03  Nordwestschweiz 
CH04  Zürich 
CH05  Ostschweiz 
CH06  Zentralschweiz 
CH07  Ticino 
RS  Serbia 
RS11  Belgrade 
RS12  Vojvodina 

4 Innovation Leader 
151.2 
69 Strong Innovator 
112.1 
36 Innovation Leader 
127.6 
115.9 
59 Strong Innovator 
101.2  102 Strong Innovator 

153.7 
129.0 
105.2 
146.9 
136.4 

2 Innovation Leader 
31 Innovation Leader 
91 Strong Innovator 
5 Innovation Leader 
16 Innovation Leader 

154.3 
115.8 
130.9 
116.6 
105.2 

155.2 
131.2 
109.5 
144.5 
138.4 

2 Innovation Leader 
59 Strong Innovator 
28 Innovation Leader 
56 Strong Innovator 
94 Strong Innovator 

1 Innovation Leader 
27 Innovation Leader 
75 Strong Innovator 
9 Innovation Leader 
13 Innovation Leader 

89.8  131 Moderate Innovator 
90.9  126 Moderate Innovator 

96.8  117 Moderate Innovator 
96.9  116 Moderate Innovator 

116.3 

56 Strong Innovator 

116.3 

58 Strong Innovator 

138.7 

13 Innovation Leader 

139.1 

12 Innovation Leader 

86.7  139 Moderate Innovator 

91.0  129 Moderate Innovator 

100.1  105 Strong Innovator 
85 Strong Innovator 
106.0 
64 Strong Innovator 
114.5 
20 Innovation Leader 
134.2 

103.8 
106.7 
115.5 
130.5 

96 Strong Innovator 
90 Strong Innovator 
61 Strong Innovator 
30 Innovation Leader 

97.5  109 Moderate Innovator 

98.8  108 Moderate Innovator 

128.4 
123.0 
133.7 
141.2 
125.5 
126.6 
141.3 

34 Innovation Leader 
47 Strong Innovator 
22 Innovation Leader 
10 Innovation Leader 
41 Innovation Leader 
38 Innovation Leader 
9 Innovation Leader 

133.1 
128.7 
139.2 
147.9 
133.3 
135.4 
144.5 

24 Innovation Leader 
37 Innovation Leader 
11 Innovation Leader 
6 Innovation Leader 
22 Innovation Leader 
18 Innovation Leader 
8 Innovation Leader 

-3.0 
-3.7 
-3.4 
-0.8 
-4.0 

-1.6 
-2.2 
-4.2 
2.4 
-2.1 
-7.0 
-6.0 
0.0 

-0.4 
-4.3 
-3.6 
-0.7 
-1.0 
3.7 
-1.3 

-4.8 
-5.7 
-5.5 
-6.7 
-7.8 
-8.8 
-3.2 

92.8  117 Moderate Innovator 
66.2  178 Emerging Innovator 

82.1  149 Moderate Innovator 
60.1  188 Emerging Innovator 

10.7 
6.1 

RS21 

61.0  190 Emerging Innovator 

60.7  191 Emerging Innovator 

Šumadija and Western 
Serbia 
Southern and Eastern 
Serbia 
RS22 
UK  United Kingdom 
115.6 
UKC  North East 
UKD  North West 
116.9 
UKE  Yorkshire and The Humber  117.1 
122.8 
UKF  East Midlands 
126.5 
UKG  West Midlands 
137.3 
UKH  East of England 
139.5 
UKI  London 
145.8 
UKJ  South East 
130.1 
UKK  South West  
116.5 
UKL  Wales 
123.9 
UKM  Scotland 
106.2 
UKN  Northern Ireland 

112.0 
114.8 
114.7 
120.0 
122.8 
133.2 
135.6 
140.3 
126.3 
115.2 
119.9 
106.4 
Note: Performance groups or rank increases highlighted in green, decreases in red. 

60 Strong Innovator 
52 Strong Innovator 
51 Strong Innovator 
48 Strong Innovator 
39 Innovation Leader 
15 Innovation Leader 
11 Innovation Leader 
6 Innovation Leader 
28 Innovation Leader 
55 Strong Innovator 
46 Strong Innovator 
83 Strong Innovator 

55.8  202 Emerging Innovator 

56.4  201 Emerging Innovator 

72 Strong Innovator 
65 Strong Innovator 
66 Strong Innovator 
49 Strong Innovator 
44 Strong Innovator 
23 Innovation Leader 
16 Innovation Leader 
10 Innovation Leader 
39 Innovation Leader 
62 Strong Innovator 
50 Strong Innovator 
92 Strong Innovator 

4.9 

4.6 

3.6 
2.1 
2.4 
2.9 
3.7 
4.1 
4.0 
5.5 
3.8 
1.3 
4.0 
-0.2 

79 

-15 
-7 
-9 
-7 
-7 

-10 
-10 
0 
-8 
-14 
0 
-6 
-7 

4 
4 
2 
-2 
-2 
-8 
-1 

-13 
0 
-5 
11 
3 
-5 
1 

-8 
-9 

-5 

0 

0 
3 
-5 
-3 
2 
-7 
3 
-1 
-2 
0 
-2 
4 

 
 
 
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
 
 
 
Table  36  RIS  2021  results  using  the  current  method  (z-score  +/-  2  cut-off  values)  vs.  Z-score 
method 

Region  
BE  Belgium 

Z-score results 

RIS 2021 results 

Differences 

RII  Rank  Performance group  RII  Rank  Performance group 

RII 

Out-
liers 

Région de Bruxelles-
Capitale / Brussels 
Hoofdstedelijk Gewest 

104.9 

141.6 

90 Strong Innovator 

6 Innovation Leader 

134.3 
128.8 
113.0 

107.2 
126.6 
113.2 

98.6  113 Moderate Innovator 

15 Innovation Leader 
25 Innovation Leader 
67 Strong Innovator 

80 Strong Innovator 
33 Innovation Leader 
64 Strong Innovator 

36.7  233 Emerging Innovator 
43.4  230 Emerging Innovator 
44.1  229 Emerging Innovator 
35.6  235 Emerging Innovator 
65.6  189 Emerging Innovator 
45.1  228 Emerging Innovator 

87.6  144 Moderate Innovator 
74.7  172 Moderate Innovator 
52.9  221 Emerging Innovator 
80.8  158 Moderate Innovator 
88.6  139 Moderate Innovator 
78.1  165 Moderate Innovator 
82.2  155 Moderate Innovator 

BE1 
BE2  Vlaams Gewest 
BE3  Région Wallonne 
BG  Bulgaria 
BG31  Severozapaden 
BG32  Severen tsentralen 
BG33  Severoiztochen 
BG34  Yugoiztochen 
BG41  Yugozapaden 
BG42  Yuzhen tsentralen 
CZ  Czech Republic 
CZ01  Praha 
CZ02  Strední Cechy 
CZ03  Jihozápad 
CZ04  Severozápad 
CZ05  Severovýchod 
CZ06  Jihovýchod 
CZ07  Strední Morava 
CZ08  Moravskoslezsko 
DK  Denmark 
DK01  Hovedstaden 
DK02  Sjælland 
DK03  Syddanmark 
DK04  Midtjylland 
DK05  Nordjylland 
DE  Germany 
DE11  Stuttgart 
DE12  Karlsruhe 
DE13  Freiburg 
DE14  Tübingen 
DE21  Oberbayern 
DE22  Niederbayern 
DE23  Oberpfalz 
DE24  Oberfranken 
DE25  Mittelfranken 
DE26  Unterfranken 
DE27  Schwaben 
DE3  Berlin 
DE4  Brandenburg 
DE5  Bremen 
DE6  Hamburg 
DE71  Darmstadt 
DE72  Gießen 
DE73  Kassel 
98.8  108 Moderate Innovator 
DE8  Mecklenburg-Vorpommern  94.7  125 Moderate Innovator 
DE91  Braunschweig 
DE92  Hannover 
DE93  Lüneburg 
DE94  Weser-Ems 
DEA1  Düsseldorf 
DEA2  Köln 
DEA3  Münster 
DEA4  Detmold 

60 Strong Innovator 
74 Strong Innovator 
37 Strong Innovator 
52 Strong Innovator 
83 Strong Innovator 
7 Innovation Leader 

32 Innovation Leader 
13 Innovation Leader 
44 Strong Innovator 
24 Innovation Leader 
4 Innovation Leader 

69 Strong Innovator 
27 Innovation Leader 
39 Strong Innovator 
46 Strong Innovator 

79 Strong Innovator 
36 Innovation Leader 
82 Strong Innovator 
57 Strong Innovator 

92.6  130 Moderate Innovator 
88.7  138 Moderate Innovator 

22 Innovation Leader 
61 Strong Innovator 

115.7 
109.8 
124.6 
117.3 
106.8 
140.4 

127.5 
135.5 
120.7 
129.3 
144.1 

112.3 
128.4 
122.9 
120.3 

107.6 
125.2 
107.1 
116.2 

99.1  106 Moderate Innovator 

98.4  115 Moderate Innovator 

130.6 
115.6 

80 

137.6 
131.9 
113.1 

14 Innovation Leader 
25 Innovation Leader 
69 Strong Innovator 

29.1  234 Emerging Innovator 
36.7  229 Emerging Innovator 
36.2  230 Emerging Innovator 
27.8  235 Emerging Innovator 
59.8  191 Emerging Innovator 
37.9  228 Emerging Innovator 

108.3 

80 Strong Innovator 

89.2  134 Moderate Innovator 
72.4  169 Moderate Innovator 
48.1  218 Emerging Innovator 
80.4  154 Moderate Innovator 
90.3  131 Moderate Innovator 
76.6  161 Moderate Innovator 
81.0  151 Moderate Innovator 

149.2 

5 Innovation Leader 

98.8  106 Moderate Innovator 

107.6 
131.5 
115.0 

130.4 
145.0 
125.3 
134.9 
154.1 

86 Strong Innovator 
26 Innovation Leader 
63 Strong Innovator 

32 Innovation Leader 
7 Innovation Leader 
40 Innovation Leader 
19 Innovation Leader 
3 Innovation Leader 

99.3  105 Moderate Innovator 

117.4 
111.7 
130.0 
120.1 
107.9 
149.3 

53 Strong Innovator 
73 Strong Innovator 
34 Innovation Leader 
48 Strong Innovator 
83 Strong Innovator 
4 Innovation Leader 

97.9  114 Moderate Innovator 

114.0 
136.1 
129.5 
125.0 

67 Strong Innovator 
15 Innovation Leader 
35 Innovation Leader 
41 Innovation Leader 

98.7  110 Moderate Innovator 
94.1  122 Moderate Innovator 

135.5 
118.4 

17 Innovation Leader 
51 Strong Innovator 

90.9  130 Moderate Innovator 
86.1  142 Moderate Innovator 

108.9 
130.6 
107.8 
117.9 

76 Strong Innovator 
29 Innovation Leader 
84 Strong Innovator 
52 Strong Innovator 

-3.3 
-3.0 
-0.1 

7.5 
6.6 
7.9 
7.8 
5.8 
7.3 

-3.4 
-1.6 
2.2 
4.7 
0.4 
-1.6 
1.5 
1.2 

-7.7 
-0.3 
-0.4 
-4.9 
-1.8 

-2.9 
-9.5 
-4.6 
-5.6 
-9.9 
-0.2 
-1.7 
-1.9 
-5.4 
-2.8 
-1.1 
-8.9 
0.5 
-1.6 
-7.8 
-6.6 
-4.8 
0.1 
0.5 
-4.9 
-2.8 
1.6 
2.6 
-1.4 
-5.4 
-0.7 
-1.7 

14 
6 
7 

9 
1 
3 
4 
8 
6 

13 
0 
0 
0 
0 
0 
0 
2 

35 
1 
0 
7 
3 

18 
17 
7 
12 
28 
0 
0 
1 
9 
4 
1 
8 
0 
6 
0 
0 
0 
3 
0 
6 
1 
1 
0 
0 
8 
0 
8 

 
 
 
   
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
Z-score results 

RIS 2021 results 

Differences 

RII  Rank  Performance group  RII  Rank  Performance group 

109.4 

101.5 
112.6 

107.6 
106.1 

75 Strong Innovator 

96.4  123 Moderate Innovator 

94.5  126 Moderate Innovator 

98.8  109 Moderate Innovator 

78 Strong Innovator 
86 Strong Innovator 

100.1  104 Strong Innovator 
29 Innovation Leader 
128.0 
85 Strong Innovator 
106.2 
122.8 
40 Strong Innovator 
100.4  103 Strong Innovator 
55 Strong Innovator 
116.4 

Region  
DEA5  Arnsberg 
DEB1  Koblenz 
DEB2  Trier 
DEB3  Rheinhessen-Pfalz 
DEC  Saarland 
DED2 Dresden 
DED4 Chemnitz 
DED5 Leipzig 
DEE  Sachsen-Anhalt 
DEF  Schleswig-Holstein 
DEG  Thüringen 
EE  Estonia 
IE 
Ireland 
IE04  Northern and Western 
IE05  Southern 
IE06  Eastern and Midland 
EL  Greece 
90.9  133 Moderate Innovator 
EL3  Attiki 
69.0  182 Emerging Innovator 
EL41  Voreio Aigaio 
51.4  224 Emerging Innovator 
EL42  Notio Aigaio 
EL43  Kriti 
83.5  150 Moderate Innovator 
EL51  Anatoliki Makedonia, Thraki  64.9  191 Emerging Innovator 
87.8  143 Moderate Innovator 
EL52  Kentriki Makedonia 
58.1  215 Emerging Innovator 
EL53  Dytiki Makedonia 
72.1  174 Moderate Innovator 
EL54  Ipeiros 
76.2  168 Moderate Innovator 
EL61  Thessalia 
63.0  203 Emerging Innovator 
EL62  Ionia Nisia 
74.9  171 Moderate Innovator 
EL63  Dytiki Ellada 
63.4  198 Emerging Innovator 
EL64  Sterea Ellada 
EL65  Peloponnisos 
60.2  208 Emerging Innovator 
ES  Spain 
ES11  Galicia 
ES12  Principado de Asturias 
ES13  Cantabria 
ES21  País Vasco 

81.7  156 Moderate Innovator 
79.7  160 Moderate Innovator 
79.1  163 Moderate Innovator 

99 Strong Innovator 
68 Strong Innovator 

88 Strong Innovator 

105.2 

100.6  102 Strong Innovator 

98.7  110 Moderate Innovator 
82.9  154 Moderate Innovator 
83.3  151 Moderate Innovator 

Comunidad Foral de 
ES22 
Navarra 
ES23  La Rioja 
ES24  Aragón 
ES3  Comunidad de Madrid 
ES41  Castilla y León 
ES42  Castilla-la Mancha 
ES43  Extremadura 
ES51  Cataluña 
92.0  132 Moderate Innovator 
ES52  Comunidad Valenciana 
70.5  178 Moderate Innovator 
ES53  Illes Balears 
72.8  173 Moderate Innovator 
ES61  Andalucía 
ES62  Región de Murcia 
79.5  161 Moderate Innovator 
ES63  Ciudad Autónoma de Ceuta  38.0  232 Emerging Innovator 

78.9  164 Moderate Innovator 
68.8  183 Emerging Innovator 
63.2  200 Emerging Innovator 

101.1  100 Strong Innovator 

110.6 

74 Strong Innovator 

97.5  115 Moderate Innovator 
98.7  109 Moderate Innovator 

20 Innovation Leader 
93 Strong Innovator 
38 Innovation Leader 

134.8 
106.3 
127.7 
101.3  101 Strong Innovator 
47 Strong Innovator 
120.2 

94.2  121 Moderate Innovator 

108.2 
107.6 

81 Strong Innovator 
85 Strong Innovator 

95.1  120 Moderate Innovator 

103.6 
115.6 

97 Strong Innovator 
60 Strong Innovator 

89.6  133 Moderate Innovator 
64.3  181 Emerging Innovator 
46.4  221 Emerging Innovator 
82.5  148 Moderate Innovator 
61.1  186 Emerging Innovator 
84.7  144 Moderate Innovator 
52.1  212 Emerging Innovator 
68.8  172 Emerging Innovator 
74.2  167 Moderate Innovator 
58.9  193 Emerging Innovator 
69.2  171 Emerging Innovator 
59.8  190 Emerging Innovator 
55.6  205 Emerging Innovator 

78.1  158 Moderate Innovator 
75.7  163 Moderate Innovator 
75.0  164 Moderate Innovator 

106.7 

91 Strong Innovator 

98.8  107 Moderate Innovator 
80.8  152 Moderate Innovator 
79.9  156 Moderate Innovator 

101.8  100 Strong Innovator 

74.5  165 Moderate Innovator 
62.8  184 Emerging Innovator 
56.6  199 Emerging Innovator 

102.3 

98 Strong Innovator 

91.5  128 Moderate Innovator 
65.5  180 Emerging Innovator 
67.1  177 Emerging Innovator 
76.2  162 Moderate Innovator 
31.0  232 Emerging Innovator 

Ciudad Autónoma de 
Melilla 

ES64 
ES7  Canarias 
France 
FR 
FR1 
Île de France 
FRB  Centre - Val de Loire 
Bourgogne - Franche-
Comté 

FRC 
FRD  Normandie 
FRE  Hauts-de-France 

47.9  226 Emerging Innovator 
58.3  214 Emerging Innovator 

41.8  226 Emerging Innovator 
50.9  213 Emerging Innovator 

128.8 

26 Innovation Leader 

134.7 

21 Innovation Leader 

88.2  141 Moderate Innovator 

87.2  139 Moderate Innovator 

89.4  136 Moderate Innovator 
79.4  162 Moderate Innovator 
87.8  142 Moderate Innovator 

88.2  136 Moderate Innovator 
76.7  160 Moderate Innovator 
85.8  143 Moderate Innovator 

81 

RII 

-1.3 
1.3 
1.4 
-6.9 
-0.1 
-4.9 
-0.8 
-3.9 
0.3 
-0.6 
-1.5 

1.3 
-2.1 
-3.0 

1.2 
4.7 
5.0 
1.0 
3.7 
3.1 
6.0 
3.2 
2.0 
4.0 
5.6 
3.5 
4.6 

3.7 
4.0 
4.1 
-1.5 

-0.2 
2.1 
3.4 
-1.2 
4.5 
6.0 
6.6 
-1.2 
0.4 
5.0 
5.8 
3.4 
7.0 

6.1 
7.3 

-5.8 
1.0 

1.2 
2.8 
2.1 

Out-
liers 
5 
1 
0 
10 
1 
1 
0 
7 
0 
0 
0 

0 
0 
6 

0 
2 
0 
1 
1 
9 
8 
2 
4 
3 
0 
0 
0 

0 
0 
2 
6 

0 
3 
0 
8 
0 
0 
0 
0 
0 
0 
0 
0 
15 

8 
0 

8 
0 

0 
0 
0 

 
 
 
   
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
Z-score results 

RIS 2021 results 

Differences 

Region  
FRF  Grand Est 
FRG  Pays de la Loire 
FRH  Bretagne 
FRI  Nouvelle-Aquitaine 
FRJ  Occitanie 
FRK  Auvergne - Rhône-Alpes 

Provence-Alpes-Côte 
FRL 
d'Azur 
FRM  Corse 

RII  Rank  Performance group  RII  Rank  Performance group 

RII 

95.6  124 Moderate Innovator 
97.8  118 Moderate Innovator 

95.4  119 Moderate Innovator 
98.7  111 Moderate Innovator 

106.3 

84 Strong Innovator 

108.5 

79 Strong Innovator 

92.8  128 Moderate Innovator 

91.8  127 Moderate Innovator 

110.9 
112.1 

71 Strong Innovator 
70 Strong Innovator 

116.9 
116.7 

54 Strong Innovator 
55 Strong Innovator 

104.9 

89 Strong Innovator 

106.8 

88 Strong Innovator 

51.9  223 Emerging Innovator 

44.7  223 Emerging Innovator 

0.2 
-0.9 
-2.1 
1.0 
-6.0 
-4.6 

-1.8 
7.2 

Régions ultrapériphériques 
françaises 

75.7  170 Moderate Innovator 

FRY 
HR  Croatia 
HR02  Panonska Hrvatska 
HR03  Jadranska Hrvatska 
HR05  Grad Zagreb 
HR06  Sjeverna Hrvatska 
IT 
Italy 
ITC1  Piemonte 
ITC2  Valle d'Aosta/Vallée d'Aoste  65.6  188 Emerging Innovator 
85.8  147 Moderate Innovator 
ITC3  Liguria 
ITC4  Lombardia 

85.8  148 Moderate Innovator 
64.5  193 Emerging Innovator 
90.3  135 Moderate Innovator 
88.2  140 Moderate Innovator 

94 Strong Innovator 

103.5 

69.5  170 Emerging Innovator 

6.2 

82.9  146 Moderate Innovator 
61.4  185 Emerging Innovator 
89.0  135 Moderate Innovator 
86.2  141 Moderate Innovator 

63.0  183 Emerging Innovator 
86.3  140 Moderate Innovator 

107.9 

82 Strong Innovator 

93.2  126 Moderate Innovator 

98.6  111 Moderate Innovator  102.1 

99 Strong Innovator 

Provincia Autonoma 
Bolzano/Bozen 

ITH1 
ITH2  Provincia Autonoma Trento  103.4 
104.6 
ITH3  Veneto 
103.6 
ITH4  Friuli-Venezia Giulia 
ITH5  Emilia-Romagna 
107.1 
ITI1  Toscana 
ITI2  Umbria 
ITI3  Marche 
ITI4  Lazio 
ITF1  Abruzzo 
ITF2  Molise 
ITF3  Campania 
ITF4  Puglia 
ITF5  Basilicata 
ITF6  Calabria 
ITG1  Sicilia 
ITG2  Sardegna 
CY  Cyprus 
Latvia 
LV 
LT 
Lithuania 
LT01  Sostinės regionas 

92.5  131 Moderate Innovator 

106.7 
108.7 
107.0 
112.0 

98.6  112 Moderate Innovator 
89.7  132 Moderate Innovator 

95 Strong Innovator 
91 Strong Innovator 
93 Strong Innovator 
81 Strong Innovator 

89 Strong Innovator 
78 Strong Innovator 
87 Strong Innovator 
71 Strong Innovator 
98.6  112 Moderate Innovator  100.9  102 Strong Innovator 
98.1  116 Moderate Innovator 
89.3  137 Moderate Innovator 
97.2  121 Moderate Innovator  100.8  103 Strong Innovator 
84.1  149 Moderate Innovator 
81.4  157 Moderate Innovator 
83.3  152 Moderate Innovator 
76.2  169 Moderate Innovator 
78.0  166 Moderate Innovator 
68.7  184 Emerging Innovator 
71.2  176 Moderate Innovator 
69.8  179 Emerging Innovator 

83.7  145 Moderate Innovator 
80.5  153 Moderate Innovator 
82.6  147 Moderate Innovator 
74.2  166 Moderate Innovator 
77.4  159 Moderate Innovator 
66.1  178 Emerging Innovator 
68.6  173 Emerging Innovator 
67.2  176 Emerging Innovator 

98.9  107 Moderate Innovator  103.9 

95 Strong Innovator 

-5.0 

Vidurio ir vakarų Lietuvos 
regionas  
Luxembourg 

LT02 
LU 
HU  Hungary 
HU11  Budapest 
HU12  Pest 
HU21  Közép-Dunántúl 
HU22  Nyugat-Dunántúl 
HU23  Dél-Dunántúl 
HU31  Észak-Magyarország 
HU32  Észak-Alföld 
HU33  Dél-Alföld 
MT  Malta 
NL  Netherlands 
NL11  Groningen 
NL12  Friesland 

69.1  181 Emerging Innovator 

67.6  174 Emerging Innovator 

1.4 

97.6  119 Moderate Innovator 
70.9  177 Moderate Innovator 
62.4  204 Emerging Innovator 
59.7  210 Emerging Innovator 
56.4  218 Emerging Innovator 
57.5  216 Emerging Innovator 
58.7  213 Emerging Innovator 
63.9  196 Emerging Innovator 

99.7  104 Moderate Innovator 
67.5  175 Emerging Innovator 
57.9  195 Emerging Innovator 
54.9  206 Emerging Innovator 
50.1  216 Emerging Innovator 
52.2  211 Emerging Innovator 
52.2  210 Emerging Innovator 
58.5  194 Emerging Innovator 

118.2 

49 Strong Innovator 

115.0 

64 Strong Innovator 

99.5  105 Moderate Innovator 

96.0  118 Moderate Innovator 

82 

-2.1 
3.4 
4.5 
4.8 
6.2 
5.3 
6.5 
5.4 

3.3 
3.5 

Out-
liers 
0 
0 
0 
0 
7 
0 

0 
4 

0 

4 
0 
4 
4 

0 
0 
0 
7 

1 
0 
10 
1 
0 
0 
0 
8 
0 
0 
0 
0 
0 
0 
0 
0 
0 

6 

1 

6 
1 
0 
1 
0 
1 
0 
0 

7 
2 

2.9 
3.1 
1.3 
2.0 

-3.5 
2.7 
-0.5 
-4.5 

-0.8 
-3.3 
-4.1 
-3.4 
-4.9 
-2.3 
-0.5 
-0.4 
-3.6 
0.4 
0.9 
0.7 
1.9 
0.6 
2.6 
2.6 
2.6 

 
 
 
   
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
Region  
NL13  Drenthe 
NL21  Overijssel 
NL22  Gelderland 
NL23  Flevoland 
NL31  Utrecht 
NL32  Noord-Holland 
NL33  Zuid-Holland 
NL34  Zeeland 
NL41  Noord-Brabant 
NL42  Limburg 
Austria 
AT 
AT1  Ostösterreich 
AT2  Südösterreich 
AT3  Westösterreich 
PL 
Poland 
PL21  Malopolskie 
PL22  Slaskie 
PL41  Wielkopolskie 
PL42  Zachodniopomorskie 
PL43  Lubuskie 
PL51  Dolnoslaskie 
PL52  Opolskie 
PL61  Kujawsko-Pomorskie 
PL62  Warminsko-Mazurskie 
PL63  Pomorskie 
PL71  Lódzkie 
PL72  Swietokrzyskie 
PL81  Lubelskie 
PL82  Podkarpackie 
PL84  Podlaskie 
PL91  Warszawski stoleczny 
PL92  Mazowiecki regionalny 
PT 
PT11  Norte 
PT15  Algarve 
PT16  Centro 
PT17  Lisboa 
PT18  Alentejo 

Portugal 

PT2 

Região Autónoma dos 
Açores 
Região Autónoma da 
PT3 
Madeira 
RO  Romania 
RO11  Nord-Vest 
RO12  Centru 
RO21  Nord-Est 
RO22  Sud-Est 
RO31  Sud - Muntenia 
RO32  Bucuresti - Ilfov 
RO41  Sud-Vest Oltenia 
RO42  Vest 
SI 
SI03  Vzhodna Slovenija 
SI04  Zahodna Slovenija 
SK  Slovakia 
SK01  Bratislavský kraj 
SK02  Západné Slovensko 
SK03  Stredné Slovensko 
SK04  Východné Slovensko 

Slovenia 

Z-score results 

RIS 2021 results 

Differences 

RII  Rank  Performance group  RII  Rank  Performance group 

RII 

97.5  120 Moderate Innovator 

93.4  125 Moderate Innovator 

113.3 
121.6 
108.7 
132.1 
127.7 
122.5 

63 Strong Innovator 
42 Strong Innovator 
77 Strong Innovator 
17 Innovation Leader 
31 Innovation Leader 
41 Strong Innovator 

112.1 
122.2 
108.9 
130.4 
130.3 
123.4 

70 Strong Innovator 
45 Strong Innovator 
77 Strong Innovator 
31 Innovation Leader 
33 Innovation Leader 
43 Strong Innovator 

97.9  117 Moderate Innovator 

94.1  124 Moderate Innovator 

127.7 
120.9 

30 Innovation Leader 
43 Strong Innovator 

128.7 
121.6 

36 Innovation Leader 
46 Strong Innovator 

117.5 
113.2 
110.4 

51 Strong Innovator 
65 Strong Innovator 
72 Strong Innovator 

123.7 
116.4 
113.6 

42 Strong Innovator 
57 Strong Innovator 
68 Strong Innovator 

80.5  159 Moderate Innovator 
63.8  197 Emerging Innovator 
62.2  205 Emerging Innovator 
54.3  220 Emerging Innovator 
55.2  219 Emerging Innovator 
71.4  175 Moderate Innovator 
59.0  212 Emerging Innovator 
59.8  209 Emerging Innovator 
52.5  222 Emerging Innovator 
68.4  185 Emerging Innovator 
64.3  195 Emerging Innovator 
51.2  225 Emerging Innovator 
61.5  206 Emerging Innovator 
65.5  190 Emerging Innovator 
56.8  217 Emerging Innovator 
93.3  127 Moderate Innovator 
46.3  227 Emerging Innovator 

73.3  168 Moderate Innovator 
54.1  207 Emerging Innovator 
53.4  208 Emerging Innovator 
45.2  222 Emerging Innovator 
46.6  220 Emerging Innovator 
63.9  182 Emerging Innovator 
50.8  214 Emerging Innovator 
50.7  215 Emerging Innovator 
42.9  224 Emerging Innovator 
60.5  187 Emerging Innovator 
55.7  203 Emerging Innovator 
42.1  225 Emerging Innovator 
52.7  209 Emerging Innovator 
57.9  196 Emerging Innovator 
47.5  219 Emerging Innovator 
87.8  138 Moderate Innovator 
38.2  227 Emerging Innovator 

92.6  129 Moderate Innovator 
68.1  186 Emerging Innovator 
90.8  134 Moderate Innovator 

102.9 

96 Strong Innovator 

77.5  167 Moderate Innovator 

81.5  150 Moderate Innovator 
55.6  204 Emerging Innovator 
79.1  157 Moderate Innovator 
94.1  123 Moderate Innovator 
65.6  179 Emerging Innovator 

4.1 
1.2 
-0.6 
-0.2 
1.7 
-2.7 
-0.9 
3.8 
-1.0 
-0.6 

-6.2 
-3.2 
-3.1 

7.2 
9.8 
8.8 
9.1 
8.6 
7.5 
8.2 
9.2 
9.7 
7.9 
8.7 
9.1 
8.8 
7.6 
9.3 
5.5 
8.0 

11.2 
12.5 
11.7 
8.7 
11.9 

63.3  199 Emerging Innovator 

49.9  217 Emerging Innovator 

13.4 

69.7  180 Emerging Innovator 

56.5  200 Emerging Innovator 

13.1 

38.5  231 Emerging Innovator 
32.5  236 Emerging Innovator 
30.2  237 Emerging Innovator 
22.6  240 Emerging Innovator 
25.2  238 Emerging Innovator 
64.6  192 Emerging Innovator 
25.0  239 Emerging Innovator 
36.3  234 Emerging Innovator 

31.4  231 Emerging Innovator 
26.5  236 Emerging Innovator 
22.7  237 Emerging Innovator 
17.9  240 Emerging Innovator 
19.6  238 Emerging Innovator 
59.5  192 Emerging Innovator 
19.6  239 Emerging Innovator 
30.9  233 Emerging Innovator 

83.0  153 Moderate Innovator 
98.5  114 Moderate Innovator 

80.2  155 Moderate Innovator 
98.0  113 Moderate Innovator 

87.1  145 Moderate Innovator 
59.7  211 Emerging Innovator 
63.0  201 Emerging Innovator 
60.5  207 Emerging Innovator 

88.2  137 Moderate Innovator 
56.7  198 Emerging Innovator 
59.9  189 Emerging Innovator 
57.3  197 Emerging Innovator 

7.1 
6.0 
7.6 
4.7 
5.5 
5.1 
5.5 
5.4 

2.8 
0.5 

-1.1 
2.9 
3.1 
3.2 

83 

Out-
liers 
0 
1 
1 
2 
27 
2 
0 
0 
0 
0 

0 
7 
8 

8 
8 
0 
0 
0 
0 
3 
0 
2 
0 
6 
2 
0 
0 
0 
16 
1 

2 
2 
2 
2 
0 

9 

5 

11 
11 
11 
13 
11 
9 
18 
19 

0 
0 

11 
0 
0 
0 

 
 
 
   
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
Z-score results 

RIS 2021 results 

Differences 

RII  Rank  Performance group  RII  Rank  Performance group 

RII 

Out-
liers 

Finland 

Region  
FI 
FI1B  Helsinki-Uusimaa 
FI1C  Etelä-Suomi 
FI19  Länsi-Suomi 
FI1D  Pohjois- ja Itä-Suomi 
FI2  Åland 
SE  Sweden 
SE11  Stockholm 
SE12  Östra Mellansverige 
SE21  Småland med öarna 
SE22  Sydsverige 
SE23  Västsverige 
SE31  Norra Mellansverige 
SE32  Mellersta Norrland 
SE33  Övre Norrland 
NO  Norway 
NO01  Oslo og Akershus 
NO02  Hedmark og Oppland 
NO03  Sør-Østlandet 
NO04  Agder og Rogaland 
NO05  Vestlandet 
NO06  Trøndelag 
NO07  Nord-Norge 
CH  Switzerland 
CH01  Région lémanique 
CH02  Espace Mittelland 
CH03  Nordwestschweiz 
CH04  Zürich 
CH05  Ostschweiz 
CH06  Zentralschweiz 
CH07  Ticino 
RS  Serbia 
RS11  Belgrade 
RS12  Vojvodina 

148.7 
116.6 
128.0 
116.2 
104.5 

155.0 
131.7 
113.1 
143.7 
138.5 
101.9 
101.9 
120.2 

2 Innovation Leader 
54 Strong Innovator 
28 Innovation Leader 
56 Strong Innovator 
92 Strong Innovator 

1 Innovation Leader 
19 Innovation Leader 
66 Strong Innovator 
5 Innovation Leader 
10 Innovation Leader 
97 Strong Innovator 
98 Strong Innovator 
47 Strong Innovator 

154.3 
115.8 
130.9 
116.6 
105.2 

155.2 
131.2 
109.5 
144.5 
138.4 

2 Innovation Leader 
59 Strong Innovator 
28 Innovation Leader 
56 Strong Innovator 
94 Strong Innovator 

1 Innovation Leader 
27 Innovation Leader 
75 Strong Innovator 
9 Innovation Leader 
13 Innovation Leader 

96.8  117 Moderate Innovator 
96.9  116 Moderate Innovator 

116.3 

58 Strong Innovator 

139.4 

9 Innovation Leader 

139.1 

12 Innovation Leader 

96.4  122 Moderate Innovator 

91.0  129 Moderate Innovator 

87 Strong Innovator 
73 Strong Innovator 
53 Strong Innovator 
16 Innovation Leader 

105.7 
109.8 
117.0 
132.2 
100.7  101 Strong Innovator 

103.8 
106.7 
115.5 
130.5 

96 Strong Innovator 
90 Strong Innovator 
61 Strong Innovator 
30 Innovation Leader 

98.8  108 Moderate Innovator 

131.9 
125.4 
138.0 
147.5 
131.2 
131.3 
140.1 

18 Innovation Leader 
34 Innovation Leader 
11 Innovation Leader 
3 Innovation Leader 
21 Innovation Leader 
20 Innovation Leader 
8 Innovation Leader 

133.1 
128.7 
139.2 
147.9 
133.3 
135.4 
144.5 

24 Innovation Leader 
37 Innovation Leader 
11 Innovation Leader 
6 Innovation Leader 
22 Innovation Leader 
18 Innovation Leader 
8 Innovation Leader 

86.5  146 Moderate Innovator 
67.3  187 Emerging Innovator 

82.1  149 Moderate Innovator 
60.1  188 Emerging Innovator 

RS21 

64.3  194 Emerging Innovator 

63.0  202 Emerging Innovator 

Šumadija and Western 
Serbia 
Southern and Eastern 
Serbia 
RS22 
UK  United Kingdom 
113.6 
UKC  North East 
UKD  North West 
115.9 
UKE  Yorkshire and The Humber  116.2 
119.4 
UKF  East Midlands 
123.3 
UKG  West Midlands 
130.0 
UKH  East of England 
135.3 
UKI  London 
137.9 
UKJ  South East 
125.3 
UKK  South West  
117.7 
UKL  Wales 
120.4 
UKM  Scotland 
108.9 
UKN  Northern Ireland 

112.0 
114.8 
114.7 
120.0 
122.8 
133.2 
135.6 
140.3 
126.3 
115.2 
119.9 
106.4 
Note: Performance groups or rank increases highlighted in green, decreases in red. 

62 Strong Innovator 
59 Strong Innovator 
58 Strong Innovator 
48 Strong Innovator 
38 Strong Innovator 
23 Innovation Leader 
14 Innovation Leader 
12 Innovation Leader 
35 Innovation Leader 
50 Strong Innovator 
45 Strong Innovator 
76 Strong Innovator 

55.8  202 Emerging Innovator 

56.4  201 Emerging Innovator 

72 Strong Innovator 
65 Strong Innovator 
66 Strong Innovator 
49 Strong Innovator 
44 Strong Innovator 
23 Innovation Leader 
16 Innovation Leader 
10 Innovation Leader 
39 Innovation Leader 
62 Strong Innovator 
50 Strong Innovator 
92 Strong Innovator 

84 

-5.6 
0.8 
-2.9 
-0.4 
-0.7 

-0.3 
0.4 
3.7 
-0.8 
0.1 
5.1 
5.0 
3.8 

0.3 
5.4 
2.0 
3.1 
1.6 
1.8 
1.9 

-1.2 
-3.2 
-1.2 
-0.4 
-2.1 
-4.1 
-4.4 

4.4 
7.3 

7.2 

7.9 

1.6 
1.1 
1.5 
-0.5 
0.5 
-3.2 
-0.3 
-2.5 
-1.0 
2.5 
0.5 
2.5 

28 
1 
6 
2 
6 

42 
29 
8 
9 
14 
8 
2 
20 

38 
5 
2 
2 
0 
13 
8 

10 
8 
21 
27 
13 
9 
13 

9 
5 

9 

9 

4 
12 
4 
7 
8 
2 
18 
13 
4 
7 
5 
4 

 
 
 
   
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
 
 
  ANNEX E. Outliers identified and replaced, by region – RIS 2021 

Country/Region 
Belgium 
Région de Bruxelles-Capitale / Brussels 
Hoofdstedelijk Gewest 
Vlaams Gewest 
Région Wallonne 
Bulgaria 
Severozapaden 
Severen tsentralen 
Severoiztochen 
Yugoiztochen 
Yugozapaden 
Yuzhen tsentralen 
Czech Republic 
Praha 
Strední Cechy 
Jihozápad 
Severozápad 
Severovýchod 
Jihovýchod 
Strední Morava 
Moravskoslezsko 
Denmark 
Hovedstaden 
Sjælland 
Syddanmark 
Midtjylland 
Nordjylland 
Germany 
Stuttgart 
Karlsruhe 
Freiburg 
Tübingen 
Oberbayern 
Niederbayern 
Oberpfalz 
Oberfranken 
Mittelfranken 
Unterfranken 
Schwaben 
Berlin 
Brandenburg 
Bremen 
Hamburg 
Darmstadt 

BE 

BE1 
BE2 
BE3 
BG 
BG31 
BG32 
BG33 
BG34 
BG41 
BG42 
CZ 
CZ01 
CZ02 
CZ03 
CZ04 
CZ05 
CZ06 
CZ07 
CZ08 
DK 
DK01 
DK02 
DK03 
DK04 
DK05 
DE 
DE11 
DE12 
DE13 
DE14 
DE21 
DE22 
DE23 
DE24 
DE25 
DE26 
DE27 
DE3 
DE4 
DE5 
DE6 
DE71 

RIS 2021 (Z-score +/-2) 
Lower 
limit 

Upper 
limit 

Total 

Upper 
limit 

Box-plot 
Lower 
limit 

Total 

Upper 
limit 

MAD 
Lower 
limit 

Total 

Upper 
limit 

Z-score (+/-3) 
Lower 
limit 

Total 

Total sum of outliers 
identified and replaced (all 
methods combined) 
Lower 
limit 

Upper 
limit 

Total 

18 
8 
8 

2 
0 
5 
0 
8 
5 

20 
0 
0 
0 
0 
0 
0 
2 

47 
1 
0 
7 
3 

28 
17 
7 
17 
28 
0 
8 
1 
13 
4 
1 
16 
0 
8 
0 
0 

0 
0 
0 

7 
1 
2 
4 
0 
1 

0 
0 
0 
0 
0 
0 
0 
0 

0 
0 
0 
0 
0 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 

18 
8 
8 

9 
1 
7 
4 
8 
6 

20 
0 
0 
0 
0 
0 
0 
2 

47 
1 
0 
7 
3 

28 
17 
7 
17 
28 
0 
8 
1 
13 
4 
1 
16 
0 
8 
0 
0 

24 
10 
10 

2 
0 
5 
0 
7 
0 

17 
0 
2 
3 
0 
0 
0 
0 

32 
1 
0 
6 
1 

30 
16 
6 
14 
17 
0 
8 
2 
9 
6 
1 
15 
0 
8 
0 
0 

0 
0 
0 

0 
0 
0 
0 
0 
0 

0 
0 
0 
0 
0 
0 
0 
0 

0 
0 
0 
0 
0 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 

33 
11 
10 

7 
0 
5 
0 
9 
9 

26 
0 
2 
5 
2 
2 
1 
8 

55 
8 
8 
18 
9 

31 
33 
13 
23 
35 
3 
8 
11 
22 
12 
8 
29 
0 
13 
5 
9 

24 
10 
10 

2 
0 
5 
0 
7 
0 

17 
0 
2 
3 
0 
0 
0 
0 

32 
1 
0 
6 
1 

30 
16 
6 
14 
17 
0 
8 
2 
9 
6 
1 
15 
0 
8 
0 
0 

85 

0 
0 
0 

0 
0 
0 
0 
0 
0 

0 
0 
0 
0 
0 
0 
0 
0 

0 
0 
0 
0 
0 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 

33 
11 
10 

7 
0 
5 
0 
9 
9 

26 
0 
2 
5 
2 
2 
1 
8 

55 
8 
8 
18 
9 

31 
33 
13 
23 
35 
3 
8 
11 
22 
12 
8 
29 
0 
13 
5 
9 

4 
2 
1 

0 
0 
4 
0 
0 
0 

7 
0 
0 
0 
0 
0 
0 
0 

12 
0 
0 
0 
0 

10 
0 
0 
5 
0 
0 
8 
0 
4 
0 
0 
8 
0 
2 
0 
0 

0 
0 
0 

0 
0 
0 
0 
0 
0 

0 
0 
0 
0 
0 
0 
0 
0 

0 
0 
0 
0 
0 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 

4 
2 
1 

0 
0 
4 
0 
0 
0 

7 
0 
0 
0 
0 
0 
0 
0 

12 
0 
0 
0 
0 

10 
0 
0 
5 
0 
0 
8 
0 
4 
0 
0 
8 
0 
2 
0 
0 

79 
31 
29 

11 
0 
19 
0 
24 
14 

70 
0 
4 
8 
2 
2 
1 
10 

146 
10 
8 
31 
13 

99 
66 
26 
59 
80 
3 
32 
14 
48 
22 
10 
68 
0 
31 
5 
9 

0 
0 
0 

7 
1 
2 
4 
0 
1 

0 
0 
0 
0 
0 
0 
0 
0 

0 
0 
0 
0 
0 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 

79 
31 
29 

18 
1 
21 
4 
24 
15 

70 
0 
4 
8 
2 
2 
1 
10 

146 
10 
8 
31 
13 

99 
66 
26 
59 
80 
3 
32 
14 
48 
22 
10 
68 
0 
31 
5 
9 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Country/Region 
Gießen 
Kassel 
Mecklenburg-Vorpommern 
Braunschweig 
Hannover 
Lüneburg 
Weser-Ems 
Düsseldorf 
Köln 
Münster 
Detmold 
Arnsberg 
Koblenz 
Trier 
Rheinhessen-Pfalz 
Saarland 
Dresden 
Chemnitz 
Leipzig 
Sachsen-Anhalt 
Schleswig-Holstein 
Thüringen 
Estonia 
Ireland 
Northern and Western 
Southern 
Eastern and Midland 
Greece 
Attiki 
Voreio Aigaio 
Notio Aigaio 
Kriti 
Anatoliki Makedonia, Thraki 
Kentriki Makedonia 
Dytiki Makedonia 
Ipeiros 
Thessalia 
Ionia Nisia 
Dytiki Ellada 
Sterea Ellada 
Peloponnisos 
Spain 
Galicia 
Principado de Asturias 
Cantabria 
País Vasco 

RIS 2021 (Z-score +/-2) 
Lower 
limit 

Upper 
limit 

Total 

Upper 
limit 

Box-plot 
Lower 
limit 

Total 

Upper 
limit 

MAD 
Lower 
limit 

Total 

Upper 
limit 

Z-score (+/-3) 
Lower 
limit 

Total 

Total sum of outliers 
identified and replaced (all 
methods combined) 
Lower 
limit 

Upper 
limit 

Total 

DE72 
DE73 
DE8 
DE91 
DE92 
DE93 
DE94 
DEA1 
DEA2 
DEA3 
DEA4 
DEA5 
DEB1 
DEB2 
DEB3 
DEC 
DED2 
DED4 
DED5 
DEE 
DEF 
DEG 
EE 
IE 
IE04 
IE05 
IE06 
EL 
EL3 
EL41 
EL42 
EL43 
EL51 
EL52 
EL53 
EL54 
EL61 
EL62 
EL63 
EL64 
EL65 
ES 
ES11 
ES12 
ES13 
ES21 

0 
3 
0 
22 
1 
1 
0 
0 
8 
0 
10 
5 
1 
7 
10 
1 
9 
0 
7 
0 
0 
0 
6 

3 
3 
9 

1 
2 
0 
5 
1 
10 
4 
0 
4 
2 
1 
0 
0 

0 
3 
2 
6 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 

0 
0 
0 

0 
0 
0 
0 
0 
0 
4 
2 
0 
1 
0 
0 
0 

0 
0 
0 
0 

0 
3 
0 
22 
1 
1 
0 
0 
8 
0 
10 
5 
1 
7 
10 
1 
9 
0 
7 
0 
0 
0 
6 

3 
3 
9 

1 
2 
0 
5 
1 
10 
8 
2 
4 
3 
1 
0 
0 

0 
3 
2 
6 

0 
0 
0 
16 
0 
3 
0 
0 
8 
0 
8 
8 
0 
7 
8 
3 
9 
0 
7 
1 
0 
0 
4 

3 
4 
11 

2 
1 
3 
8 
3 
9 
2 
0 
3 
5 
1 
3 
0 

0 
3 
2 
4 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 

0 
0 
0 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 

0 
0 
0 
0 

0 
0 
0 
25 
0 
3 
0 
0 
9 
1 
8 
11 
1 
7 
13 
3 
9 
0 
9 
1 
0 
1 
8 

3 
4 
14 

2 
2 
3 
12 
9 
9 
7 
0 
6 
7 
1 
4 
0 

0 
3 
2 
4 

0 
0 
0 
16 
0 
3 
0 
0 
8 
0 
8 
8 
0 
7 
8 
3 
9 
0 
7 
1 
0 
0 
4 

3 
4 
11 

2 
1 
3 
8 
3 
9 
2 
0 
3 
5 
1 
3 
0 

0 
3 
2 
4 

86 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 

0 
0 
0 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 

0 
0 
0 
0 

0 
0 
0 
25 
0 
3 
0 
0 
9 
1 
8 
11 
1 
7 
13 
3 
9 
0 
9 
1 
0 
1 
8 

3 
4 
14 

2 
2 
3 
12 
9 
9 
7 
0 
6 
7 
1 
4 
0 

0 
3 
2 
4 

0 
0 
0 
16 
0 
0 
0 
0 
0 
0 
2 
0 
0 
7 
0 
0 
8 
0 
0 
0 
0 
0 
1 

3 
3 
3 

1 
0 
0 
4 
0 
1 
0 
0 
0 
0 
1 
0 
0 

0 
3 
0 
0 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 

0 
0 
0 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 

0 
0 
0 
0 

0 
0 
0 
16 
0 
0 
0 
0 
0 
0 
2 
0 
0 
7 
0 
0 
8 
0 
0 
0 
0 
0 
1 

3 
3 
3 

1 
0 
0 
4 
0 
1 
0 
0 
0 
0 
1 
0 
0 

0 
3 
0 
0 

0 
3 
0 
79 
1 
7 
0 
0 
25 
1 
28 
24 
2 
28 
31 
7 
35 
0 
23 
2 
0 
1 
19 

12 
14 
37 

6 
5 
6 
29 
13 
29 
13 
0 
13 
14 
4 
7 
0 

0 
12 
6 
14 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 

0 
0 
0 

0 
0 
0 
0 
0 
0 
4 
2 
0 
1 
0 
0 
0 

0 
0 
0 
0 

0 
3 
0 
79 
1 
7 
0 
0 
25 
1 
28 
24 
2 
28 
31 
7 
35 
0 
23 
2 
0 
1 
19 

12 
14 
37 

6 
5 
6 
29 
13 
29 
17 
2 
13 
15 
4 
7 
0 

0 
12 
6 
14 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Country/Region 
Comunidad Foral de Navarra 
La Rioja 
Aragón 
Comunidad de Madrid 
Castilla y León 
Castilla-la Mancha 
Extremadura 
Cataluña 
Comunidad Valenciana 
Illes Balears 
Andalucía 
Región de Murcia 
Ciudad Autónoma de Ceuta 
Ciudad Autónoma de Melilla 
Canarias 
France 
Île de France 
Centre - Val de Loire 
Bourgogne - Franche-Comté 
Normandie 
Hauts-de-France 
Grand Est 
Pays de la Loire 
Bretagne 
Nouvelle-Aquitaine 
Occitanie 
Auvergne - Rhône-Alpes 
Provence-Alpes-Côte d'Azur 
Corse 
Régions ultrapériphériques françaises 
Croatia 
Panonska Hrvatska 
Jadranska Hrvatska 
Grad Zagreb 
Sjeverna Hrvatska 
Italy 
Piemonte 
Valle d'Aosta/Vallée d'Aoste 
Liguria 
Lombardia 
Provincia Autonoma Bolzano/Bozen 
Provincia Autonoma Trento 
Veneto 
Friuli-Venezia Giulia 
Emilia-Romagna 
Toscana 

ES22 
ES23 
ES24 
ES3 
ES41 
ES42 
ES43 
ES51 
ES52 
ES53 
ES61 
ES62 
ES63 
ES64 
ES7 
FR 
FR1 
FRB 
FRC 
FRD 
FRE 
FRF 
FRG 
FRH 
FRI 
FRJ 
FRK 
FRL 
FRM 
FRY 
HR 
HR02 
HR03 
HR05 
HR06 
IT 
ITC1 
ITC2 
ITC3 
ITC4 
ITH1 
ITH2 
ITH3 
ITH4 
ITH5 
ITI1 

RIS 2021 (Z-score +/-2) 
Lower 
limit 

Upper 
limit 

Total 

Upper 
limit 

Box-plot 
Lower 
limit 

Total 

Upper 
limit 

MAD 
Lower 
limit 

Total 

Upper 
limit 

Z-score (+/-3) 
Lower 
limit 

Total 

Total sum of outliers 
identified and replaced (all 
methods combined) 
Lower 
limit 

Upper 
limit 

Total 

0 
3 
0 
8 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 

9 
0 
0 
0 
0 
0 
0 
0 
0 
7 
0 
0 
0 
9 
0 
4 
0 
4 
4 

0 
0 
0 
7 
1 
0 
10 
7 
0 
0 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
15 
8 
0 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
4 
0 
0 
0 
0 
0 
0 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 

0 
3 
0 
8 
0 
0 
0 
0 
0 
0 
0 
0 
15 
8 
0 

9 
0 
0 
0 
0 
0 
0 
0 
0 
7 
0 
0 
4 
9 
0 
4 
0 
4 
4 

0 
0 
0 
7 
1 
0 
10 
7 
0 
0 

0 
6 
0 
8 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 

6 
0 
0 
0 
0 
3 
0 
5 
0 
1 
0 
3 
0 
9 
0 
4 
3 
4 
8 

0 
0 
0 
0 
3 
0 
8 
9 
3 
0 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 

2 
9 
0 
8 
0 
0 
0 
0 
8 
0 
0 
0 
0 
0 
0 

12 
0 
0 
0 
0 
3 
0 
8 
0 
8 
7 
3 
0 
9 
0 
7 
3 
7 
11 

0 
0 
0 
7 
3 
5 
14 
9 
8 
0 

0 
6 
0 
8 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 

6 
0 
0 
0 
0 
3 
0 
5 
0 
1 
0 
3 
0 
9 
0 
4 
3 
4 
8 

0 
0 
0 
0 
3 
0 
8 
9 
3 
0 

87 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
1 
0 
0 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 

2 
9 
0 
8 
0 
0 
0 
0 
8 
0 
0 
0 
1 
0 
0 

12 
0 
0 
0 
0 
3 
0 
8 
0 
8 
7 
3 
0 
9 
0 
7 
3 
7 
11 

0 
0 
0 
7 
3 
5 
14 
9 
8 
0 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 

1 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
9 
0 
0 
0 
0 
0 

0 
0 
0 
0 
0 
0 
0 
6 
0 
0 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 

1 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
9 
0 
0 
0 
0 
0 

0 
0 
0 
0 
0 
0 
0 
6 
0 
0 

2 
18 
0 
24 
0 
0 
0 
0 
8 
0 
0 
0 
0 
0 
0 

28 
0 
0 
0 
0 
6 
0 
13 
0 
16 
7 
6 
0 
36 
0 
15 
6 
15 
23 

0 
0 
0 
14 
7 
5 
32 
31 
11 
0 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
16 
8 
0 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
4 
0 
0 
0 
0 
0 
0 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 

2 
18 
0 
24 
0 
0 
0 
0 
8 
0 
0 
0 
16 
8 
0 

28 
0 
0 
0 
0 
6 
0 
13 
0 
16 
7 
6 
4 
36 
0 
15 
6 
15 
23 

0 
0 
0 
14 
7 
5 
32 
31 
11 
0 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Country/Region 
Umbria 
Marche 
Lazio 
Abruzzo 
Molise 
Campania 
Puglia 
Basilicata 
Calabria 
Sicilia 
Sardegna 
Cyprus 
Latvia 
Lithuania 
Sostinės regionas 
Vidurio ir vakarų Lietuvos regionas  
Luxembourg 
Hungary 
Budapest 
Pest 
Közép-Dunántúl 
Nyugat-Dunántúl 
Dél-Dunántúl 
Észak-Magyarország 
Észak-Alföld 
Dél-Alföld 
Malta 
Netherlands 
Groningen 
Friesland 
Drenthe 
Overijssel 
Gelderland 
Flevoland 
Utrecht 
Noord-Holland 
Zuid-Holland 
Zeeland 
Noord-Brabant 
Limburg 
Austria 
Ostösterreich 
Südösterreich 
Westösterreich 
Poland 
Malopolskie 

ITI2 
ITI3 
ITI4 
ITF1 
ITF2 
ITF3 
ITF4 
ITF5 
ITF6 
ITG1 
ITG2 
CY 
LV 
LT 
LT01 
LT02 
LU 
HU 
HU11 
HU12 
HU21 
HU22 
HU23 
HU31 
HU32 
HU33 
MT 
NL 
NL11 
NL12 
NL13 
NL21 
NL22 
NL23 
NL31 
NL32 
NL33 
NL34 
NL41 
NL42 
AT 
AT1 
AT2 
AT3 
PL 
PL21 

RIS 2021 (Z-score +/-2) 
Lower 
limit 

Upper 
limit 

Total 

Upper 
limit 

Box-plot 
Lower 
limit 

Total 

Upper 
limit 

MAD 
Lower 
limit 

Total 

Upper 
limit 

Z-score (+/-3) 
Lower 
limit 

Total 

Total sum of outliers 
identified and replaced (all 
methods combined) 
Lower 
limit 

Upper 
limit 

Total 

5 
8 
0 
0 
0 
0 
0 
0 
0 
0 
0 
13 
0 
0 
8 
0 
20 
0 
8 
0 
0 
0 
0 
0 
0 
0 
12 

14 
2 
0 
1 
1 
4 
29 
2 
0 
0 
8 
0 

0 
7 
8 

8 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
1 
1 
0 
0 
1 
1 
0 
1 
0 
1 
0 
0 
0 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 

0 
0 
0 

0 

5 
8 
0 
0 
0 
0 
0 
0 
0 
0 
0 
13 
0 
0 
9 
1 
20 
0 
9 
1 
0 
1 
0 
1 
0 
0 
12 

14 
2 
0 
1 
1 
4 
29 
2 
0 
0 
8 
0 

0 
7 
8 

8 

5 
8 
1 
0 
1 
0 
0 
1 
0 
0 
0 
10 
0 
0 
9 
0 
13 
0 
11 
2 
0 
3 
0 
0 
0 
3 
13 

14 
0 
0 
0 
0 
3 
21 
1 
0 
0 
9 
0 

1 
7 
6 

0 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
1 
1 
0 
0 
1 
1 
0 
0 
0 
0 
0 
0 
0 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 

0 
0 
0 

0 

6 
8 
5 
0 
1 
0 
0 
1 
0 
0 
0 
10 
0 
0 
9 
0 
13 
0 
16 
2 
0 
3 
0 
0 
0 
3 
14 

16 
0 
0 
0 
1 
5 
24 
11 
7 
0 
9 
8 

3 
13 
8 

8 

5 
8 
1 
0 
1 
0 
0 
1 
0 
0 
0 
10 
0 
0 
10 
1 
13 
0 
12 
3 
0 
3 
0 
0 
0 
3 
13 

14 
0 
0 
0 
0 
3 
21 
1 
0 
0 
9 
0 

1 
7 
6 

0 

88 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
1 
1 
0 
0 
1 
1 
0 
0 
0 
0 
0 
0 
0 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 

0 
0 
0 

0 

6 
8 
5 
0 
1 
0 
0 
1 
0 
0 
0 
10 
0 
0 
10 
1 
13 
0 
17 
3 
0 
3 
0 
0 
0 
3 
14 

16 
0 
0 
0 
1 
5 
24 
11 
7 
0 
9 
8 

3 
13 
8 

8 

5 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
9 
0 
0 
3 
0 
10 
0 
3 
0 
0 
0 
0 
0 
0 
0 
11 

7 
0 
0 
0 
0 
2 
2 
0 
0 
0 
8 
0 

0 
0 
0 

0 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 

0 
0 
0 

0 

5 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
9 
0 
0 
3 
0 
10 
0 
3 
0 
0 
0 
0 
0 
0 
0 
11 

7 
0 
0 
0 
0 
2 
2 
0 
0 
0 
8 
0 

0 
0 
0 

0 

21 
24 
6 
0 
2 
0 
0 
2 
0 
0 
0 
42 
0 
0 
29 
0 
56 
0 
38 
4 
0 
6 
0 
0 
0 
6 
50 

51 
2 
0 
1 
2 
14 
76 
14 
7 
0 
34 
8 

4 
27 
22 

16 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
3 
3 
0 
0 
3 
3 
0 
1 
0 
1 
0 
0 
0 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 

0 
0 
0 

0 

21 
24 
6 
0 
2 
0 
0 
2 
0 
0 
0 
42 
0 
0 
32 
3 
56 
0 
41 
7 
0 
7 
0 
1 
0 
6 
50 

51 
2 
0 
1 
2 
14 
76 
14 
7 
0 
34 
8 

4 
27 
22 

16 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Country/Region 
Slaskie 
Wielkopolskie 
Zachodniopomorskie 
Lubuskie 
Dolnoslaskie 
Opolskie 
Kujawsko-Pomorskie 
Warminsko-Mazurskie 
Pomorskie 
Lódzkie 
Swietokrzyskie 
Lubelskie 
Podkarpackie 
Podlaskie 
Warszawski stoleczny 
Mazowiecki regionalny 
Portugal 
Norte 
Algarve 
Centro 
Lisboa 
Alentejo 
Região Autónoma dos Açores 
Região Autónoma da Madeira 
Romania 
Nord-Vest 
Centru 
Nord-Est 
Sud-Est 
Sud - Muntenia 
Bucuresti - Ilfov 
Sud-Vest Oltenia 
Vest 
Slovenia 
Vzhodna Slovenija 
Zahodna Slovenija 
Slovakia 
Bratislavský kraj 
Západné Slovensko 
Stredné Slovensko 
Východné Slovensko 
Finland 
Helsinki-Uusimaa 
Etelä-Suomi 
Länsi-Suomi 
Pohjois- ja Itä-Suomi 

RIS 2021 (Z-score +/-2) 
Lower 
limit 

Upper 
limit 

Total 

Upper 
limit 

Box-plot 
Lower 
limit 

Total 

Upper 
limit 

MAD 
Lower 
limit 

Total 

Upper 
limit 

Z-score (+/-3) 
Lower 
limit 

Total 

Total sum of outliers 
identified and replaced (all 
methods combined) 
Lower 
limit 

Upper 
limit 

Total 

PL22 
PL41 
PL42 
PL43 
PL51 
PL52 
PL61 
PL62 
PL63 
PL71 
PL72 
PL81 
PL82 
PL84 
PL91 
PL92 
PT 
PT11 
PT15 
PT16 
PT17 
PT18 
PT2 
PT3 
RO 
RO11 
RO12 
RO21 
RO22 
RO31 
RO32 
RO41 
RO42 
SI 
SI03 
SI04 
SK 
SK01 
SK02 
SK03 
SK04 
FI 
FI1B 
FI1C 
FI19 
FI1D 

8 
0 
0 
0 
0 
0 
0 
0 
0 
6 
0 
0 
0 
0 
21 
0 

2 
2 
2 
2 
0 
3 
0 

0 
0 
0 
0 
0 
3 
0 
4 

0 
0 

13 
0 
0 
0 

39 
1 
6 
2 

0 
0 
0 
0 
0 
3 
0 
2 
0 
0 
2 
0 
0 
0 
1 
1 

0 
0 
0 
0 
0 
6 
5 

11 
11 
11 
13 
13 
6 
20 
20 

0 
0 

0 
0 
0 
0 

0 
0 
0 
0 

8 
0 
0 
0 
0 
3 
0 
2 
0 
6 
2 
0 
0 
0 
22 
1 

2 
2 
2 
2 
0 
9 
5 

11 
11 
11 
13 
13 
9 
20 
24 

0 
0 

13 
0 
0 
0 

39 
1 
6 
2 

8 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
16 
0 

3 
0 
0 
0 
0 
3 
0 

0 
0 
0 
0 
0 
3 
0 
1 

0 
0 

8 
0 
0 
2 

36 
0 
5 
0 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
1 
1 

0 
0 
0 
0 
0 
6 
5 

0 
0 
0 
0 
2 
0 
2 
5 

0 
0 

0 
0 
0 
0 

0 
0 
0 
0 

8 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
17 
1 

3 
0 
0 
0 
0 
9 
5 

0 
0 
0 
0 
2 
3 
2 
6 

0 
0 

8 
0 
0 
2 

36 
0 
5 
0 

89 

8 
2 
0 
0 
0 
0 
0 
2 
0 
7 
0 
0 
0 
0 
22 
0 

5 
2 
0 
2 
0 
3 
0 

0 
0 
0 
0 
0 
4 
0 
3 

3 
0 

11 
0 
0 
2 

54 
8 
15 
9 

0 
0 
0 
0 
0 
0 
0 
2 
0 
0 
0 
0 
0 
0 
1 
1 

0 
0 
0 
0 
0 
6 
5 

4 
8 
6 
9 
11 
4 
13 
13 

0 
0 

0 
0 
0 
0 

0 
0 
0 
0 

8 
2 
0 
0 
0 
0 
0 
4 
0 
7 
0 
0 
0 
0 
23 
1 

5 
2 
0 
2 
0 
9 
5 

4 
8 
6 
9 
11 
8 
13 
16 

3 
0 

11 
0 
0 
2 

54 
8 
15 
9 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
6 
0 

0 
0 
0 
0 
0 
0 
0 

0 
0 
0 
0 
0 
0 
0 
0 

0 
0 

2 
0 
0 
0 

11 
0 
0 
0 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 

0 
0 
0 
0 
0 
0 
0 

0 
0 
0 
0 
2 
0 
2 
5 

0 
0 

0 
0 
0 
0 

0 
0 
0 
0 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
6 
0 

0 
0 
0 
0 
0 
0 
0 

0 
0 
0 
0 
2 
0 
2 
5 

0 
0 

2 
0 
0 
0 

24 
2 
0 
0 
0 
0 
0 
2 
0 
13 
0 
0 
0 
0 
65 
0 

10 
4 
2 
4 
0 
9 
0 

0 
0 
0 
0 
0 
10 
0 
8 

3 
0 

34 
0 
0 
4 

11 
0 
0 
0 

140 
9 
26 
11 

0 
0 
0 
0 
0 
3 
0 
4 
0 
0 
2 
0 
0 
0 
3 
3 

0 
0 
0 
0 
0 
18 
15 

15 
19 
17 
22 
28 
10 
37 
43 

0 
0 

0 
0 
0 
0 

0 
0 
0 
0 

24 
2 
0 
0 
0 
3 
0 
6 
0 
13 
2 
0 
0 
0 
68 
3 

10 
4 
2 
4 
0 
27 
15 

15 
19 
17 
22 
28 
20 
37 
51 

3 
0 

34 
0 
0 
4 

140 
9 
26 
11 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Country/Region 
Åland 
Sweden 
Stockholm 
Östra Mellansverige 
Småland med öarna 
Sydsverige 
Västsverige 
Norra Mellansverige 
Mellersta Norrland 
Övre Norrland 
Norway 
Oslo og Akershus 
Hedmark og Oppland 
Sør-Østlandet 
Agder og Rogaland 
Vestlandet 
Trøndelag 
Nord-Norge 
Switzerland 
Région lémanique 
Espace Mittelland 
Nordwestschweiz 
Zürich 
Ostschweiz 
Zentralschweiz 
Ticino 
Serbia 
Belgrade 
Vojvodina 
Šumadija and Western Serbia 
Southern and Eastern Serbia 
United Kingdom 
North East 
North West 
Yorkshire and The Humber 
East Midlands 
West Midlands 
East of England 
London 
South East 
South West  
Wales 
Scotland 
Northern Ireland 

RIS 2021 (Z-score +/-2) 
Lower 
limit 

Upper 
limit 

Total 

Upper 
limit 

Box-plot 
Lower 
limit 

Total 

Upper 
limit 

MAD 
Lower 
limit 

Total 

Upper 
limit 

Z-score (+/-3) 
Lower 
limit 

Total 

Total sum of outliers 
identified and replaced (all 
methods combined) 
Lower 
limit 

Upper 
limit 

Total 

FI2 
SE 
SE11 
SE12 
SE21 
SE22 
SE23 
SE31 
SE32 
SE33 
NO 
NO01 
NO02 
NO03 
NO04 
NO05 
NO06 
NO07 
CH 
CH01 
CH02 
CH03 
CH04 
CH05 
CH06 
CH07 
RS 
RS11 
RS12 
RS21 
RS22 
UK 
UKC 
UKD 
UKE 
UKF 
UKG 
UKH 
UKI 
UKJ 
UKK 
UKL 
UKM 
UKN 
Total 

6 

52 
29 
8 
18 
17 
8 
1 
27 

41 
5 
2 
2 
1 
35 
7 

22 
8 
30 
49 
21 
18 
16 

11 
4 
19 
17 

8 
12 
6 
11 
12 
4 
26 
19 
10 
12 
7 
4 
1379 

0 

0 
0 
0 
0 
0 
0 
1 
1 

0 
0 
0 
0 
0 
0 
1 

0 
0 
0 
0 
0 
0 
0 

1 
1 
1 
1 

6 

52 
29 
8 
18 
17 
8 
2 
28 

41 
5 
2 
2 
1 
35 
8 

22 
8 
30 
49 
21 
18 
16 

12 
5 
20 
18 

0 

45 
21 
1 
23 
15 
0 
1 
20 

29 
1 
0 
1 
1 
30 
5 

16 
4 
30 
30 
9 
16 
8 

6 
11 
17 
10 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
187 

8 
12 
6 
11 
12 
4 
26 
19 
10 
12 
7 
4 
1566 

4 
7 
11 
12 
9 
2 
18 
13 
9 
10 
7 
0 
1163 

13 

62 
39 
8 
26 
31 
8 
8 
35 

37 
1 
0 
4 
3 
43 
9 

35 
8 
35 
38 
18 
23 
15 

19 
13 
24 
17 

8 
9 
11 
14 
10 
11 
23 
20 
12 
12 
9 
0 

0 

0 
0 
0 
0 
0 
0 
0 
0 

0 
0 
0 
0 
0 
0 
0 

0 
0 
0 
0 
0 
0 
0 

1 
1 
1 
1 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
30 

0 

45 
21 
1 
23 
15 
0 
1 
20 

29 
1 
0 
1 
1 
30 
5 

16 
4 
30 
30 
9 
16 
8 

7 
12 
18 
11 

4 
7 
11 
12 
9 
2 
18 
13 
9 
10 
7 
0 
1193 

90 

0 

0 
0 
0 
0 
0 
0 
0 
0 

0 
0 
0 
0 
0 
0 
0 

0 
0 
0 
0 
0 
0 
0 

1 
1 
1 
1 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 

13 

62 
39 
8 
26 
31 
8 
8 
35 

37 
1 
0 
4 
3 
43 
9 

35 
8 
35 
38 
18 
23 
15 

20 
14 
25 
18 

8 
9 
11 
14 
10 
11 
23 
20 
12 
12 
9 
0 

0 

10 
0 
0 
9 
3 
0 
0 
8 

3 
0 
0 
0 
1 
22 
0 

12 
0 
9 
22 
8 
9 
3 

3 
0 
11 
9 

4 
0 
2 
4 
4 
2 
8 
6 
6 
5 
2 
0 

0 

0 
0 
0 
0 
0 
0 
0 
0 

0 
0 
0 
0 
0 
0 
0 

0 
0 
0 
0 
0 
0 
0 

0 
0 
0 
0 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 

0 

10 
0 
0 
9 
3 
0 
0 
8 

3 
0 
0 
0 
1 
22 
0 

12 
0 
9 
22 
8 
9 
3 

3 
0 
11 
9 

4 
0 
2 
4 
4 
2 
8 
6 
6 
5 
2 
0 

19 

169 
89 
17 
76 
66 
16 
10 
90 

110 
7 
2 
7 
6 
130 
21 

85 
20 
104 
139 
56 
66 
42 

39 
28 
71 
53 

24 
28 
30 
41 
35 
19 
75 
58 
37 
39 
25 
4 

0 

0 
0 
0 
0 
0 
0 
1 
1 

0 
0 
0 
0 
0 
0 
1 

0 
0 
0 
0 
0 
0 
0 

3 
3 
3 
3 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 

19 

169 
89 
17 
76 
66 
16 
11 
91 

110 
7 
2 
7 
6 
130 
22 

85 
20 
104 
139 
56 
66 
42 

42 
31 
74 
56 

24 
28 
30 
41 
35 
19 
75 
58 
37 
39 
25 
4 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
  ANNEX F. Box-plot overviews RIS 

Figure 9 provides a visual overview of the country dataset distribution for each of the indicators that 
are  part  of  the  Framework  conditions  dimension.  The  Framework  condition  dimension  includes  5 
indicators, of which 4 include outliers.  

Figure 9. Outlier treatment by method and indicator for the RIS 2021: Framework conditions 

Note: No treatment is original raw data distribution. Original approach is the Z-score method with cut-off values 
+/-2. Z-score refers to Z-score method with cut-off values +/-3. 

91 

 
 
 
 
 
Figure 10 provides a visual overview of the country dataset distribution for each of the indicators that 
are  part  of  the  Investments  dimension.  The  Investments  dimension  includes  5  indicators,  all  of 
which include outliers.  

Figure 10. Outlier treatment by method and indicator for the RIS 2021: Investment 

Note: No treatment is original raw data distribution. Original approach is the Z-score method with cut-off values 
+/-2. Z-score refers to Z-score method with cut-off values +/-3. 

92 

 
 
 
 
 
 
Figure 11 provides a visual overview of the country dataset distribution for each of the indicators that 
are part of the Innovation Activities dimension. The Investments dimension includes 7 indicators, of 
which 7 include outliers.  

Figure 11. Outlier treatment by method and indicator for the RIS 2021: Innovation activities 

Note: No treatment is original raw data distribution. Original approach is the Z-score method with cut-off values 
+/-2. Z-score refers to Z-score method with cut-off values +/-3. 

93 

 
 
 
 
 
 
Figure 12 provides a visual overview of the country dataset distribution for each of the indicators that 
are part of the Impacts dimension. The Impact dimension includes 4 indicators, of which 4 include 
outliers.  

Figure 12. Outlier treatment by method and indicator for the RIS 2021: Impacts 

Note: No treatment is original raw data distribution. Original approach is the Z-score method with cut-off values 
+/-2. Z-score refers to Z-score method with cut-off values +/-3. 

94 

 
 
 
 
 
 
 
 
GETTING IN TOUCH WITH THE EU 

In person 
All over the European Union there are hundreds of Europe Direct centres. You can find the address of 
the centre nearest you online (european-union.europa.eu/contact-eu/meet-us_en). 

On the phone or in writing 
Europe Direct is a service that answers your questions about the European Union. 
You can contact this service: 
  by freephone: 00 800 6 7 8 9 10 11 (certain operators may charge for these calls), 
  at the following standard number: +32 22999696,  
  via the following form: european-union.europa.eu/contact-eu/write-us_en. 

FINDING INFORMATION ABOUT THE EU 

Online 
Information about the European Union in all the official languages of the EU is available on the Europa 
website (european-union.europa.eu). 

EU publications 
You can view or order EU publications at op.europa.eu/en/publications. Multiple copies of free 
publications can be obtained by contacting Europe Direct or your local documentation centre 
(european-union.europa.eu/contact-eu/meet-us_en). 

EU law and related documents 
For access to legal information from the EU, including all EU law since 1951 in all the official language 
versions, go to EUR-Lex (eur-lex.europa.eu). 

EU open data 
The portal data.europa.eu provides access to open datasets from the EU institutions, bodies and 
agencies. These can be downloaded and reused for free, for both commercial and non-commercial 
purposes. The portal also provides access to a wealth of datasets from European countries. 

 
 
 
 
 
 
 
− 

This report presents the results for different methods used for 
identifying and correcting statistical outliers using data from the 
2021 editions of both the European Innovation Scoreboard (EIS) 
and Regional Innovation Scoreboard (RIS). The findings indicate 
that the choice of outlier treatment method can lead to differences 
in ranks and performance group placement compared to the 
current approach used in the EIS and RIS. 

Studies and reports