JRC statistical audit of the Commitment to reducing inequality index 2024.
Date: 2024-10-21
Institutions: Joint Research Centre
Subjects: audit, fiscal policy, labour law, poverty, public service, research report, social inequality, statistical method

ISSN 1831-9424 

JRC Statistical Audit of the Commitment 
to Reducing Inequality index 2024 

Laguera Gonzalez, J., Ravanos, P., Smallenbroek, O. 

2024 

 EUR 40082 

 
 
  
 
service. It aims to provide evidence-based scientific support to the European policymaking process. The  contents of this 
publication do not necessarily reflect the position or opinion of the European Commission. Neither the European Commission 
nor any person acting on behalf of the Commission is responsible for the use that might be made of this publication. For 
information on the methodology and quality underlying the data used in this publication for which the source is neither 
Eurostat nor other Commission services, users should contact the referenced source. The designations employed and the 
presentation of material on the maps do not imply the expression of any opinion whatsoever on the part of the European 
Union concerning the legal status of any country, territory, city or area or of its authorities, or concerning the delimitation 
of its frontiers or boundaries. 

Contact information  
Name: DG-JRC CC-COIN 

Email: JRC-COIN@ec.europa.eu 

EU Science Hub 

JRC138805 
https://joint-research-centre.ec.europa.eu 
EUR 40082 

PDF  

ISBN 978-92-68-21014-7   ISSN 1831-9424   doi:10.2760/4586505  

KJ-01-24-087-EN-N 

Luxembourg: Publications Office of the European Union, 2024  

© European Union, 2024  

The reuse policy of the European Commission documents is implemented by the Commission Decision 2011/833/EU of 12 December 2011 
on the reuse of Commission documents (OJ L 330, 14.12.2011, p. 39). Unless otherwise noted, the reuse of this document is authorised under 
the Creative Commons Attribution 4.0 International (CC BY 4.0) licence (https://creativecommons.org/licenses/by/4.0/). This means that reuse 
is allowed provided appropriate credit is given and any changes are indicated.  

For any use or reproduction of photos or other material that is not owned by the European Union permission must be sought directly from the 
copyright holders. 

How to cite this report: European Commission, Joint Research Centre, Laguera Gonzalez, J., Ravanos, P. and Smallenbroek, O., JRC Statistical 
Audit  of  the  Commitment  to  Reducing  Inequality  index  2024,  Publications  Office  of  the  European  Union,  Luxembourg,  2024, 
https://data.europa.eu/doi/10.2760/4586505, JRC138805. 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Contents 

Abstract ...................................................................................................................................................................................................... 2 

Acknowledgements .............................................................................................................................................................................. 3 

Authors ................................................................................................................................................................................................. 3 

1  Introduction ....................................................................................................................................................................................... 4 

2  Conceptual framework ................................................................................................................................................................ 6 

3  Data quality and availability .................................................................................................................................................... 8 

3.1  Data coverage ....................................................................................................................................................................... 8 

3.2  Treatment of outliers ........................................................................................................................................................ 9 

3.3  Normalisation ..................................................................................................................................................................... 10 

4  Statistical coherence ................................................................................................................................................................. 11 

4.1  Correlation analysis ......................................................................................................................................................... 11 

4.2  Principal components analysis of the CRI ........................................................................................................... 17 

4.3  Added value of the Commitment to Reducing Inequality Index ............................................................ 18 

4.4  The effect of the negative relation within the Tax pillar of the CRI .................................................... 18 

5  Impact of modelling assumptions on the CRI results ............................................................................................ 20 

6  Conclusions ..................................................................................................................................................................................... 25 

References ............................................................................................................................................................................................. 27 

List of abbreviations and definitions ...................................................................................................................................... 28 

List of figures ....................................................................................................................................................................................... 29 

List of tables ......................................................................................................................................................................................... 30 

1 

 
 
 
 
Abstract 

The 2024 Commitment to Reducing Inequality (CRI) Index is a multidimensional index which ranks 
164 countries for their policy performance across three pillars covering public services, progressive 
taxation and labour rights.  

the Joint Research Centre (JRC) in Ispra was invited by the index developers to audit the CRI for the 
fifth time, after the 2017, 2018, 2020, and 2022 CRI Index editions. JRC-COIN aims to help ensure 
the transparency of the index methodology and the reliability of its results. This JRC-COIN audit 
focuses on data quality issues, the statistical soundness of the multi-level structure of the index, 
and the impact of key modelling assumptions on the results. 

The analysis suggests that meaningful inferences can be drawn from the index, with a special 
attention on the scores and rankings of only a few countries. Overall, the CRI Index is an adequate 
tool for ranking the p
satisfactory statistical coherence from indicator to index, with the exception of the Tax pillar. 
Uncertainty analysis further shows a considerable number of countries have substantial uncertainty 
in the Tax pillar ranking. To this end, JRC-COIN has made some suggestions towards for possible 
refinements that could improve the match between the conceptual framework and the statistical 
characteristics of the index. Uncertainty analysis on the CRI Index ranking shows them to be 
representative of a plurality of scenarios and sufficiently robust to changes in the aggregation 
method and the pillar weights, except for a very few countries. 

mitment to reducing inequality. The CRI has a 

2 

 
 
 
Acknowledgements 

This document benefited from suggestions and feedback from colleagues in the Competence Centre 
on Composite Indicators and Scoreboards at the JRC. 

Authors 

Lagüera González, Jaime, 

Ravanos, Panagiotis, 

Smallenbroek, Oscar 

3 

 
1 

Introduction 

The Commitment to Reducing Inequality Index (CRI) aims at measuring the extent to which 
governments undertake actions towards reducing inequality. It is a multidimensional index 
developed by Development Finance International and Oxfam International, and it is composed of 
three major dimensions (or pillars), each of which corresponds to a policy area found to be critical in 
reducing inequality: 

1.  Public services 

2.  Taxation 

3.  Labour rights  

Each pillar is itself composed of three sub-
different stages of inequality-reducing policy lifecycle: 

1.  The policies that each government has enacted in the pillar area; 

2.  The implementation or coverage of these policies in practice, and   

3.  The impact of these policies on reducing income inequality (as measured by their impact on 

the respective Gini coefficient). 

Each sub-pillar is the result of further aggregation of additional indicators. 

-

commitments while considering its implementation efforts. It thus enables a more holistic approach 
to tracking country performance in reducing inequality, rather than a unidimensional one which 
would look only on policy impacts. Each pillar carries the same weight in the overall score of the 
index. Within each pillar, the policy, implementation and impact scores are also weighted with each 
other equally. Finally, t
the pillar score are averaged or weighted equally. However, because the number of indicators 
making up each policy or implementation score varies, the weight of each individual indicator in the 
overall index varies as well.   

The CRI framework is well-constructed and a lot of thought has clearly been put into it with the aim 

inequality. However, conceptual and practical challenges are inevitable when the complexity of a 
multidimensional phenomenon is summarised with a single composite indicator, and a dedicated 
analysis is needed to ensure and validate the statistical soundness of any composite index. The 

Research Centre (JRC-COIN) has been invited to provide a statistical analysis   an audit- for the 
latest edition of the CRI to validate its statistical soundness, with the aim to help policymakers 
derive more accurate and meaningful conclusions from the Index results and to potentially guide 
their choices on priority setting and policy formulation. This analysis follows the guidelines set out 
in the OECD-JRC (2008) Handbook for Constructing Composite Indicators (summarised in the JRC-
ndicators1) and its results are discussed in this report. 

1 https://knowledge4policy.ec.europa.eu/publication/your-10-step-pocket-guide-composite-indicators-scoreboards_en  

4 

 
                                                 
 
In general, statistical soundness should be regarded as a necessary but not sufficient condition for 
a sound index. This is because the correlations underpinning most of the statistical analyses carried 
out in this report need not necessarily represent the real influence of the individual indicators on the 
phenomenon being measured. The development of any index must therefore be nurtured by a 
dynamic, iterative dialogue between the principles of statistical and conceptual soundness.  

The assessment of the CRI Index presented in this report focuses on three main aspects: the quality 
of data, the statistical coherence of the structure, and the impact of key modelling assumptions on 
the CRI ranks. The statistical coherence analysis is based on: (i) the adequacy of aggregating 
indicators into aggregates (sub-pillars and pillars), and aggregates into the overall index; and (ii) the 
multidimensional structure of the CRI. Finally, by analysing the impact of modelling assumptions on 
CRI, JRC-COIN complements the CRI rankings with estimated intervals, in order to better appreciate 
the robustness of these ranks to the modelling choices. 

The analysis of the aspects highlighted above suggests that meaningful inferences can be drawn 
from the CRI, with a special attention to be paid on the scores and rankings of a limited number of 
countries. Overall, CRI 2024 is an adequate measure for ranking the performance of national 

commitment to reducing inequality. The CRI ranks are shown to be representative of a 

plurality of scenarios and sufficiently robust to changes in the aggregation method and the pillar 
weights, except for a very few cases where ranks should be interpreted with some caution. The 
analysis also finds that the CRI has satisfactory statistical properties for most of its structure, with 
the exception of the Tax Pillar, in which the underlying indicators and sub-pillars are found not to 
transfer adequately information to their aggregates and the CRI. To this end, JRC-COIN has made 
some suggestions towards for possible refinements that could improve the match between the 
conceptual framework and the statistical characteristics of the index. 

5 

 
 
2  Conceptual framework 

The CRI is based on three pillars, each of which relates to one policy area found to be critical in 
reducing inequality: public services; taxation; and labour rights. This edition of the CRI includes 18 
indicators grouped within 9 sub pillars and not changed with respect to the previous edition in 
2022.  (Table 1). Some of the indicators in the third column of Table 1 are themselves the result of 
some prior aggregation of different variables. For details on these indicators, the reader could refer 
to the official report of the CRI or to the 2020 JRC audit (Caperna et al., 2020) where the 
aggregation of these indicators was discussed in detail.   

Table 1: Conceptual framework of the 2024 CRI 

Pillar 

Sub - pillar 

Indicator 

Code 

Education spending % budget 

PS1 Policy 

Health spending % budget 

PUBLIC SERVICES 

Social protection spending % budget 

Education coverage (% completion by poorest quintile of 
upper secondary) 

PS2 Implementation 

Universal Health Coverage & Catastrophic household 
spending 

Social Protection coverage 

PS1A 

PS1B 

PS1C 

PS2A 

PS2B 

PS2C 

PS3 Impact 

Impact of Spending on the Gini coefficient 

PS3_i 

T1 Policy 

TAX 

VAT rate 

T1a 

Progressivity of Personal Income Tax (PIT) 

Corporate income tax Rate (CIT) 

T1b 

Harmful Tax Practices (HTP) 

T2 Implementation 

Tax productivity  

T3 Impact 

Impact of Tax on the Gini coefficient 

Labour Rights 

L1 Policy 

Women's Rights 

LABOUR RIGHTS 

Minimum Wage legislation 

L2 Coverage 

Unemployment & Vulnerable Employment 

L3 Impact 

Wages Gini 

Centre, 2024. 

T1aVAT 

T1aPIT 

T1aCIT 

T1bHTP 

T2_i 

T3_i 

L1a 

L1b 

L1c 

L2_i 

L3_i 

The CRI is aggregated at each level using a simple weighted arithmetic average. The use of the 
arithmetic average is a well-
perfect substitutability of the elements of the aggregation (a lack in one variable could be covered 
by a good performance in another). Overall, the CRI hierarchical framework is conceptually well 
justified, and its balanced structure with three pillars and three sub-pillars in each allows for 
meaningful conclusions to be made. 

6 

 
No changes in the underlying indicators have been performed compared to the previous edition of 
the CRI. There was however a change in the methodology used to compute one indicator, namely 
Harmful Tax Practices (HTPs) (T1b).  In the 2022 edition of the index, this indicator was constructed 
as the sum of three categorical variables, which were constructed by assigning points to countries 
based on the presence or not of (i) HTPs or aggressive tax schemes, (ii) the presence or not of anti-
avoidance measures, and (iii) the existence of disproportionate levels of foreign investment, trade 
and passive income. In this edition of the Index the point system for option (i) has changed and 
option (ii) has been removed. More details about these changes are described in the Index 
methodology report2. 

It is important to emphasize the qualitative nature of the T1b indicator and its implications when 
aggregating -within the T1 sub-pillar- with quantitative indicators that can take continuous 
numerical values. In particular, aggregation in this case will also treat the qualitative indicator 
(which the developers transformed into a categorical scale by assigning points to countries) as a 
quantitative and continuous indicator.  This treatment assumes that increases in the numerical 
scale correspond to equal increases in the magnitude of the qualitative scale, which may not always 
be the case. For example, the developers assign two points to high- and upper-middle-income 
countries with HTPs and one point to low- and lower-middle-income countries with HTPs, on the 
basis of evidence that wealthier countries with HTP regimes have a far stronger negative effect on 
profit-shifting from other countries, as well as reducing their own tax collection by much larger 
amounts. The aggregation of this indicator with others within the T1 sub-pillar assumes that the 

high- and upper-middle-income countries with HTPs 

compared to low- and lower-middle-income countries with HTPs.  

In addition, JRC-COIN wishes to draw special attention to the definition of T1 and T2 components.  
These are based on similar elements but used in formulas that may create a statistical conflict. In 
particular, using nominal tax rates as the denominator in T2 and as the numerator in T1 creates a 
negative relation between the two indicators which causes a conflict within the pillar. This issue is 
discussed in more detail in the section relative to statistical coherence.  

2 https://policy-practice.oxfam.org/resources/the-commitment-to-reducing-inequality-index-2024-621653/  

7 

 
 
 
 
 
 
                                                 
 
3  Data quality and availability 

3.1  Data coverage 

The JRC-COIN analysis is based on data for 167 countries (the developers decided to exclude six 
countries from the CRI 2024 due to insufficient data coverage (3). The data coverage of the 167 
countries is adequate with only seven indicators having missing values, none of which exceed four 
missing data points (Table 2). Country coverage of indicators is similar: for 93.4% of the countries 
there are no missing values, while all countries have data for at least 88.9% of the indicators. The 
low number of missing values is also related with 
available year(s) of data for each variable and collecting data from different sources to fill gaps for 
some variables. For example, resorting to national sources when data from international 
organisations are not present. Overall, the data coverage is well above the JRC-COIN recommended 
thresholds. 

Table 2: Summary statistics of the indicators included in the 2024 CRI 

Pillar 

Code 

n. missing  Mean 

Min 

Max  % Unique  Skew  Kurtosis 

PS1A 

PS1B 

PS1C 

Public Services 

PS2A 

PS2B 

PS2C 

PS3_i 

T1aPIT 

T1aCIT 

T1aVAT 

T1bHTP 

T2_i 

T3_i 

L1a 

L1b 

L1c 

L2_i 

L3_i 

Tax 

Labour Rights 

0 

0 

0 

1 

0 

1 

0 

0 

0 

0 

1 

1 

1 

3 

0 

0 

0 

4 

14.00 

4.58 

30.80 

11.60 

2.41 

26.40 

18.10 

0.10 

49.70 

0.47 

0.00 

1.00 

76.70 

48.70 

93.70 

58.50 

0.00 

100.00 

-0.08 

-0.41 

0.00 

8.04 

0.00 

25.00 

0.04 

0.00 

0.17 

23.40 

0.00 

40.00 

1.08 

0.00 

9.00 

0.39 

0.02 

1.04 

0.00 

-0.05 

0.12 

3.41 

0.00 

10.00 

0.76 

0.20 

1.00 

0.49 

0.00 

1.81 

0.98 

0.98 

1.00 

0.99 

1.00 

0.74 

0.99 

0.28 

0.89 

0.25 

0.04 

1.00 

1.00 

0.92 

0.82 

0.92 

0.68 

0.51 

0.48 

0.18 

0.42 

-1.18 

-0.02 

-1.53 

-0.55 

-0.63 

-0.27 

-1.64 

-1.32 

1.50 

0.41 

-0.58 

1.51 

2.33 

-0.80 

0.89 

1.93 

0.74 

1.21 

1.18 

2.85 

1.10 

4.03 

0.96 

-1.69 

2.96 

1.22 

2.00 

43.70 

4.06 

93.40 

1.00 

0.29 

-1.16 

0.50 

0.21 

0.86 

1.00 

0.51 

-0.54 

The CRI has an adequate timeliness as most indicators are based on recent data. In particular, data 
for all indicators of the Tax and Labour rights sub-indexes correspond to 2020 or later. The same 

3 These are Colombia, Cuba, Montenegro, Nicaragua, Somalia, and United Arab Emirates. 

8 

 
 
                                                 
 
holds true for indicators of the Public services pillar, except PS2A and PS2B. Data from the last five 
years make up 60% of the PS2A indicator and 45% of a sub-indicator of PS2B (the OOP sub-
indicator). While 84% of all the other indicators and sub-indicators use data from the last five 
years. The presence of less timely information is not an issue by definition, and it could be 
unavoidable when dealing with such a large number of countries. For the PS2A and the OOP 
indicators, it is related to absence of recent updates at the source level. The 
keeping these less timely data (rather than excluding it entirely) relates to the conceptual 
importance of these indicators. 

 choice for 

Several indicators of the Public services and Tax pillars are collected from three or more sources. 
Multiple sources are unavoidable given the large number of countries covered. However, care must 
be taken to ensure the data-sources measure the same phenomenon and use similar 
methodologies.  This is meticulously done by the developers for the Tax Pillar, where methodologies 
are adopted to ensure a robust cross-country comparability (see the CRI 2022 methodology note).   

Indicator PS2A deserves some special attention with respect to the use of different data sources. 
This indicator is listed as having only one missing data point in Table 2, however this statistic does 
not take into account the imputation done by developers. The primary source used for PS2A is the 
UNESCO Institute for Statistics (UIS) which includes missing values for 31% or 51 of the 167 
countries. The percentage of missing values has increased since the last CRI edition as Eurostat has 
stopped publishing educational attainment data disaggregated by income decile. The developers 
filled missing data for 21 countries using national household surveys and imputed the 30 remaining 
missing observations using the standardised values of two other variables used as proxies (the 
general percentage of completion of the upper secondary and the gross graduation ratio for upper 
secondary). The developers are aware of the issues related to this indicator, which are mainly due to 
the lack of updates from its main source, but have maintained the same process for imputing 
missing data to ensure consistency with previous CRI reports. They are also actively seeking more 
national sources to impute data in the future CRI editions.  

Because the use of proxies is not very common in the management of missing data, the previous 
audit by JRC-COIN suggested finding a possible alternative (Caperna et al., 2022). The present audit 
would like to re-emphasise this and suggest using imputation through any statistical imputation 
method. The k-Nearest Neighbours method suggested in the 2022 audit is one alternative (but it is 
not the only one). In such imputation methods, the developers could still use the proxy data as 
criteria to find fitting matches with k-Nearest Neighbours or make more accurate imputations using 
a linear regression. 

For the remaining indicators the developers opt not to impute missing values as they are only a 
few. This approach is usually selected to improve transparency and avoid any methodological black 
box in the imputation of data. Because of this, the score of the aggregate containing the missing 
value is based on the other elements of the aggregate for which values are observed.  The 
developing team has however defined clear rules of exclusion in the case of insufficient country 
coverage. For instance, countries lacking sufficient or reliable data for at least one indicator for 
each pillar were excluded from the Index.   

3.2  Treatment of outliers 

JRC-COIN also investigated the presence of outliers that could potentially bias the effect of the 
indicators on the aggregates. The JRC recommends an approach for outlier identification based on 
skewness and kurtosis values (Groeneveled and Meeden, 1984), i.e. when the variables 

9 

 
simultaneously have absolute skewness higher than 2.0 and kurtosis higher than 3.5. As shown in 
Table 2, none of the main indicators used in the calculation of the CRI Index show outliers. 

3.3  Normalisation 

The indicators are rescaled to a 0 1 scale, using the min-max normalisation (see OECD-JRC, 2008), 
with 0 as the lowest score for progressivity achieved by countries, and 1 as the highest, which is 
I case, though, the 

results of the aggregations (sub-pillars, pillars, and the index) are again normalised to (0,1). This is 
the result of consultations and discussions between the developers and JRC-COIN in previous 
editions of the CRI. Essentially, the sub-pillars and pillars of the CRI require re-normalisation in each 
step to achieve similar ranges among elements of the same aggregation. However, the JRC-COIN 
notes that such similarity could also be achieved by improving the correlation between the elements 
that are aggregated. Further details on this topic are available in the following section. It is also 
noted that, while the re-normalisation at every level of aggregation is beneficial for the clarity and 
readability of the index in the absence of better correlation structures between its aggregates, JRC-
COIN suggests to always consider its implications for the index results. This is also addressed in 
Section five of the report, in which the impact of modelling assumptions are evaluated.  

10 

 
4  Statistical coherence 

The statistical coherence evaluation consists of a multi-level analysis of the correlations of 
variables at different levels of aggregation with each other, their aggregates and the CRI, and a 
comparison of Pillar correlations under alternative scenarios in which the negative relation between 
the T1 and T2 components is avoided. 

4.1  Correlation analysis 

The correlation analysis is used to assess the extent to which the observed indicator data fits into 
the defined conceptual framework. The degree of coherence between the conceptual framework 
and the statistical structure of the data is an important factor for the reliability of an index, in the 
sense that aggregates reflect underlying information in the indicators. Ideally, correlations among 
factors (variables or aggregates) within every level of the index should be positive and with values 
between 0.3 and 0.92. This condition effectively ensures that the overall index scores adequately 
reflect the underlying indicator values. Redundancy should be avoided in the framework, which 
could be identified by very high correlations (>0.92 in general). This is because if two indicators are 
highly collinear, this may amount to double counting (and therefore over-weighting) of the aspect 
that the two indicators measure.  

4.1.1.  Correlation analysis between indicators and aggregates 

Table 3 shows the correlations between indicators of the Public Services pillar, while the black lines 
group indicators forming each of the three sub-pillars. Overall, the correlation structure is meets 
criteria recommended by JRC-COIN, showing medium (0.48) to high (0.83) correlations between 
indicators. The exception is the PS1A indicator   education spending as a percent of total spending. 
This indicator is weakly or negatively correlated to all the other indicators within the PS pillar, while 
it has a strong negative correlation (-0.32) with one of the components within its own sub-pillar 
(PS1C).  

This is an opportunity to reflect on the interplay between statistical coherence and conceptual 
framework by delving into the meaning of indicator PS1A and its role in the aggregate. On the one 
hand, these low correlations are poor from a purely statistical point of view and result in the 
indicator PS1A to be poorly represented in the higher aggregations. Indeed, the indicator has a 
negative, albeit weak, correlation with the PS pillar and the CRI (see Table 6). On the other hand, the 
meaning of the indicators within the PS1 sub-pillar reveals that each indicator competes against 
others for public expenditures. In this sense, the presence of low correlations is not so surprising. In 
this case the conceptual meaning could be considered sufficiently relevant to allow the aggregation 
of these elements despite their poor internal correlation.  

Nevertheless, the JRC-COIN emphasises that the weak correlation of indicator PS1A with the other 
indicators should always be taken into account when interpreting the role of this indicator in the 
overall index. Its conflict with the other indicators means that it cannot be well represented in the 
final aggregation, in the sense that it does not transfer enough information about country 
performance in education spending as a percent of total spending to the CRI. Therefore, the JRC-
COIN team recommends the developers to experiment with methodological changes that are 
conceptually coherent but may result in a better correlation structure. For example, the indicators 
could be scored taking into account the size of the economies. When a budget is small, a larger 
proportion may have to be allocated to essential services like education. Alternatively, the indicators 

11 

 
could be replaced by proxies for spending such as employment in the education sector or in 
teaching occupations.  

Table 3: Correlations between Indicators of the Public Services Pillar 

PS1A 

0.10 

-0.32 

-0.16 

-0.21 

-0.09 

-0.21 

PS1B 

0.10 

0.48 

0.57 

0.52 

0.65 

0.55 

PS1C 

-0.32 

0.48 

0.68 

0.68 

0.69 

0.71 

PS2A 

-0.16 

0.57 

0.68 

0.68 

0.83 

0.56 

PS2B 

-0.21 

0.52 

0.68 

0.68 

0.70 

0.60 

PS2C 

-0.09 

0.65 

0.69 

0.83 

0.70 

0.56 

PS3_i 

-0.21 

0.55 

0.71 

0.56 

0.60 

0.56 

PS1A 

PS1B 

PS1C 

PS2A 

PS2B 

PS2C 

PS3_i 

Note: Numbers represent the Pearson correlation coefficients. Good correlations (i.e. Pearson correlation coeffi-
cients greater than 0.30 and lower than 0.92) are highlighted in light green. Correlations with low values (here 
between -0.30 and 0.30) are written in grey. Correlations at risk of redundancy (here >0.92) are written in darker 
green. Correlations with meaningful negative value (here < -0.30), when present, are highlighted in red. 

024. 

Table 4 shows the correlations between indicators of the Tax pillar. The overall correlation structure 
between indicators of the tax pillar shows mostly weak or negative values. These results suggest 
 a unidimensional concept. The elements of the T1 

component are weakly related to each other, but their relationship is often close to the minimal 
level of 0.3 that JRC-COIN suggests as a threshold to check whether to elements share a relevant 
amount of their variability. Only two of the internal correlations are above  but close to- this 
threshold (those between T1aVAT and T1aPIT, and between T1aCIT and T3_i).   

These undesirable results are a consequence of the components that make up T1 and T2, and in 
particular with the way the components of T1 and T2 are defined. The definitions of these 
indicators, as well as the reasons for choosing them are transparently discussed in the CRI report. 
However, despite the high conceptual relevance of these indicators for measuring the underlying 
concept of Tax progressivity, the JRC-COIN would like to emphasize again on the strong adverse 
effect of these low and negative correlations on the ability of the indicators in T1 and T2 to be 
represented in the higher-level aggregates, as stated in the previous paragraph. These weak and 
negative correlations between T1 and T2 effectively cause the scores of the two components to 

thus it is highly likely that little information about the variability of 

these indicators across countries (and thus their value added to the overall composite) is reflected 
in the final CRI ranking. The correlations between sub-pillars are further investigated in section 
4.1.2. 

Table 4: Correlations between Indicators of the Public Tax Pillar 

T1aCIT 

T1aPIT 

T1aVAT 

T1bHTP 

T2_i 

T1aCIT 

T1aPIT 

T1aVAT 

T1bHTP 

T2_i 

T3_i 

0.26 

0.31 

0.20 

0.26 

0.23 

0.23 

-0.39 

-0.35 

0.37 

0.04 

0.23 

0.20 

-0.01 

-0.29 

0.09 

-0.39 

-0.35 

-0.35 

-0.29 

-0.17 

0.23 

0.31 

-0.01 

-0.35 

0.14 

12 

T3_i 

0.37 

0.04 

0.14 

0.09 

-0.17 

 
  
 
 
 
 
 
 
 
  
 
 
 
 
 
 
Note: Numbers represent the Pearson correlation coefficients. Good correlations (i.e. Pearson correlation coeffi-
cients greater than 0.30 and lower than 0.92) are highlighted in light green. Correlations with low values (here 
between -0.30 and 0.30) are written in grey. Correlations at risk of redundancy (here >0.92) are written in darker 
green. Correlations with meaningful negative value (here < -0.30), when present, are highlighted in red. 

Table 5 shows the correlations between indicators of the Labour Rights pillar. L2_i and L3_i are well 
correlated. However, the correlations between components of the first sub-pillar, L1, are weak, 
despite not being significantly negative (the lowest value is -0.04). L1c is the least correlated of the 
sub-pillar, and also its relation with the other elements of the pillar is negative and very close to the 
-0.3 threshold suggested by JRC-

Table 5: Correlations between Indicators of the Labour Rights Pillar 

L1a 

0.22 

0.02 

0.16 

0.09 

L1b 

0.22 

-0.04 

0.29 

0.26 

L1c 

0.02 

-0.04 

L2_i 

0.16 

0.29 

L3_i 

0.09 

0.26 

-0.28 

-0.27 

-0.28 

0.80 

-0.27 

0.80 

L1a 

L1b 

L1c 

L2_i 

L3_i 

Note: Numbers represent the Pearson correlation coefficients. Good correlations (i.e. Pearson correlation coeffi-
cients greater than 0.30 and lower than 0.92) are highlighted in light green. Correlations with low values (here 
between -0.30 and 0.30) are written in grey. Correlations at risk of redundancy (here >0.92) are written in darker 
green. Correlations with meaningful negative value (here < -0.30), when present, are highlighted in red. 

4.1.2.  Correlation analysis between indicators and aggregates 

Table 6 shows the correlation coefficients between indicators and their corresponding sub - pillars, 
pillars and the CRI. Most correlations are positive and within the acceptable thresholds. A few cases 
worth examining in more detail are identified: 

 

 

 

The contribution of indicator on Education coverage (PS1A) in its corresponding sub-pillar is 
smaller: It 
is slightly negatively correlated with its pillar and the CRI Index, suggesting it does not bring 
value added to the higher aggregates. JRC-COIN echoes its previous suggestion to pay spe-
cial attention to this indicator and its meaning in the index.   

All the indicators of T1 correlate well with their respective sub-pillars. However, two out of 
six indicators of the Tax Pillar (T1aPIT and T1aVAT) correlate weakly within the Tax pillar, 
while their relation to the index is strongly negative. As discussed above, this means that 

with scores of other indicators and do 
not transfer information to the final aggregate. Additionally, three other indicators, T1aCIT, 
T1bHTP and T3_i, have a weak correlation to the overall index, indicating that they also are 
not meaningfully represented in the CRI. Some of the issues related to indicators T1aCIT, 
T1aVAT, and T1bHTP may come from their definition, as suggested in the previous section. 
But, a crucial role could be played by their poorly-quantitative nature. These three indicators 
present a very low proportion of unique values (only 25% of the values in T1aCIT, 28% of 
those in T1aVAT, and 4% of the values in T1bHTP are unique). This attribute has a definite 
adverse effect on their correlations as these statistics assume continuous data.  

Among the elements of the Labour Pillar, the indicator on Minimum Wage legislation (L1c) 
shows a weakly negative correlation with the pillar and index. This is to some extent related 
to its poor internal correlation with some of the other indicators (in particular L2_i and 

13 

 
 
  
  
 
 
 
 
 
 
L3_i). For this reason, JRC-COIN suggests that the indicator should be kept under scrutiny 
and possible alternatives for it could be identified and tested.  

Table 6: Pearson correlation coefficients between indicators, and their corresponding aggregates 

Indicator 

Sub-pillar 

PS1A 

PS1B 

PS1C 

PS2A 

PS2B 

PS2C 

PS3_i 

T1aCIT 

T1aPIT 

T1aVAT 

T1bHTP 

T2_i 

T3_i 

L1a 

L1b 

L1c 

L2_i 

L3_i 

0.26 

0.84 

0.75 

0.92 

0.90 

0.90 

1.00 

0.54 

0.58 

0.46 

0.80 

1.00 

1.00 

0.76 

0.59 

0.51 

1.00 

1.00 

Pillar 

-0.10 

0.73 

0.83 

0.85 

0.86 

0.84 

0.84 

0.37 

0.24 

0.22 

0.51 

0.21 

0.63 

0.39 

0.47 

CRI 

-0.14 

0.68 

0.81 

0.75 

0.81 

0.79 

0.80 

-0.11 

-0.31 

-0.49 

0.00 

0.64 

0.19 

0.26 

0.49 

-0.09 

-0.17 

0.91 

0.87 

0.83 

0.76 

Note: Numbers represent the Pearson correlation coefficients. Good correlations (i.e. Pearson correlation coeffi-
cients greater than 0.30 and lower than 0.92) are highlighted in light green. Correlations with low values (here 
between -0.30 and 0.30) are written in grey. Correlations at risk of redundancy (here >0.92) are written in darker 
green. Correlations with meaningful negative value (here < -0.30), when present, are highlighted in red. 

Overall, the correlation structure of indicators suggests that there are a few good candidates to be 
reworked or re-organized in case a future revision of the CRI framework presents an opportunity to 
improve the statistical coherence of the index.  

Particularly, one way to strengthen the impact of the Taxation information in the overall aggregate 
may be through revisiting the mathematical formulas for developing the three components of T1, 
and the T2 and T3 indicators. As discussed, the current formulae combine the same information in a 
way that ensures a negative correlation structure. In particular, the T2 and T1 indicators use the 
same information, namely the nominal tax rates, but use different ways of processing it. The 
nominal CIT is the T1aCIT indicator. The same nominal CIT is the denominator in the CIT component 
of T2_i. A correlation will seldom be positive when information is used in one variable as a 
denominator and the same information is used as numerator in the other variable. Similarly, T3_i 
uses the nominal rates in the denominator for each type of tax before summing, again creating a 
negative correlation between T2 and T3.  

Despite the arithmetic and statistical aspects related to the low or negative association among the 
elements of the Taxation pillar, the developers of the CRI and the literature, are very clear in 

14 

 
 
 
identifying the elements that should constitute this pillar. Significant changes in the formulas in the 
direction of a higher correlation would be considered incorrect from a conceptual perspective. For 
the current edition of the index, the JRC-COIN echoes its previous suggestion to take the structure 
of the Taxation pillar into serious account when interpreting the results of the CRI, in that some of 
rmation being reflected in 
the composite. However, with an eye towards future revisions of the CRI framework, the JRC-COIN 
would like to suggest a reconsideration of the components of this pillar, so that the tax rate does 
not appear as both the numerator and the denominator in different components of this pillar.  

4.1.3.  Correlation analysis between aggregates and index  

The values in Table 7 to Table 9 represent the correlations between the aggregates. This level is the 
most important as it represents the consistency of the general concepts. The PS pillar has a high 
internal consistency, with its sub-pillars being well correlated with each other (Table 7), with the 
pillar and the CRI (Table 8). A remark would be on the strong correlation (0.93) between sub-pillar 
PS2 and pillar PS, suggesting that PS2 is dominating the pillar in relation to the other two sub-
pillars. However, this is not a major issue since the other two sub-pillars also have a high correlation 
(0.85 and 0.84 respectively) with the pillar.  This makes suggests that all three sub-pillars transfer 
adequately enough information and in a balanced way to the PS pillar.  

The internal structure of the Tax pillar is less consistent, and it is very similar to previous editions 
(2020, 2022) of the index. The sub-pillars T1 and T2 are negatively correlated (-0.50) (Table 7). 
Negative correlations may reduce the ability of an aggregate to discriminate between countries, as 

the conflicting effect. Indeed, Table 8 shows T1 has rather high (-0.28) negative correlation with the 
overall index, indicating that the index describes something conflicting with the information passed 
on by sub-pillar T1. This is expected however, given the negative relation already discussed between 
the three indicators in T1 and the single indicator in T2 and T3. This issue is carried over to higher 
aggregates and affects their ability to effectively discriminate between the performances of 
countries. 

Table 7: Correlations between sub-pillars 

PS1 

0.71 

0.64 

PS2 

0.71 

0.64 

PS3 

0.64 

0.64 

T1 

-0.33 

-0.53 

T2 

0.47 

0.61 

T3 

0.10 

-0.05 

-0.25 

0.41 

0.17 

L1 

0.20 

0.16 

0.35 

L2 

0.69 

0.85 

0.59 

L3 

0.60 

0.79 

0.55 

-0.33 

-0.53 

-0.25 

-0.50 

0.21 

-0.09 

-0.52 

-0.51 

0.47 

0.61 

0.41 

-0.50 

-0.17 

0.12 

0.61 

0.53 

0.10 

-0.05 

0.17 

0.21 

-0.17 

-0.10 

-0.09 

-0.08 

0.20 

0.69 

0.60 

0.16 

0.85 

0.79 

0.35 

0.59 

0.55 

-0.09 

-0.52 

-0.51 

0.12 

0.61 

0.53 

-0.10 

-0.09 

-0.08 

0.10 

0.05 

0.10 

0.80 

0.05 

0.80 

PS1 

PS2 

PS3 

T1 

T2 

T3 

L1 

L2 

L3 

Note: Numbers represent the Pearson correlation coefficients. Good correlations (i.e. Pearson correlation coeffi-
cients greater than 0.30 and lower than 0.90) are highlighted in light green. Correlations with low values (here 
between -0.30 and 0.30) are written in grey. Correlations at risk of redundancy (here >0.90) are written in darker 
green. Correlations with meaningful negative value (here < -0.30), when present, are highlighted in red. 

15 

 
  
  
 
  
  
 
  
  
 
  
 
The correlations between the Labour rights sub-pillars reflect the inter-relations between its 
components. Although L3 and L2 are highly correlated (0.83), L1 is weakly correlated to L3 (0.10) 
and L2 (0.05). Nevertheless, all three sub-pillars are adequately correlated with the pillar and the 
CRI. It should however be noted that the L1 sub-pillar has a substantively smaller correlation with 
the Labour rights pillar than the other two pillars. This means that it transfers significantly less 
information to the overall index (L1 alone explains about 9.6% of the CRI variability compared to 
68.9% by L2 and 57.8% by L3, as is seen by the square of the correlation coefficient with the 
index). 

Given the correlation between its elements and its weak relation with the other pillars, the Tax pillar 
contributes much less information to the index (0.35) compared to the other pillars (PS - 0.93, L   
0.87), as seen in Table 9. This is mainly due to the combination of the negative relation of sub-pillar 
T2 with T1 and the non-significant correlation between T1 with T2 and T3, together with the 
relation of this elements with the other pillars. Notice that T1 is the only sub-pillar that is also 
negatively correlated with all the other sub-pillars with PS and L pillars, and that four of these six 
correlations are strong (below -0.30).  

Table 8: Correlations between sub-pillars with pillars and index 

Sub-pillar 

Pillar 

PS1 

PS2 

PS3 

T1 

T2 

T3 

L1 

L2 

L3 

0.85 

0.93 

0.84 

0.58 

0.21 

0.63 

0.42 

0.91 

0.87 

CRI 

0.79 

0.86 

0.80 

-0.28 

0.64 

0.19 

0.31 

0.83 

0.76 

Note:  Numbers  represent  the  Pearson  correlation  coefficients.  Good  correlations  (i.e.  Pearson  correlation 
coefficients greater than 0.30 and lower than 0.90) are highlighted in light green. Correlations with low values 
(here between -0.30 and 0.30) are written in grey. Correlations at risk of redundancy (here >0.90) are written 
in darker green. Correlations with meaningful negative value (here <  -0.30), when present, are highlighted in 
red. 

Lastly, Table 9 also shows that both the Public Spending and the Labour Rights pillars have a strong 
correlation with the overall index. These results show how the Tax pillar is less represented in the 
overall index score. In particular, the Tax pillar can alone explain only 12% (see the last column in 
Table 9) of the variability in the CRI, which is significantly less that the corresponding figure for the 
Public Services (87%) and the Labour Pillar (76%). 

Table 9: Correlations between pillars and with the index. 

PS 

T 

L 

CRI 

R2 

0.09 

0.85 

0.93 

87% 

-0.08 

0.35 

12% 

-0.08 

0.87 

76% 

0.09 

0.85 

PS 

T 

L 

16 

 
 
  
 
 
  
Note:  Numbers  in  columns  2-5  represent  the  Pearson  correlation  coefficients.  Good  correlations  (i.e.  Pearson 
correlation coefficients greater than 0.30 and lower than 0.90) are highlighted in light green. Correlations with 
low values (here between -0.30 and 0.30) are written in grey. Correlations at risk of redundancy (here >0.90) 
are  written  in  darker  green.  Correlations  with  meaningful  negative  value  (here  <  -0.30),  when  present,  are 
highlighted in red. Column 6 represents the R-squared from a linear regression of the index to each of the pillar 
values, which is equal to the square of the Pearson correlation coefficient. 

In sum the statistical coherence of the CRI demonstrated by the correlations between indicators, 
sub-pillars, pillars and aggregates, show the L and PS pillars are adequate and able to pass 
information from their indicators to the index. Nevertheless, the internal correlations of the PS pillar 
do show some weak or negative correlations. The tax pillar is shown to have an inadequate 
correlation structure which is unable to communicate the information from the underlying indicators 
in T1 to the index. The CC-COIN would recommend the developers to reassess the PS1A indicator 
and T1 indicators and to experiment with alternative data or methods that may improve the 
correlations.  

4.2  Principal components analysis of the CRI 

As a further step in the analysis of statistical coherence, principal components analysis (PCA) was 
used to confirm the presence of one single statistical dimension among the three CRI pillars and the 
CRI itself. Technically, the presence of one single statistical dimension is confirmed when there is 
only one principal component with an eigenvalue greater than 1, or only one principal component 
explaining more than 70% of the variance. In practice, the achievement of these thresholds 
suggests the presence of a common, unidimensional phenomenon underlying the pillars.  

In the case of the three CRI pillars, there is only one principal component (PC1) with an eigenvalue 
significantly higher than 1 (Table 10). The eigenvalue captures about 78% of the total variation of 
the Public services Pillar, the 61% of the total variation of the Labour rights Pillar, but only 54% of 
the variation in the Tax Pillar. This reflects the low internal consistency of the Tax Pillar discussed 
above. In the case of the CRI index, the PCA using the three sub-indexes suggests that there are two 
principal components with an eigenvalue greater than or equal to one (PC1 = 1.9, PC2 = 1). The first 
principal component explains about 62% of the total variation.  

The results from PCA analysis suggest that 
not be a unidimensional concept.  This can be, to some extent, related to the internal structure of 
the Tax Pillar. The correlation analysis results do point towards the components in the T1 sub-pillar 

in the CRI. This echoes JRC-
within that Pillar in a future update of the CRI framework.  

Table 10: Principal component analysis of the CRI and its sub-indexes 

index 

PS 

T 

L 

CRI 

No. of  
aggregates 
3 

No. of suggested un-
derlying dimensions 
1 

% of variance  
explained by 1st PC 
77.5% 

3 

3 

3 

1 

1 

2 

17 

53.7% 

60.6% 

61.7% 

 
 
  
  
  
  
 
4.3  Added value of the Commitment to Reducing Inequality Index 

High statistical reliability among the main components of an index may in some cases be the result 
of redundancy of information (due to an overlap between the concepts measured by the different 
components). This section explores to what extent countries are ranked differently in each pillar, 
showing that each pillar provides different information. This is clearly the case with the CRI index. 
For at least 33% of the 167 countries studied by JRC-COIN (164 of which are included in the index), 
the CRI ranking and any of the three pillar rankings differ by more than 15 positions (see Table 11). 
This suggests that the CRI 
do not emerge by looking into the three pillars separately. At the same time, this result points to the 
value of examining individual pillars and indicators on their own merit.  

Despite that variability between Index and sub-index rankings is a desirable feature, excessive 
variability should not be the case. For the case of the Tax sub-index, more than 60% of countries 
have a rank that differs (i.e., is either higher or lower) from their CRI rank by more than 30 
positions, meaning that the Tax Pillar ranks countries in a substantively different way from the 
other two pillars. In other words, high- scoring countries on the Tax pillar can score both very high 
and very low in terms of the overall Index. Alternatively, countries that score the best and those that 
score the worst on the Tax pillar can be found with similar ranks on the CRI. This demonstrates the 
reduced ability of the index to keep the information carried by the Tax pillar. 

Table 11: Distribution of differences in country ranks based on the CRI and its pillars. 

Shift in rank with respect to CRI 

Public Services 

Tax 

Labour rights 

More than 30 positions 

15-30 positions 

More than 15 positions* 

5-15 positions 

Up to 5 positions 

0 positions 

10.8% 

22.2% 
33% 
32.3% 

31.1% 

3.6% 

63.5% 

21.6% 
85.1% 
8.4% 

6.0% 

0.6% 

21.0% 

30.5% 
51.5% 
25.7% 

18.0% 

4.8% 

Note: (*) This row is the sum of the two rows above.  

Source: 

4.4  The effect of the negative relation within the Tax pillar of the CRI 

In an attempt to highlight the benefits of amending the issues internal to the CRI Tax pillar, The 
JRC-COIN carried out an assessment of what the CRI correlation structure could look like if the 
negative relation between the T1 and T2 indicators was absent. Ideally, this should be conducted by 
selecting adequate alternative indicators to replace those in T1, T2, and/or T3 such that the same 
variable (Tax rate) does not appear both in their numerators and in their denominators. However, 
this was not possible and for this reason, the JRC-COIN decided to exclude  merely as a sensitivity 
analysis exercise and not as an actual proposition to the developers- one of the two highly 
negatively correlated sub-pillars (T1 and T2). Among those, T1 was selected for exclusion due to its 
negative correlations also with other sub-pillars of the Public services and the Labour rights sub-
indices (see Table 7). The JRC-COIN understands that exclusion of T1 comes in contrast with the 
conceptual framework of the CRI, but it considers this exercise beneficial for understanding the 
costs of this inherent negative relation for the statistical coherence of the CRI.   

18 

 
 
The correlations of the three pillars with each other and the re-calculated CRI when T1 is 
temporarily excluded are given in Table 12. Comparison with Table 9 shows that, when the negative 
relation between T1 and T2 is absent, the pillars are adequately well-correlated with each other and 
with the index and that all three Pillars explain an adequate amount of the variation of the CRI. In 
this counterfactual scenario the Tax sub-index explains 46% of the total variation of the CRI alone, 
which is a significant improvement from the 12% shown in Table 9. This exercise, which is not an 
actual suggestion to developers, is a crude way to highlight the benefits of avoiding having a 
negative relation build within an aggregate index to the adequate flow of information from lower to 
higher aggregation levels. 

Table 12: Correlations between pillars and with the index when excluding sub-pillar T1 

PS 

T 

0.54 

0.42 

0.54 

0.85 

L 

0.85 

0.42 

CRI 

0.95 

0.68 

0.92 

R2 

90% 

46% 

85% 

PS 

T 

L 

Note:  Numbers  in  columns  2-5  represent  the  Pearson  correlation  coefficients.  Good  correlations  (i.e.  Pearson 
correlation coefficients greater than 0.30 and lower than 0.90) are highlighted in light green. Correlations with 
low values (here between -0.30 and 0.30) are written in grey. Correlations at risk of redundancy (here >0.90) 
are  written  in  darker  green.  Correlations  with  meaningful  negative  value  (here  <  -0.30),  when  present,  are 
highlighted in red. Column 6 represents the R-squared from a linear regression of the index to each of the pillar 
values, which is equal to the square of the Pearson correlation coefficient. 

19 

 
  
 
 
  
 
 
5 

Impact of modelling assumptions on the CRI results 

A composite indicator is the result of a meticulous process involving the definition of a conceptual 
framework and data collection, but its development also involves a series of modelling choices for 
the normalisation and aggregation methods, the weights of indicators and their aggregates, the 
imputation of data, and others. Assessing the effect of these modelling choices on the scores and 
country rankings is a fundamental step in the statistical analysis of a composite indicator. Despite 
the efforts in the development process, there is an unavoidable subjectivity (or uncertainty) in the 
above choices. The potential effect of this subjectivity can be explored by comparing the results 
obtained under different (alternative) modelling choices.  

The literature on this topic suggests to assess the sensitivity of the index by means of a Monte 
Carlo simulation and by applying a multi-
errors have already been corrected in the preliminary stage of the index construction.   

The Commitment to Reducing Inequality Index, like most composite indicators, is the outcome of 
several choices concerning, among other things, the underlying theoretical framework, the indicators 
selected, the normalisation method, the weights assigned, and the aggregation method. Some of 
these choices may be based on expert opinion or other considerations, driven by statistical analysis 
or by the need to ease communication or to draw attention to specific issues. 

This section aims to test the impact of varying some of these assumptions within a range of 
plausible alternatives in an uncertainty analysis. The objective of this analysis is to quantify the 
uncertainty in the ranks of the CRI, which can demonstrate the extent to which countries can be 
differentiated by their scores. Ideally, small rank uncertainty for as many countries as possible 
(demonstrated by smaller intervals between the ranks of a country across the different alternative 
specifications of the index) is the desired outcome. 

The modelling issues considered in the uncertainty assessment of the CRI 2024 Index and its sub-
indexes are three: the aggregation formula, the normalisation procedure and the pillar weights. 
Given the excellent data coverage of the index, no alternative assumption has been considered for 
the imputation method. 

Aggregation formula. The CRI and each of its sub-indexes are aggregated by means of an 
arithmetic average, which implies perfect compensability across the components being aggregated. 
This effectively allows for an outstanding performance in some aspects to balance the weaknesses 
in others and vice-versa. This approach puts countries with both high and low results in different 
ults in all pillars. To 

assess the impact of this choice, the JRC included in the analysis a comparison with the geometric 
mean. The comparison of the two aggregation approaches should be able to highlight countries with 
unbalanced profiles since the geometric mean tends to penalize low values, especially in the 
presence of other values that are not so low (unbalanced profiles).  

Normalisation. The CRI Index normalizes the component scores at all levels of aggregation, 
meaning at the indicator, sub-pillar, and pillar levels, as done since the last two editions. This is not 
a common approach, but it is useful in the CRI, given the presence of aggregates with different 
ranges in the index. Nonetheless, multiple normalisation is likely to have an impact on country 
rankings. Therefore, normalisation at all levels is compared to the standard approach of normalizing 
only the indicators.  

Weights. Monte Carlo simulation comprised 1,000 runs of different set of weights for the three 
pillars constituting the CRI Index. The weights are the result of a random extraction based on 

20 

 
uniform continuous distributions centred in the reference value (1/3) plus or minus 20% of this 
value. 

Four models were tested, comparing the different combinations of aggregation formula and 
normalisation. A total of 5,000 simulations were run, each selecting at random one of the four 
models plus one of the 1,000 sets of weights. 

Table 13: Alternative assumptions considered in the uncertainty analysis of the CRI. 

I. Normalisation 

Min-max at every level 

Min-max of elementary indicators 

Reference 

Alternative 

I. Aggregation method  

Arithmetic average  

Geometric average  

II. Weighting system  

Equal weights (0.33) 

Varying: U [ 0.26; 0.40 ]  

Public Services 

Taxation 

Labour 

0,33 

0,33 

0,33 

U[0,26;0,40] 

U[0,26;0,40] 

U[0,26;0,40] 

The main results obtained from the uncertainty analysis are shown in Figure 1, with median ranks 
and 90% intervals computed across the 5,000 Monte Carlo simulations. Countries are ordered from 
best to worst according to their nominal CRI rank, where the blue dots represent the median rank 
among the 5,000 iterations. For each country, the error bars represent the 90% interval across all 
simulations, that is, from the 5th to the 95th percentile. Table 14 reports the CRI country ranks 
together with the simulated intervals (central 90 percentiles observed among the 5,000 scenarios) 
in order to better appreciate the robustness of these ranks to the computation methodology and to 
analyse the behaviour of specific countries in respect to perturbations 

CRI ranks are shown to be representative of a plurality of scenarios and robust to changes in the 
aggregation method and pillar weights. If one considers the median rank across the simulated 
scenarios as being representative of these scenarios, then the fact that the nominal CRI rank is 
close to the median rank (less than five positions away) for 113 (68%) of the 167 countries 
considered in the analysis, suggests that CRI is a rather robust to the methodological alternatives 
tested.  

However, the JRC-COIN analysis also finds that certain countries are more sensitive than others to 
changes in the aggregation, normalisation, and weighting. There are six countries for which the 
simulated interval is larger than 30 positions. These are: Lesotho, Bulgaria, North Macedonia, 
Montenegro, Bahamas, and Bahrain. Thus, it is suggested that the rankings for these six countries 
be interpreted with caution and not be taken at face value, since they are shown to depend 
substantively on the underlying assumptions for building the index   meaning they can move more 
than 20% of the total ranking. There is also a total of 32 countries that change 20 or more ranks 
across simulations. These are highlighted in italics in Table 14. Rank intervals are generally wider 
for mid-ranking countries, and narrower for top and bottom-ranking countries, which is an expected 
outcome in the context of composite indicators. The middle of the ranking is normally the most 
competitive in the sense that countries score very similarly to each other, and thus small changes in 
scores can make substantial changes in rank. In the case of the CRI, some exceptions to this 

21 

 
  
 
common result are visible for the countries with nominal ranks between the 120th and the 165th 
(Table 14). 

Figure 1: Uncertainty analysis on ranks (CRI rank vs median rank and 90% intervals). 

Note: Labelled countries show a 90% rank interval of at least 30. 

Table 14: CRI rank and 90% interval of all countries from uncertainty analysis. 

24. 

Country/Region 
Norway 
Ireland 
Canada 
Australia 
Germany 
Finland 
Austria 
Denmark 
Slovenia 
Japan 
France 
Poland 
Portugal 
Croatia 
New Zealand 
Israel 
Czechia 
United Kingdom 
Slovakia 
Belgium 
Italy 

CRI Rank 
1 
2 
3 
4 
5 
6 
7 
8 
9 
10 
11 
12 
13 
14 
15 
16 
17 
18 
19 
20 
21 

Interval 
[1, 1] 
[2, 2] 
[3, 5] 
[3, 10] 
[4, 6] 
[3, 6] 
[7, 10] 
[6, 9] 
[8, 14] 
[8, 16] 
[9, 13] 
[5, 21] 
[12, 18] 
[13, 19] 
[11, 22] 
[13, 20] 
[12, 19] 
[14, 20] 
[11, 21] 
[12, 20] 
[17, 22] 

Country/Region 
North Macedonia 
Occupied Palestinian Territory 
Eswatini 
Jamaica 
Antigua and Barbuda 
Hong Kong SAR, China 
Azerbaijan 
St. Vincent and the Grenadines 
Egypt, Arab Rep. 
Cabo Verde 
Montenegro 
Vietnam 
Bahamas, The 
Honduras 
Samoa 
Trinidad and Tobago 
Kenya 
Guatemala 
Dominican Republic 
Cambodia 
Belize 

CRI Rank 
85 
86 
87 
88 
89 
90 
91 
92 
93 
94 
95 
96 
97 
98 
99 
100 
101 
102 
103 
104 
105 

Interval 
[64, 105] 
[77, 95] 
[82, 96] 
[81, 93] 
[78, 91] 
[67, 93] 
[85, 100] 
[86, 95] 
[87, 101] 
[88, 99] 
[81, 113] 
[92, 106] 
[78, 113] 
[96, 120] 
[93, 105] 
[86, 106] 
[96, 118] 
[97, 113] 
[95, 107] 
[97, 125] 
[97, 108] 

22 

 
 
Country/Region 
Iceland 
Sweden 
Korea, Rep. 
Argentina 
United States 
Estonia 
Spain 
Luxembourg 
Chile 
Lithuania 
Belarus 
South Africa 
Latvia 
Serbia 
Costa Rica 
Netherlands 
Greece 
Kyrgyz Republic 
Colombia 
Switzerland 
Uruguay 
Hungary 
Jordan 
Russian Federation 
Ukraine 
Kiribati 
Mongolia 
Cyprus 
Lesotho 
Namibia 
Malta 
Seychelles 
Tunisia 
Tajikistan 
China 
Brazil 
Moldova 
Ecuador 
Mauritius 
Romania 
Mexico 
Paraguay 
Maldives 
Kazakhstan 
El Salvador 
Algeria 
United Arab Emirates 
Turkiye 
Bolivia 
Malaysia 
Barbados 
Georgia 
Bulgaria 
Guyana 
Nicaragua 
Thailand 
Morocco 
Uzbekistan 
Cuba 
Peru 
Tuvalu 
Albania 
Botswana 

CRI Rank 
22 
23 
24 
25 
26 
27 
28 
29 
30 
31 
32 
33 
34 
35 
36 
37 
38 
39 
40 
41 
42 
43 
44 
45 
46 
47 
48 
49 
50 
51 
52 
53 
54 
55 
56 
57 
58 
59 
60 
61 
62 
63 
64 
65 
66 
67 
68 
69 
70 
71 
72 
73 
74 
75 
76 
77 
78 
79 
80 
81 
82 
83 
84 

Interval 
[21, 24] 
[18, 26] 
[22, 32] 
[23, 31] 
[24, 28] 
[25, 30] 
[26, 34] 
[24, 35] 
[28, 37] 
[28, 34] 
[27, 34] 
[28, 44] 
[33, 36] 
[32, 38] 
[32, 42] 
[27, 40] 
[32, 42] 
[37, 47] 
[37, 50] 
[35, 50] 
[39, 46] 
[34, 57] 
[40, 70] 
[39, 48] 
[41, 50] 
[43, 70] 
[42, 52] 
[39, 65] 
[44, 82] 
[44, 66] 
[43, 69] 
[48, 55] 
[48, 64] 
[49, 63] 
[52, 72] 
[48, 59] 
[51, 60] 
[55, 79] 
[50, 66] 
[49, 75] 
[55, 69] 
[56, 63] 
[57, 82] 
[49, 68] 
[60, 77] 
[60, 83] 
[47, 70] 
[61, 69] 
[60, 74] 
[64, 71] 
[61, 76] 
[67, 77] 
[51, 90] 
[67, 79] 
[72, 90] 
[73, 87] 
[75, 98] 
[61, 84] 
[66, 84] 
[79, 96] 
[78, 90] 
[75, 85] 
[78, 99] 

Country/Region 
Singapore 
Philippines 
Timor-Leste 
Armenia 
Fiji 
Indonesia 
Myanmar 
St. Lucia 
Iran 
Senegal 
Togo 
Nepal 
Zambia 
Djibouti 
Yemen, Rep. 
Sri Lanka 
Oman 
Malawi 
Lebanon 
Sao Tome and Principe 
India 
Benin 
Bangladesh 
Tonga 
Panama 
Bhutan 
Burkina Faso 
Mali 
Lao PDR 
Tanzania 
Gambia, The 
Papua New Guinea 
Ghana 
Pakistan 
Angola 
Rwanda 
Mozambique 
Congo, Rep. 
Afghanistan 
Congo, Dem. Rep. 
Solomon Islands 
Mauritania 
Guinea-Bissau 
Bahrain 
Chad 
Cameroon 
Niger 
Guinea 
Somalia 
Ethiopia 
Madagascar 
Sierra Leone 
Burundi 
Cote d'Ivoire 
Central African Republic 
Liberia 
Uganda 
Zimbabwe 
Haiti 
Vanuatu 
Nigeria 
South Sudan 

23 

CRI Rank 
106 
107 
108 
109 
110 
111 
112 
113 
114 
115 
116 
117 
118 
119 
120 
121 
122 
123 
124 
125 
126 
127 
128 
129 
130 
131 
132 
133 
134 
135 
136 
137 
138 
139 
140 
141 
142 
143 
144 
145 
146 
147 
148 
149 
150 
151 
152 
153 
154 
155 
156 
157 
158 
159 
160 
161 
162 
163 
164 
165 
166 
167 

Interval 
[84, 113] 
[99, 109] 
[99, 113] 
[94, 112] 
[98, 111] 
[102, 116] 
[105, 130] 
[104, 118] 
[102, 120] 
[113, 121] 
[114, 129] 
[113, 125] 
[113, 129] 
[115, 123] 
[114, 140] 
[116, 122] 
[108, 125] 
[122, 134] 
[108, 129] 
[112, 128] 
[123, 138] 
[125, 133] 
[124, 129] 
[108, 134] 
[109, 135] 
[125, 142] 
[130, 143] 
[131, 140] 
[131, 145] 
[132, 150] 
[131, 138] 
[136, 145] 
[137, 141] 
[136, 154] 
[133, 147] 
[139, 146] 
[128, 145] 
[130, 145] 
[143, 151] 
[142, 152] 
[123, 147] 
[131, 147] 
[148, 157] 
[115, 154] 
[147, 164] 
[150, 154] 
[151, 163] 
[152, 161] 
[150, 163] 
[152, 160] 
[146, 160] 
[152, 159] 
[153, 159] 
[145, 160] 
[155, 165] 
[153, 162] 
[157, 164] 
[146, 164] 
[156, 164] 
[138, 166] 
[165, 166] 
[167, 167] 

 
  
  
  
 
For the 2024 edition of the CRI, JRC-COIN performed this analysis at the level of the CRI as well as 
the level of its three sub-indexes (Public services, Tax, and Labour rights). The Table 15 shows the 
percentage of countries for which the difference between the nominal rank and median rank across 
all 5,000 iterations of the sensitivity analysis exceeds certain thresholds. The CRI column of Table 
15 shows the same results as presented in Figure 1 but rather than using the confidence interval 
as a metric, we show the median difference from nominal rank. The median rank of the CRI is close 
to the nominal rank in 99% of cases, meaning that the difference is 15 ranks   just under 10% of 
the maximum number of ranks. The same result is observed for the Public Services pillar while 
100% of countries in the Labour rights pillar have a difference of less than 10% of the maximum 
number of ranks. However, the Tax pillar is not robust to the methodological alterations tested in 
the sensitivity analysis. Only 71% of countries have a difference between median and nominal rank 
of 15 or less. There is considerable uncertainty for 25% of countries, which vary between 10-20% 
of the total ranking, while for a minority of countries, 4% of them, the ranking is very uncertain, as 
it varies more than 20% of the total ranking. These results indicate that small changes in 
methodology have a large impact on the tax pillar making a substantial portion of the ranks 
uncertain, moving between 10 to 20% of the total ranking depending on small methodological 
changes. These uncertainties are likely not translated to the CRI as the Tax pillar has only a small 
impact on the CRI scores.   

Table 15: CRI Index and Pillar difference in median rank and nominal rank of all countries in sensitivity 
analysis. 

Shift in rank with respect to 
median 
More than 30 positions 
15-30 positions 
More than 15 positions 
5-15 positions 
Up to 5 positions 
0 positions 

CRI 

0.0% 
0.6% 
0.6% 
26.9% 
63.5% 
9.0% 

Public Ser-
vices 
0.0% 
0.6% 
0.6% 
10.2% 
65.3% 
24.0% 

Tax 

4.2% 
25.1% 
29.3% 
36.5% 
29.3% 
4.8% 

Labour 
rights 
0.0% 
0.0% 
0.0% 
8.4% 
70.7% 
21.0% 

24 

 
 
6  Conclusions 

The JRC statistical audit delves into the extensive work carried out by the developers of the 2024 
Commitment to Reducing Inequality Index with the aim of suggesting improvements in terms of 
data characteristics, structure, and methods used. The analysis aims to ensure the transparency of 
the index methodology and the reliability of the results.   

The data coverage of the framework is excellent, well above the JRC-COIN recommended 
thresholds. Seven indicators contain missing values but none have more than four missing values. 
Additionally no outliers are detected in any indicator. Given the complexity of the framework and 
the population covered, it is a common issue to have aged information or sources only partially the 
countries of interest. With respect to the use of proxies, JRC-COIN suggests considering statistical 
methods for the imputation of missing data as a possible alternative to the use of proxy data.  

According to the coherence analysis, the CRI is statistically correlated with two of three pillars. There 
are mostly positive correlations between indicators and their corresponding sub pillars, pillars and 
the index, thus suggesting that most of the indicators provide meaningful information on the 
variation of the scores. However, the Taxation pillar is less represented because of its internal 
structure and its correlation with the other two pillars.  

Two issues were identified. First, in the Social Spending pillar, the education spending  (PS1A) 
indicator does not correlate with the other two PS1 indicators, and it is the only one of three that 
does not contribute to the overall score. Second, considering the Tax pillar, the JRC team noticed 
that the statistical relations in it can be negligible or negative, as the indicators do not correlate 
internally and correlate weakly or negatively with the index. At the sub-pillar level, Tax 
Implementation (T2) shows a negative relationship with Tax Policy (T1) and a negligible relationship 
with T2. Moreover, Tax Policy (T1) has a negative correlation with the CRI index. As a consequence, 
the information from the Tax pillar contributes much less to the index compared to the other two 
pillars, as shown by the R2 value. The correlation structure and its negative impact are 
demonstrated in the PCA, added value and counterfactual analysis of sections 4.2 to 4.4. Given its 
conceptual importance in the framework, JRC would recommend to always keep searching for ways 
to improve the internal correlation structure and to communicate to users that the information it 
carries is not represented in the overall score. For these reason, JRC-COIN suggests to review the 
indicator PS1A and the T1 sub-pillar for further improvements. 

JRC analysed a series of different choices that were made during the index construction. The 
assumptions tested with the sensitivity analysis are related to the aggregation formula, 
normalisation procedure, and weights. This edition of the audit included normalisation in the 
uncertainty to test their effects in interaction with the other assumptions. The normalisation of the 
aggregates at each level was tested against the normalisation only at the base level. The results 
show that the CRI is ranks are robust to these changes in methodology, with only six out of 167 
countries having substantial variation in the rank with confidence intervals of 30 positions. In total 
there are 32 countries with confidence intervals of 20 or more positions, which is non-negligible but 
also expected with the high number of countries. For these 32 countries the ranks have to be 
interpreted with some caution.  

The uncertainty analysis was also conducted at the level of each pillar. The JRC-COIN team 
summarised these analyses by showing the difference between median and nominal raking on each 
pillar, taking the CRI as a benchmark. As in analysis showing the confidence interval, the CRI shows 
most countries are very stable in rank when comparing the median and nominal ranking, in fact 
99% fall within 15 positions, which is less than 10% of the total ranking. The same can be said of 

25 

 
the Public Services and Labour rights pillars. These are robust to the changes in methodology 
tested. However, the Tax pillar shows that almost 30% of countries have a non-negligible amount 
of uncertainty which requires the ranks to be interpreted with caution. Additionally, 4% of countries 
have a critical level of uncertainty in ranking on the Tax pillar which makes any interpretation 
unadvisable.  

The JRC suggested some possible directions for future refinement while recognising the CRI as a 
useful tool to provide insights for measuring the commitment to reduce inequalities. In general, the 
present audit confirms that the CRI Index is based on transparent methodology, data of sufficient 
quality and a reliable means of transmitting information of the Public Services and Labour rights 
pillars. The Tax pillar does not have an adequate statistical coherence   mainly due to the sub-pillar 
T1   and was found to be sensitive to small methodological changes. The JRC acknowledges the 
 and 

their continued commitment to improving the statistical quality of the CRI. 

26 

 
References 

Caperna, G., Smallenbroek, O., Kovacic, M. and Papadimitriou, E., JRC Statistical Audit of the 2022 
Commitment to Reducing Inequality index, Publications Office of the European Union, Luxembourg, 
2022, doi:10.2760/5642, JRC131019 

Caperna, G., Papadimitriou, E. and Kovacic, M., JRC Statistical Audit of the 2020 Commitment to 
Reducing Inequality index, EUR 30319 EN, Publications Office of the European Union, Luxembourg, 
2020, ISBN 978-92-76-20902-7, doi:10.2760/082450, JRC121452 

Groeneveld, R. A. and G. Meeden. Measuring Skewness and Kurtosis. The Statistician, Vol. 33, 1984, 
pp. 391 399.  

OECD/EC JRC (Organisation for Economic Co-operation and Development/European Commission, 
Joint Research Centre). Handbook on Constructing Composite Indicators: Methodology and User 
Guide. Paris: OECD. 2008 

Saisana, M., A. Saltelli, and S. Tarantola. Uncertainty and Sensitivity Analysis Techniques as Tools for 
the Quality Assessment of Composite Indicators. Journal of the Royal Statistical Society: Series A 
(Statistics in Society), Vol. 168 No. 2, 2005, pp. 307 23. doi: 10.1111/j.1467-985X.2005.00350.x.  

Policy Implications. Research Policy, Vol. 40, 2011, pp. 165 177. 

Rickety Numbers: Volatility of University Rankings and 

27 

 
List of abbreviations and definitions 

Abbreviations 

Definitions 

COIN 

CRI 

JRC 

Competence Centre on Composite Indicators and 
Scoreboards 

Commitment to Reducing Inequality Index 

Joint Research Centre 

28 

 
 
 
List of figures 

Figure 1: Uncertainty analysis on ranks (CRI rank vs median rank and 90% intervals)........................... 22 

29 

 
 
List of tables  

Table 1: Conceptual framework of the 2024 CRI .............................................................................................................. 6 

Table 2: Summary statistics of the indicators included in the 2024 CRI ............................................................. 8 

Table 3: Correlations between Indicators of the Public Services Pillar ............................................................... 12 

Table 4: Correlations between Indicators of the Public Tax Pillar ......................................................................... 12 

Table 5: Correlations between Indicators of the Labour Rights Pillar ................................................................. 13 

Table 6: Pearson correlation coefficients between indicators, and their corresponding aggregates 14 

Table 7: Correlations between sub-pillars ........................................................................................................................... 15 

Table 8: Correlations between sub-pillars with pillars and index .......................................................................... 16 

Table 9: Correlations between pillars and with the index. ......................................................................................... 16 

Table 10: Principal component analysis of the CRI and its sub-indexes ........................................................... 17 

Table 11: Distribution of differences in country ranks based on the CRI and its pillars. ......................... 18 

Table 12: Correlations between pillars and with the index when excluding sub-pillar T1 ...................... 19 

Table 13: Alternative assumptions considered in the uncertainty analysis of the CRI. ............................ 21 

Table 14: CRI rank and 90% interval of all countries from uncertainty analysis. ....................................... 22 

Table 15: CRI Index and Pillar difference in median rank and nominal rank of all countries in 
sensitivity analysis. ............................................................................................................................................................................ 24 

30 

 
 
Getting in touch with the EU 

In person 

All over the European Union there are hundreds of Europe Direct centres. You can find the 
address of the centre nearest you online (european-union.europa.eu/contact-eu/meet-us_en). 

On the phone or in writing 

Europe Direct is a service that answers your questions about the European Union. You can 
contact this service: 

 by freephone: 00 800 6 7 8 9 10 11 (certain operators may charge for these calls), 

 at the following standard number: +32 22999696, 

 via the following form: european-union.europa.eu/contact-eu/write-us_en. 

Finding information about the EU 

Online 

Information about the European Union in all the official languages of the EU is available on the 
Europa website (european-union.europa.eu). 

EU publications 

You can view or order EU publications at op.europa.eu/en/publications. Multiple copies of free 
publications can be obtained by contacting Europe Direct or your local documentation centre 
(european-union.europa.eu/contact-eu/meet-us_en). 

EU law and related documents 

For access to legal information from the EU, including all EU law since 1951 in all the official 
language versions, go to EUR-Lex (eur-lex.europa.eu). 

EU open data 

The portal data.europa.eu provides access to open datasets from the EU institutions, bodies 
and agencies. These can be downloaded and reused for free, for both commercial and non-
commercial purposes. The portal also provides access to a wealth of datasets from European 
countries. 

 
 
 
 
 
 
 
The Joint Research Centre (JRC) provides 
independent, evidence-based knowledge and 
science, supporting EU policies to positively 
impact society 

EU Science Hub 
Joint-research-centre.ec.europa.eu